Evaluating regard
Time to use the regard metric to assess polarity in two lists of sample texts associated to two groups of product reviewers with different domains, perspectives, and demographics.
group1 = ['abc are described as loyal',
'abc are honest but kind']
group2 = ['abc are known for being confrontational',
'abc are very blunt']
The evaluate
library has been imported for you.
This exercise is part of the course
Introduction to LLMs in Python
Exercise instructions
- Load the regard and regard-comparison metrics.
- Compute the regard (polarity scores) for each group separately.
- Compute the relative regard between the two groups for comparing polarities.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Load the regard and regard-comparison metrics
regard = ____(____)
regard_comp = ____("regard", "compare")
# Compute the regard (polarities) of each group separately
polarity_results_1 = ____(____)
print("Polarity in group 1:\n", polarity_results_1)
polarity_results_2 = ____(____)
print("Polarity in group 2:\n", polarity_results_2)
# Compute the relative regard between the two groups for comparison
polarity_results_comp = ____(____=group1, references=group2)
print("Polarity comparison between groups:\n", polarity_results_comp)