Self-consistency to assess confidence

The prompt below aims to ask ChatGPT to classify the sentiment of a review. You have to estimate the confidence of ChatGPT in each class using self-consistency.

The get_response() function to communicate with ChatGPT is pre-loaded for you.

This exercise is part of the course

Explainable AI in Python

View Course

Exercise instructions

For each review, append the sentiment to the responses list.
Compute the confidence for each sentiment.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

prompt = """Classify the following review as positive or negative. 
You should reply with either "positive" or "negative", nothing else.
Review: 'The coffee was excellent, but the service left much to be desired.'"""

responses = []
for i in range(5):
    sentiment = get_response(prompt)
    # Append the response to the responses list
    ____

# Compute the confidence for each sentiment
confidence = {
    'positive': ____,
    'negative': ____
}

print(confidence)

Edit and Run Code