1. Learn
    2. /
  2. Courses
    3. /
  3. Introduction to LLMs in Python
Connected

Exercise

Using evaluate metrics

It's time to evaluate your LLM that classifies customer support interactions. Picking up from where you left your fine-tuned model, you'll now use a new validation dataset to assess the performance of your model.

Some interactions and their corresponding labels have been loaded for you as validate_text and validate_labels. The model and tokenizer are also loaded.

Instructions

100 XP
  • Extract the predicted labels from the model logits found in the outputs.
  • Compute the four loaded metrics by comparing real (validate_labels) and predicted labels.