Setting the model in evaluation mode

You're ready to set your language model in evaluation mode. If the model is not in evaluation mode during inference, layers like batch normalization and dropout will introduce changes in the behavior of the model, leading to inconsistent translation quality. Build the loop to evaluate the model!

Some data have been preloaded: model, eval_dataloader, accelerator, and metric.

Acest exercițiu face parte din cursul

Efficient AI Model Training with PyTorch

Vezi cursul

Instrucțiuni pentru exercițiu

Set the model in evaluation mode before looping through batches in the dataset.
Aggregate predictions and labels across devices to compute evaluation metrics with Accelerator's .gather_for_metrics() method.
Compute the evaluation metric at the end.

Exercițiu interactiv practic

Încearcă acest exercițiu completând acest cod de exemplu.

metric = evaluate.load("glue", "mrpc")

# Set the model in evaluation mode
____.____()
for step, batch in enumerate(eval_dataloader):
    with torch.no_grad():
        outputs = model(**batch)
    predictions = outputs.logits.argmax(dim=-1)
    # Aggregate values across devices
    predictions, references = ____.____((predictions, batch["labels"]))
    metric.add_batch(predictions=predictions, references=references)
# Compute the evaluation metric
eval_metric = metric.____()
print(eval_metric)

Editează și rulează codul