1. Learn
    2. /
  2. Courses
    3. /
  3. Deep Learning for Text with PyTorch

Exercise

Evaluating pretrained text generation model

The PyBooks team has used a pre-trained GPT-2 model that you experimented to generate a text based on a given prompt. Now, they want to evaluate the quality of this generated text. To achieve this, they have tasked you to evaluate generated text using a reference text.

BLEUScore, ROUGEScore have been loaded for you.

Instructions

100 XP
  • Begin by initializing the two metrics (BLEU and ROUGE) provided from torchmetrics.text.
  • Use these initialized metrics to calculate the scores between the generated text and the reference text.
  • Display the calculated BLEU and ROUGE scores.