Exercise

Evaluating pretrained text generation model

The PyBooks team has used a pre-trained GPT-2 model that you experimented to generate a text based on a given prompt. Now, they want to evaluate the quality of this generated text. To achieve this, they have tasked you to evaluate generated text using a reference text.

BLEUScore, ROUGEScore have been loaded for you.

Instructions

100 XP

Begin by initializing the two metrics (BLEU and ROUGE) provided from torchmetrics.text.
Use these initialized metrics to calculate the scores between the generated text and the reference text.
Display the calculated BLEU and ROUGE scores.

.css-6su6fj{-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;}Exercise

Instructions

Exercise