Evaluating perplexity
Try your had at generating text and evaluating the perplexity score.
You've been provided some input_text
that is the start of a sentence: "Current trends show that by 2030 ".
Use an LLM to generate the rest of the sentence.
An AutoModelForCausalLM
model and its tokenizer have been loaded for you as model
and tokenizer
variables.
Este exercício faz parte do curso
Introduction to LLMs in Python
Instruções de exercício
- Encode the
input_text
and pass it to the provided text generation model. - Load and compute the
mean_perplexity
score on the generated text.
Exercício interativo prático
Experimente este exercício preenchendo este código de exemplo.
# Encode the input text, generate and decode it
input_text_ids = ____(input_text, return_tensors="pt")
output = ____(input_text_ids, max_length=20)
generated_text = ____(output[0], skip_special_tokens=True)
print("Generated Text: ", generated_text)
# Load and compute the perplexity score
perplexity = ____("perplexity", module_type="metric")
results = ____(model_id="gpt2", predictions=____)
print("Perplexity: ", results['mean_perplexity'])