1. Uczyć się
    2. /
  2. Courses
    3. /
  3. Reinforcement Learning from Human Feedback (RLHF)
Connected

Exercise

Mitigating negative KL divergence

You were fine-tuning the model using RLHF techniques and noticed that the model's performance has worsened compared to the base model. You suspect this is due to negative KL divergence, so you want to set the correct generation parameters to prevent this issue.

The tokenizer has been pre-imported.

Instrukcje

100 XP
  • Set top_k and min_length to values that help avoid KL divergence.