1. Learn
    2. /
  2. Courses
    3. /
  3. Reinforcement Learning from Human Feedback (RLHF)
Connected

Exercise

Initialize the PPO trainer

You are working for a customer service company that uses a chatbot to handle customer inquiries. The chatbot provides helpful responses, but you recently received feedback that they lack depth. You need to fine-tune the model behind the chatbot, and you start with creating a PPO trainer instance.

The dataset_cs has already been loaded.

Instructions

100 XP
  • Initialize the PPO configuration with the model name "gpt2" and a learning rate of 1.2e-5.
  • Load AutoModelForCausalLMWithValueHead, the causal language model with a value head.
  • Create the PPOTrainer() using the model, configuration, and tokenizer just defined, and with the preloaded dataset.