1. Learn
    2. /
  2. Courses
    3. /
  3. Fine-Tuning with Llama 3
Connected

Exercise

Setting up Llama training arguments

You are tasked with working with the Llama model used in a customer service chatbot by fine-tuning it on customer service data purpose built for question-answering. To ensure the best performance out of these models, your team will fine-tune a Llama model for this task using the bitext dataset.

You want to do a test run of the training loop to check if the training script works. So, you want to start by setting a small learning rate and limit the training to a handful of steps in your training arguments.

Instructions

100 XP
  • Import and instantiate the helper class to store your training arguments.
  • Set the training argument for learning rate to a value of 2e-3.