1. The Llama fine-tuning libraries
Welcome to this course on fine-tuning with Llama. I'm Francesca, and I'll be your host as we learn how to fine-tune Llama models.
2. When to use fine-tuning
Fine-tuning trains an existing model on specialized data to improve accuracy, reduce bias, and expand its knowledge base for specific tasks.
3. How to use fine-tuning
The results depend on the quality of the data, the model's capacity, and how well-defined the task is. This means that, for the full fine-tuning process, we need a few key components:
4. How to use fine-tuning
a training dataset,
5. How to use fine-tuning
a model and tokenizer,
6. How to use fine-tuning
training arguments,
7. How to use fine-tuning
and a fine-tuning class to initiate the training process and produce a new version of the model. The resulting model is then evaluated to ensure it meets the desired performance criteria.
8. The Llama fine-tuning libraries
To help with this process, we have some useful fine-tuning libraries. We'll start with TorchTune, and we'll also briefly cover other options, each with different benefits. By the end of this video, we'll have an overview of the major libraries available, and an understanding of the basics of TorchTune, including launching our first fine-tuning task!
9. Options for Llama fine-tuning
The TorchTune library, based on configurable templates, is ideal for scaling quickly from experimentation to production.
Hugging Face's SFTTrainer provides access to other LLMs, making it suitable for fine-tuning multiple models.
Unsloth, with its efficient memory usage, is perfect for setups with limited hardware.
Axolotl's modular approach excels when fine-tuning without requiring extensive reconfiguration.
10. TorchTune and the recipes for fine-tuning
TorchTune's standout feature is its recipe-based system. Recipes in TorchTune are modular templates that define the entire fine-tuning process, and can be adapted to different projects.
We can think of them as cooking recipes for model fine-tuning. They give indications on the step-by-step process, but can easily be adjusted, say, if we want to substitute certain ingredients. This way, TorchTune recipes help keep code organized and ensure it is reproducible.
11. TorchTune list
Now, let's go over a basic example of configuring a fine-tuning loop using TorchTune.
TorchTune scripts can be run from a terminal, or command prompt, in an environment where Python and TorchTune are installed.
We can install TorchTune using pip install.
With TorchTune installed, we can now list available recipes with tune ls. If you're using an IPython shell, the command will have an exclamation mark in the front.
12. TorchTune list
The output lists all available recipes, and includes an additional column, "config", that lists the models compatible with each recipe, along with their configurations, such as version and memory use.
13. TorchTune run
From the list, we can select a recipe and the corresponding configuration file, and use tune run to start the fine-tuning loop.
This code initiates a fine-tuning loop with all parameters contained in the TorchTune recipe specified.
Note that, throughout this course, we'll specify the device=cpu and epochs=0 parameters to ensure we keep runtimes to a minimum. However, in most practical applications, we'd set "cuda" as the device, and epochs to a positive integer, such as 20, to allow the model to train over multiple passes through the data.
14. Let's practice!
Stay tuned!