1. Model training
Welcome back. Let's continue our discussion of model development by learning how generative AI models are trained.
2. Pick your mode of train-sportation
Training is like traveling to a distant city. It depends on three components:
First, hardware is our mode of transportation, determining our speed. Training on a personal laptop is like walking, using local GPUs is driving a car, while a server farm of TPUs is a jet plane.
Second, time is our travel distance. As our dataset size, model design complexity, and rounds of training increase, the longer metaphorical distance we travel.
Finally, cost is like the cost of transportation. Walking is free, but we can travel faster for a cost. We might be able to walk someplace nearby but will want a car or plane for long distances.
3. Graduate to advanced techniques
Training a general, or foundation, generative AI model is often just a first step. Advanced training techniques are then applied to make that pre-trained model specialized for specific contexts.
It's like when students graduate and join the workforce. They have foundational knowledge but need experience in their trade before they can be productive.
We'll discuss three advanced techniques: transfer learning and fine-tuning, RLHF, and custom embeddings.
Let's take a look at each in more detail.
4. From cats to lions
Transfer learning takes a pre-trained model with knowledge of one task and teaches it a new, related task. Since the tasks are related, the model leverages, or transfers, its knowledge to learn the new task.
Fine-tuning is a transfer learning technique that teaches a pre-trained model a new dataset. Since the model already has relevant knowledge, it doesn't need to start from scratch, saving both time and compute resources.
For example, we can fine-tune this house cat image generator on lion images. Since our model already has relevant knowledge about feline features, it quickly learns to generate lions.
5. Where does your feedback go?
Next, let's revisit RLHF, which we learned about in the last chapter.
Often when we use generative AI tools, we see options to give feedback about responses.
In this image, we can rate this AI chatbot with thumbs up or thumbs down. But what use is one piece of feedback?
6. Thumbs up for better responses
In fact, we're not the only ones giving feedback. The model developers are collecting many users' feedback. In RLHF, they use this information to train the chatbot model further.
A thumbs-up tells the model to generate similar responses for similar prompts.
A thumbs-down does the opposite.
After the model is trained like this, it is better able to generate content more aligned with the feedback.
7. Embeddings as fingerprints
The last advanced training technique we'll cover is called embedding.
Embedding embeds a concept inside a model's structure, almost like we're teaching it to recognize someone's fingerprint.
Just as fingerprints are unique patterns that can identify an individual, embeddings are unique representations of data entities inside a model. Data entities might be a word, a physical object, or a person.
Embeddings capture the meaning, context, and relationships of data by transforming them into a more compact form. This compact form helps the model process and understand that data more efficiently.
8. Embedding Dan
For example, a headshot generator AI would know about smiles, noses, and haircuts, but suppose we want a portrait of a specific person, me.
With this advanced training technique, we can embed an understanding of what my headshots look like.
9. Embedded Dan
This allows the model to create Cyberpunk Dan, Metalhead Dan, and oil painting Dan.
And this can be done with other data types, too, not just images.
10. Let's practice!
Time to practice!