Which of the following are limitations of supervised fine-tuning?
本练习是课程的一部分
通过我们的互动练习之一，将理论转化为实践
In this module, you will explore the motivation for fine-tuning. Even when a pre-trained large language model is available, it may not always do exactly what you want it to do. Here, you will investigate the capabilities and limitations of a pre-trained language model to better understand why it is necessary to fine-tune models to new tasks.
In this module, you will appreciate the power and importance of formatting for fine-tuning large language models. You will explore the various formats required to achieve different tasks. You will pre-process a dataset derived from the Africa Galore dataset and transform it into a question-and-answer format. This will then be used in later modules to fine-tune your language model so that it can generate revision study flashcards—a task which it was not pre-trained to do.
In this module, you will learn full-parameter fine-tuning: a straightforward method for adapting pre-trained models. To gain an understanding, you will first take the small language model you built in course 04 Discover The Transformer Architecture. Then you will continue its training on a small, specialized dataset to generate the revision study flashcards you created in the previous module. This process will allow you to compare fine-tuning to training from scratch, observing the key similarities and differences in the development pipeline. Here, you will also consider how AI is made sense of within cultural contexts by reading a story about AI and then creating your own short piece of fiction. The aim here is to explore how narrative can act as a complementary approach to anticipation and reflection for revealing cultural meaning and values.
In this module, you will explore low-rank adaptation (LoRA), a more computationally efficient alternative to full-parameter fine-tuning. LoRA is a popular parameter-efficient fine-tuning (PEFT) technique. You will investigate LoRA by applying it to fine-tune the Gemma3-1B model, which has one billion parameters. This will enable you to experience first-hand how it is able to achieve excellent results with a fraction of the computational cost of full-parameter fine-tuning.
In this module, you will consider the limitations of supervised fine-tuning. You will then be given a brief overview of advanced techniques based on reinforcement learning (RL). This will introduce you to how these approaches can better align a model's behavior with human values and preferences.
当前练习
In this module, you will explore foresight and governance. You will consider how the values and meanings revealed through storytelling can inform foresight reporting and help you to design governance responses, including enforceable rules, transparency, and accountability. This equips you to think about how strong governance can protect communities, ensure equity, and align AI with societal values.
Critically evaluate and design governance approaches that include enforcement, bright-line rules, and burden of proof.
