What happens in the GPU during training and inference?

View the content before continuing.

Deze oefening maakt deel uit van de cursus

Google DeepMind: Accelerate Your Model

Cursus bekijken

Praktische interactieve oefening

Zet theorie om in actie met een van onze interactieve oefeningen.

Begin met trainen

Google DeepMind: Accelerate Your Model

SkillTag.level.intermediateSkillTag.label

5.0+

2 reviews

In this module, you will learn about specialized hardware for training neural network models, called Graphical Processing Units (GPUs). You will explore the tradeoff between model efficiency, that is, how fast a model can be trained and make predictions, and performance, that is, how well a model can solve a task. You will see that models with more parameters generally work better but can are also be slower and require more computer memory. You will also map the stakeholders affected by the potential environmental impacts of AI such as energy use, water consumption, and e-waste in order to see how different groups experience both risks and potential benefits. This exercise will help you understand why environmental justice in AI requires considering diverse perspectives, from local communities to developers, policymakers, and future generations.

Exercise 1: Efficiency versus performance Exercise 2: Maximize a GPU Exercise 3: Learning objectives Exercise 4: How to get the most out of this course Exercise 5: Lab: Compare Models of Different Sizes Exercise 6: The GPU architecture Exercise 7: Environmental impacts of LLMs Exercise 8: Stakeholder map of environmental impacts Exercise 9: Quiz 1 - Question 1 Exercise 10: Quiz 1 - Question 2

In this module, you will discover which computations are performed when you train a model on a GPU and which computations are performed when you perform inference. You will learn how computers represent numbers and how changing the number representation affects computations and computer memory requirements. You will explore techniques for reducing the computational effort without any or very little reduction in model performance. You will also explore your own role as a developer by mapping your carbon footprint, the positive impact you can make through choices that reduce energy use and resource consumption in the AI pipeline. This reflection will help you see how everyday technical decisions, like model size or training location, connect directly to broader goals of sustainability and environmental justice.

Exercise 1: What happens in the GPU during training and inference?

Huidige oefening

Exercise 2: Lab: Estimate Training FLOPs Exercise 3: Smart numbers Exercise 4: Techniques for reducing computation effort Exercise 5: Mapping your carbon footprint Exercise 6: Quiz 2 - Question 1 Exercise 7: Quiz 2 - Question 2

In this module, you will explore the details around memory when training models and performing inference on a GPU. You will learn how to estimate how much computer memory you need for training a specific model. You will then experiment with and apply methods for decreasing memory requirements, such as representing numbers as bfloat16.

Exercise 1: Lab: Hitting a Wall Exercise 2: What needs to be stored in the GPU?Exercise 3: Lab: Estimate GPU Memory Exercise 4: Lab: Fine-Tune a Model with bfloat16 Exercise 5: Quiz 3 - Question 1 Exercise 6: Quiz 3 - Question 2

In this module, you will gain an overview of more advanced techniques for reducing memory requirements. You will learn about gradient accumulation and how this can be used as an alternative to larger batch sizes, and you will apply this technique for fine-tuning a model with 4 billion parameters.

Exercise 1: Gradient accumulation Exercise 2: Lab: Apply Gradient Accumulation Exercise 3: Advanced scaling techniques Exercise 4: Quiz 4 - Question 1 Exercise 5: Quiz 4 - Question 2

In this module, you will reflect on how AI’s energy demands intersect with questions of access, equity, and environmental justice in Africa, weighing both potential benefits and risks. In the challenge activity, you will build on this reflection by designing a sustainability plan for your own LLM project, ensuring it aligns with principles of fairness, accountability, and energy justice.

Exercise 1: AI and environmental justice Exercise 2: Design a sustainability plan for your LLM project Exercise 3: Quiz 5 - Question 1 Exercise 4: Quiz 5 - Question 2

In this module, you will have the opportunity to consult additional resources and further reading to investigate the topics you have covered in more detail. Finally, you will consider your next steps and how you can build on what you have learned in the course.

Exercise 1: Summary Exercise 2: Looking forward Exercise 3: Additional resources and further reading Exercise 4: Glossary Exercise 5: Feedback