Generalization and the bias-variance trade-off

View the content before continuing.

Cet exercice fait partie du cours

<cours>Google DeepMind: Design And Train Neural Networks</cours>

Voir le cours

Exercice interactif pratique

Transformez la théorie en action avec l’un de nos exercices interactifs

Commencer l’exercice

Cet exercice fait partie du cours

<cours>Google DeepMind: Design And Train Neural Networks</cours>

IntermédiaireNiveau de compétence

4.9+

15 reviews

Commencer le cours gratuitement

In this module, you will explore the concept of generalization. You will begin by considering the concepts of signal and noise, and the role they play in model performance. You will then investigate how these concepts relate to overfitting and underfitting. Finally, you will also consider the course learning objectives and how to learn most effectively on this course.

Exercise 1: The deep learning revolution Exercise 2: Signal and noise Exercise 3: Lab: Distinguish Between Signal and Noise Exercise 4: Learning objectives Exercise 5: How to get the most out of this course

In this module, you will be formally introduced to generalization and the trade-off between bias and variance. You will explore training and test splits and the use of loss curves as a diagnostic tool. You will then move on to explore anticipation as a key research skill through a research case study. While you cannot predict every impact your technologies may have, lessons from places where the risks of AI have already been revealed can help you anticipate how similar challenges might emerge in your region.

Exercise 1: Generalization and the bias-variance trade-off

Exercice actuel

Exercise 2: Training and test splits Exercise 3: Predicting cyclones with AI weather models Exercise 4: Anticipating risks Exercise 5: Quiz 1 - Question 1 Exercise 6: Quiz 1 - Question 2

In this module, you will investigate the anatomy of neural networks, starting with single-neuron neural networks before moving on to the multilayer perceptron (MLP). You will learn why neural networks consist of multiple layers and how this allows them to do complex classification problems. Finally, you will implement an MLP from scratch and see how it can be applied to the task of predicting the next token.

Exercise 1: The multilayer perceptron (MLP)Exercise 2: Lab: Make Predictions with a Single-Layer Neural Network Exercise 3: Lab: Separate More Complex Data Exercise 4: Modeling data that is not linearly-separable Exercise 5: Lab: Design Your Own MLP Exercise 6: Quiz 2 - Question 1 Exercise 7: Quiz 2 - Question 2

In this module, you will learn about different settings of the training process called hyperparameters. You will undertake hyperparameter tuning and explore methods to mitigate overfitting including the use of a validation dataset. Moreover, you will consider alignment and safety to ensure that AI models act in ways consistent with human values, ethical principles, and intended goals. This will help you to reduce the risks of undesirable, unintended, or biased outcomes and make AI more trustworthy and beneficial for society.

Exercise 1: Lab: Tune Hyperparameters Exercise 2: Methods to mitigate overfitting Exercise 3: Lab: Mitigate Overfitting Exercise 4: Why is a validation dataset needed?Exercise 5: Alignment and safety Exercise 6: Quiz 3 - Question 1 Exercise 7: Quiz 3 - Question 2

In this module, you will discover how gradients minimize error when training artificial neural networks. You will investigate gradient descent and compute the gradient using the chain rule. You will then explore how automatic differentiation works in the deep learning library JAX by applying gradient updates to a single-layer neural network model.

Exercise 1: Gradients Exercise 2: Lab: Gradients Exercise 3: Quiz 4 - Question 1 Exercise 4: Quiz 4 - Question 2

In this module, you will examine a fundamental concept of training neural networks - backpropagation. You will then move on to consider stochastic gradient descent and more sophisticated methods for training neural networks, such as Adam (Adaptive Moment Estimation). Finally, you will train a neural network using optimizers implemented in the Python deep learning framework Keras.

Exercise 1: Backpropagation Exercise 2: Stochastic gradient descent (SGD)Exercise 3: Lab: Train Your Model with Keras Exercise 4: Quiz 5 - Question 1 Exercise 5: Quiz 5 - Question 2

In this module, you will focus on the responsible innovation principle of anticipation. Anticipation is the forward-thinking practice of exploring the potential impacts of new technologies. It does not attempt to predict the future, but rather to understand a range of plausible outcomes. You will draw on this skill to create an impact statement card that involves ethically reflecting on your work to increase the benefits of your project and reduce any potential risks.

Exercise 1: Anticipating social impacts Exercise 2: Create an impact statement card Exercise 3: Quiz 6 - Question 1 Exercise 4: Quiz 6 - Question 2

In this module, you will have the opportunity to consult additional resources and further reading to investigate the topics you have covered in more detail. Finally, you will consider your next steps and how you can build on what you have learned in the course.

Exercise 1: Summary Exercise 2: Additional resources and further reading Exercise 3: Looking forward Exercise 4: Glossary Exercise 5: Feedback