Quiz 5 - Question 1

What is the primary trade-off when using stochastic gradient descent (SGD) instead of batch gradient descent?

This exercise is part of the course

Google DeepMind: Design And Train Neural Networks

View Course

Hands-on interactive exercise

Turn theory into action with one of our interactive exercises

Start Exercise

Google DeepMind: Design And Train Neural Networks

IntermediateSkill Level

5.0+

3 reviews

In this module, you will explore the concept of generalization. You will begin by considering the concepts of signal and noise, and the role they play in model performance. You will then investigate how these concepts relate to overfitting and underfitting. Finally, you will also consider the course learning objectives and how to learn most effectively on this course.

Exercise 1: The deep learning revolution Exercise 2: Signal and noise Exercise 3: Lab: Distinguish Between Signal and Noise Exercise 4: Learning objectives Exercise 5: How to get the most out of this course

In this module, you will be formally introduced to generalization and the trade-off between bias and variance. You will explore training and test splits and the use of loss curves as a diagnostic tool. You will then move on to explore anticipation as a key research skill through a research case study. While you cannot predict every impact your technologies may have, lessons from places where the risks of AI have already been revealed can help you anticipate how similar challenges might emerge in your region.

Exercise 1: Generalization and the bias-variance trade-off Exercise 2: Training and test splits Exercise 3: Predicting cyclones with AI weather models Exercise 4: Anticipating risks Exercise 5: Quiz 1 - Question 1 Exercise 6: Quiz 1 - Question 2

In this module, you will learn about different settings of the training process called hyperparameters. You will undertake hyperparameter tuning and explore methods to mitigate overfitting including the use of a validation dataset. Moreover, you will consider alignment and safety to ensure that AI models act in ways consistent with human values, ethical principles, and intended goals. This will help you to reduce the risks of undesirable, unintended, or biased outcomes and make AI more trustworthy and beneficial for society.

Exercise 1: Lab: Tune Hyperparameters Exercise 2: Methods to mitigate overfitting Exercise 3: Lab: Mitigate Overfitting Exercise 4: Why is a validation dataset needed?Exercise 5: Alignment and safety Exercise 6: Quiz 3 - Question 1 Exercise 7: Quiz 3 - Question 2

In this module, you will discover how gradients minimize error when training artificial neural networks. You will investigate gradient descent and compute the gradient using the chain rule. You will then explore how automatic differentiation works in the deep learning library JAX by applying gradient updates to a single-layer neural network model.

Exercise 1: Gradients Exercise 2: Lab: Gradients Exercise 3: Quiz 4 - Question 1 Exercise 4: Quiz 4 - Question 2

In this module, you will examine a fundamental concept of training neural networks - backpropagation. You will then move on to consider stochastic gradient descent and more sophisticated methods for training neural networks, such as Adam (Adaptive Moment Estimation). Finally, you will train a neural network using optimizers implemented in the Python deep learning framework Keras.

Exercise 1: Backpropagation Exercise 2: Stochastic gradient descent (SGD)Exercise 3: Lab: Train Your Model with Keras Exercise 4: Quiz 5 - Question 1

Current Exercise

Exercise 5: Quiz 5 - Question 2

In this module, you will focus on the responsible innovation principle of anticipation. Anticipation is the forward-thinking practice of exploring the potential impacts of new technologies. It does not attempt to predict the future, but rather to understand a range of plausible outcomes. You will draw on this skill to create an impact statement card that involves ethically reflecting on your work to increase the benefits of your project and reduce any potential risks.

Exercise 1: Anticipating social impacts Exercise 2: Create an impact statement card Exercise 3: Quiz 6 - Question 1 Exercise 4: Quiz 6 - Question 2

In this module, you will have the opportunity to consult additional resources and further reading to investigate the topics you have covered in more detail. Finally, you will consider your next steps and how you can build on what you have learned in the course.

Exercise 1: Summary Exercise 2: Additional resources and further reading Exercise 3: Looking forward Exercise 4: Glossary Exercise 5: Feedback