Exercise

Updating the weights manually

Now that you know how to access weights and biases, you will manually perform the job of the PyTorch optimizer. PyTorch functions can do what you're about to do, but it's helpful to do the word manually at least once, to understand what's going on under the good.

A neural network of three layers has been created and stored as the model variable. This network has been used for a forward pass and the loss and its derivatives have been calculated. A default learning rate, lr, has been chosen to scale the gradients when performing the update.

Instructions 1/2

undefined XP

1

2

Create the gradient variables by accessing the local gradients of each weight tensor.