1. Learn
    2. /
  2. Courses
    3. /
  3. Introduction to Deep Learning with PyTorch

Exercise

Updating the weights manually

Now that you know how to access weights and biases, you will manually perform the job of the PyTorch optimizer. PyTorch functions can do what you're about to do, but it's helpful to do the work manually at least once, to understand what's going on under the hood.

A neural network of three layers has been created and stored as the model variable. This network has been used for a forward pass and the loss and its derivatives have been calculated. A default learning rate, lr, has been chosen to scale the gradients when performing the update.

Instructions 1/2

undefined XP
    1
    2
  • Create the gradient variables by accessing the local gradients of each weight tensor.