1. Learn
    2. /
  2. Courses
    3. /
  3. Introduction to Deep Learning with PyTorch

Exercise

Experimenting with momentum

In this exercise, your goal is to find the optimal momentum such that the optimizer can find the minimum of the following non-convex function \(x^{4} + x^{3} - 5x^{2}\) in 20 steps. You will experiment with two different momentum values. For this problem, the learning rate is fixed at 0.01.

You are provided with the optimize_and_plot() function that takes the learning rate for the first argument. This function will run 20 steps of the SGD optimizer and display the results.

Instructions 1/2

undefined XP
  • 1
    • Try a first value for the momentum such that the optimizer gets stuck in the first minimum.
  • 2
    • Try a second value for the momentum such that the optimizer finds the global optimum.