1. Learn
    2. /
  2. Courses
    3. /
  3. Introduction to Deep Learning with PyTorch

Exercise

Experimenting with learning rate

In this exercise, your goal is to find the optimal learning rate such that the optimizer can find the minimum of the non-convex function \(x^{4} + x^{3} - 5x^{2}\) in ten steps.

You will experiment with three different learning rate values. For this problem, try learning rate values between 0.001 to 0.1.

You are provided with the optimize_and_plot() function that takes the learning rate for the first argument. This function will run 10 steps of the SGD optimizer and display the results.

Instructions 1/3

undefined XP
  • 1
    • Try a small learning rate value such that the optimizer isn't able to get past the first minimum on the right.
  • 2
    • Try a large learning rate value such that the optimizer skips past the global minimum at -2.
  • 3
    • Based on the previous results, try a better learning rate value.