Implementing random search
Hyperparameter search is a computationally costly approach to experiment with different hyperparameter values. However, it can lead to performance improvements. In this exercise, you will implement a random search algorithm.
You will randomly sample 10 values of the learning rate and momentum from the uniform distribution. To do so, you will use the np.random.uniform()
function.
numpy
package has already been imported as np
, and a plot_hyperparameter_search()
function has been created to visualize your results.
This exercise is part of the course
Introduction to Deep Learning with PyTorch
Exercise instructions
- Randomly sample a learning rate factor between
2
and4
so that the learning rate (lr
) is bounded between \(10^{-2}\) and \(10^{-4}\). - Randomly sample a momentum between 0.85 and 0.99.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
values = []
for idx in range(10):
# Randomly sample a learning rate factor between 2 and 4
factor = ____
lr = 10 ** -factor
# Randomly select a momentum between 0.85 and 0.99
momentum = ____
values.append((lr, momentum))
plot_hyperparameter_search(values)