Visualizing Coarse to Fine
You're going to undertake the first part of a Coarse to Fine search. This involves analyzing the results of an initial random search that took place over a large search space, then deciding what would be the next logical step to make your hyperparameter search finer.
You have available:
combinations_list
- a list of the possible hyperparameter combinations the random search was undertaken on.results_df
- a DataFrame that has each hyperparameter combination and the resulting accuracy of all 500 trials. Each hyperparameter is a column, with the header the hyperparameter name.visualize_hyperparameter()
- a function that takes in a column of the DataFrame (as a string) and produces a scatter plot of this column's values compared to the accuracy scores. An example call of the function would bevisualize_hyperparameter('accuracy')
If you wish to view the visualize_hyperparameter()
function definition, you can run this code:
import inspect
print(inspect.getsource(visualize_hyperparameter))
This exercise is part of the course
Hyperparameter Tuning in Python
Exercise instructions
- Confirm (by printing out) the size of the
combinations_list
, justifying the need to start with a random search. - Sort the
results_df
by accuracy values and print the top 10 rows. Are there clear insights? Beware a small sample size! - Confirm (by printing out) which hyperparameters were used in this search. These are the column names in
results_df
. - Call
visualize_hyperparameter()
with each hyperparameter in turn (max_depth
,min_samples_leaf
,learn_rate
). Are there any trends?
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Confirm the size of the combinations_list
print(____(____))
# Sort the results_df by accuracy and print the top 10 rows
print(results_df.____(by=____, ascending=False).head(____))
# Confirm which hyperparameters were used in this search
print(results_df.____)
# Call visualize_hyperparameter() with each hyperparameter in turn
visualize_hyperparameter(____)
visualize_hyperparameter(____)
visualize_hyperparameter(____)