Session Ready
Exercise

Visualizing Coarse to Fine

You're going to undertake the first part of a Coarse to Fine search. This involves analyzing the results of an initial random search that took place over a large search space, then deciding what would be the next logical step to make your hyperparameter search finer.

You have available:

  • combinations_list - a list of the possible hyperparameter combinations the random search was undertaken on.
  • results_df - a DataFrame that has each hyperparameter combination and the resulting accuracy of all 500 trials. Each hyperparameter is a column, with the header the hyperparameter name.
  • visualize_hyperparameter() - a function that takes in a column of the DataFrame (as a string) and produces a scatter plot of this column's values compared to the accuracy scores. An example call of the function would be visualize_hyperparameter('accuracy')
Instructions
100 XP
  • Confirm (by printing out) the size of the combinations_list, justifying the need to start with a random search.
  • Sort the results_df by accuracy values and print the top 10 rows. Are there clear insights? Beware a small sample size!
  • Confirm (by printing out) which hyperparameters were used in this search. These are the column names in results_df.
  • Call visualize_hyperparameter() with each hyperparameter in turn (max_depth, min_samples_leaf, learn_rate). Are there any trends?