Tuning hyperparameters

1. Tuning hyperparameters

Welcome back!

2. Hyperparameters

As we discussed earlier in the course, there are several knobs that you can turn to influence the shape and complexity of your trees. Some knobs result in larger or more complex trees, while others force trees to stay small and shallow. These knobs are called "hyperparameters". They control a model's complexity and need to be specified before model training. For decision trees, we have the following three parameters: min_n, which is the minimum number of samples required to split a node, tree_depth, which stands for the maximum depth of a tree, and cost_complexity, which is a penalty number that punishes the size of the tree.

3. Why tuning?

When specifying none of these hyperparameters, parsnip chooses the following default values for you: minimum samples per node of 20, maximum depth of 30, and cost_complexity of 0-point-01. These are chosen so that they work well in many cases. However, they may not be the best choice for all datasets. The goal of tuning is now to find the set of hyperparameters that gives your model the best possible performance.

4. Tuning with tidymodels using the tune package

Tidymodels has a toolbox containing functions that make tuning hyperparameters real easy: The tune package. The first step is to create a set of hyperparameter values, called a grid, that is the pool in which we search for the best candidate.

5. Tuning with tidymodels

The second step is to create a dummy model specification in which we label hyperparameters for tuning.

6. Tuning with tidymodels

The third step is to build and evaluate one model for every hyperparameter combination at every grid point.

7. Tuning with tidymodels

The fourth step is to select the best performing hyperparameters from all results and plug these into the dummy specification. This way, we get a final specification with the optimal performing hyperparameters.

8. Step 1: Create placeholders: tune()

Let's get to coding. To label hyperparameters for tuning, set them equal to tune() in the model specification. It will let other functions know that they need to be optimized. In the printed model specification, the labeled hyperparameters appear under "Main Arguments".

9. Step 2: Create a tuning grid: grid_regular()

Tidymodels provides different ways for creating grids. One is the grid_regular() function, which creates an evenly-spaced grid. We use the parameters() function on our specification which extracts the parameters to be tuned and the levels argument to set the number of levels for each parameter to three. We get a tibble with two columns, one for each parameter, and, three times three, nine rows or parameter combinations.

10. Step 3: Tune the grid: tune_grid()

The tune_grid() function performs the hyperparameter tuning. It fits a model for every grid point and evaluates the model using cross-validation and a provided metric. It takes the following arguments: a dummy model specification with labeled parameters for tuning, a model formula, a cross-validation object, resamples, a tuning grid, grid, and a metrics function.

11. Step 3: Tune the grid: tune_grid()

The result can be visualized using the autoplot() function. This plot shows a declining accuracy with higher cost-complexity parameter, that means simpler trees perform worse.

12. Step 4: Use the best parameters: finalize_model()

The select_best() function is used to select the best performing hyperparameters. We pass our tune_results object as an argument. This function returns a tibble with the hyperparameter values that produced the largest average performance metric value. From our tuning grid, the best performing model was number 4 and had a min_n of 2 and a tree_depth of 8. One last step is needed to finalize the tuning: Once you found the best hyperparameters, you need to plug them into the specification. This is what the finalize_model() function is for. Pass the dummy specification to the finalize_model() function and provide the results of the select_best() function as an argument. In this case, we pass spec_untuned to finalize_model() and provide the final_params tibble. This returns an updated specification with the model hyperparameter values set to the optimal values found during the tuning process.

13. Let's tune!

Now it's your turn to tune your models.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.