Grid search

Recall that we've created a baseline Gradient Boosting model in the previous lesson. Your goal now is to find the best max_depth hyperparameter value for this Gradient Boosting model. This hyperparameter limits the number of nodes in each individual tree. You will be using K-fold cross-validation to measure the local performance of the model for each hyperparameter value.

You're given a function get_cv_score(), which takes the train dataset and dictionary of the model parameters as arguments and returns the overall validation RMSE score over 3-fold cross-validation.

This exercise is part of the course

Winning a Kaggle Competition in Python

View Course

Exercise instructions

Specify the grid for possible max_depth values with 3, 6, 9, 12 and 15.
Pass each hyperparameter candidate in the grid to the model params dictionary.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Possible max depth values
max_depth_grid = [____]
results = {}

# For each value in the grid
for max_depth_candidate in max_depth_grid:
    # Specify parameters for the model
    params = {'max_depth': ____}

    # Calculate validation score for a particular hyperparameter
    validation_score = get_cv_score(train, params)

    # Save the results for each max depth value
    results[max_depth_candidate] = validation_score   
print(results)

Edit and Run Code