Get startedGet started for free

Comparing Grid and Random Search

1. Comparing Grid and Random Search

In the previous lessons we looked at both Grid and Random search in depth. Before we move on to more advanced methods, it is a good idea to stop and reinforce our knowledge on these two valuable techniques.

2. What's the same?

There are a number of similarities for these techniques. Both are automated ways of tuning different hyperparameters. For both you set the grid to sample from (which hyperparameters and values for each). For both you need to think carefully about which hyperparameters and values to sample from as your model will only be as good as the grid you set! And for both you set a cross-validation scheme and scoring function.

3. What's different?

There are also some key differences between these two techniques. Grid search exhaustively tries all the combinations in the grid or sample space and therefore has no sampling methodology. Random search tries a subset of combinations randomly and you can additionally define how to sample. Remember the default in Scikit Learn is to have every combination equally likely to be chosen, also known as uniform distribution. Because of this, grid search is very computationally expensive compared to random search But Grid Search is guaranteed to find the best score in the sample space whereas random search is not. Though random search has a higher chance to find a good score faster.

4. Which should I use?

So which technique should you use? As we often say in data science - 'It depends'. However there are some things you can consider. Firstly, the more data you have, the stronger the argument for random search. More hyperparameters and values to try also means random search might be a better option. Additionally if you don't have a lot of time or computing power, random search will be more economical. Remember, random search has more chance of getting a good result faster, even if not the absolute best.

5. Let's practice!

Let's do some exercises to illustrate the differences between grid and random search visually and conceptually.