Get startedGet started for free

Selecting the best precision model

Your boss has offered to pay for you to see three sports games this year. Of the 41 home games your favorite team plays, you want to ensure you go to three home games that they will definitely win. You build a model to decide which games your team will win.

To do this, you will build a random search algorithm and focus on model precision (to ensure your team wins). You also want to keep track of your best model and best parameters, so that you can use them again next year (if the model does well, of course). You have already decided on using the random forest classification model rfc and generated a parameter distribution param_dist.

This exercise is part of the course

Model Validation in Python

View Course

Exercise instructions

  • Create a precision scorer, precision using make_scorer(<scoring_function>).
  • Complete the random search method by using rfc and param_dist.
  • Use rs.cv_results_ to print the mean test scores.
  • Print the best overall score.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

from sklearn.metrics import precision_score, make_scorer

# Create a precision scorer
precision = ____(____)
# Finalize the random search
rs = RandomizedSearchCV(
  estimator=____, param_distributions=____,
  scoring = precision,
  cv=5, n_iter=10, random_state=1111)
rs.fit(X, y)

# print the mean test scores:
print('The accuracy for each run was: {}.'.format(rs.cv_results_['____']))
# print the best model score:
print('The best accuracy for a single model was: {}'.format(rs.____))
Edit and Run Code