Challenge the champion
Having pushed your random forest to production, you suddenly worry that a naive Bayes classifier might be better. You want to run a champion-challenger test, by comparing a naive Bayes, acting as the challenger, to exactly the model which is currently in production, which you will load from file to make sure there is no confusion. You will use the F1 score for assessment. You have the data X_train
, X_test
, y_train
and y_test
available as before and GaussianNB()
, f1_score()
and pickle()
.
This exercise is part of the course
Designing Machine Learning Workflows in Python
Exercise instructions
- Load the existing model from memory using
pickle
. - Fit a Gaussian Naive Bayes classifier to the training data.
- Print the F1 score of the champion and then the challenger on the test data.
- Overwrite the current model to disk with the one that performed best.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Load the current model from disk
champion = pickle.____(open('model.pkl', ____))
# Fit a Gaussian Naive Bayes to the training data
challenger = ____.____(X_train, y_train)
# Print the F1 test scores of both champion and challenger
print(____(y_test, champion.____(X_test)))
print(____)
# Write back to disk the best-performing model
with open('model.pkl', 'wb') as file:
pickle.____(____, file=file)