Get startedGet started for free

Modifying the thresholds

In the video, you observed how NannyML calculates threshold values and learned how to customize them to suit your solution.

In this exercise, your task is to define two custom standard deviation and custom thresholds and then apply them to the results obtained from the CBPE algorithm for the US Census dataset.

The reference and analysis sets have been pre-loaded as reference and analysis, along with the nannyml library.

This exercise is part of the course

Monitoring Machine Learning in Python

View Course

Exercise instructions

  • Import ConstantThreshold, and StandardDeviationThreshold from nannyml.thresholds.
  • Initialize the standard deviation method and set std_lower_multiplier and std_upper_multiplier parameters to 2.
  • Initialize the constant threshold method and set the lower parameter to 0.9 and upper to 0.98.
  • Pass the constant threshold method for the f1 metric and the standard deviation method for accuracy to the CBPE algorithm.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Import custom thresholds
from ____.____ import ____, ____

# Initialize custom thresholds
stdt = ____(____=____, ____=____)
ct = ____(____=____, ____=____)

# Initialize the CBPE algorithm
estimator = nannyml.CBPE(
    problem_type='classification_binary',
    y_pred_proba='predicted_probability',
    y_pred='prediction',
    y_true='employed',
    metrics=['roc_auc', 'accuracy', 'f1'],
    thresholds={____: ____, ____: ____})
Edit and Run Code