1. Lernen
    2. /
  2. Kurse
    3. /
  3. Monitoring Machine Learning in Python
  • 1

    Data Preparation and Performance Estimation

    Kostenlos

    In this chapter, you will be introduced to the NannyML library and its fundamental functions. Initially, you will learn the process of preparing raw data to create reference and analysis sets ready for production monitoring. As a practical example, you will investigate predicting the tip amount for taxi rides in New York. Toward the end of the chapter, you will also discover how to estimate the performance of the tip prediction model using NannyML.

  • 2

    Monitoring Performance and Business Value

    In this chapter, you will be introduced to realized performance calculators used when ground truth becomes available. You will learn about the more advanced methods for handling results, including filtering, plotting, converting them to data frames, chunking, and establishing custom thresholds. Lastly, you'll apply this knowledge to calculate the business value of a model trained on the hotel booking dataset.

  • 3

    Root Cause Analysis and Issue Resolution

    Having detected the performance degradation in the hotel booking model, you will now learn how to identify the underlying issue causing it. In this chapter, you will be introduced to multivariate and univariate drift detection methods. You will also learn how to identify data quality issues and how to address the underlying problems you detect.

Initializing

Ubung

Modifying the thresholds

In the video, you observed how NannyML calculates threshold values and learned how to customize them to suit your solution.

In this exercise, your task is to define two custom standard deviation and custom thresholds and then apply them to the results obtained from the CBPE algorithm for the US Census dataset.

The reference and analysis sets have been pre-loaded as reference and analysis, along with the nannyml library.

Anweisungen

100 XP
  • Import ConstantThreshold, and StandardDeviationThreshold from nannyml.thresholds.
  • Initialize the standard deviation method and set std_lower_multiplier and std_upper_multiplier parameters to 2.
  • Initialize the constant threshold method and set the lower parameter to 0.9 and upper to 0.98.
  • Pass the constant threshold method for the f1 metric and the standard deviation method for accuracy to the CBPE algorithm.