Een monitoringworkflow implementeren

In de hele cursus heb je geleerd over de monitoringworkflow. De eerste stap is prestatiemonitoring. Als er negatieve veranderingen zijn, volg je met multivariate driftdetectie om te bepalen of drift de prestatiedaling heeft veroorzaakt, gevolgd door univariate driftdetectie om de oorzaak in individuele features te achterhalen. Zodra je de resultaten van het onderzoek hebt, kun je stappen zetten om het probleem op te lossen.

Om deze kennis te verankeren, pas je in deze oefening dit proces toe op de US Consensus-gegevensset. De referentie- en analysegegevenssets zijn al geladen, en je hebt toegang tot de CBPE-estimator, de univariate calculator uv_calc en een alert_count_ranker om featuredrift te rangschikken.

Deze oefening maakt deel uit van de cursus

Monitoring Machine Learning in Python

Cursus bekijken

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

# Estimate the performance
estimator.____(____)
estimated_results = estimator.____(____)
estimated_results.____.____

Code bewerken en uitvoeren

Deze oefening maakt deel uit van de cursus

Monitoring Machine Learning in Python

SkillTag.level.advancedSkillTag.label

4.8+

Begin de cursus gratis

In this chapter, you will be introduced to the NannyML library and its fundamental functions. Initially, you will learn the process of preparing raw data to create reference and analysis sets ready for production monitoring. As a practical example, you will investigate predicting the tip amount for taxi rides in New York. Toward the end of the chapter, you will also discover how to estimate the performance of the tip prediction model using NannyML.

Exercise 1: What is NannyML?Exercise 2: Key features of NannyML Exercise 3: Load the dataset Exercise 4: Data preparation for NannyML Exercise 5: Reference or analysis period?Exercise 6: Loading and splitting the data Exercise 7: Creating reference and analysis set Exercise 8: Performance estimation Exercise 9: Specify the algorithm and problem type Exercise 10: Interpreting results Exercise 11: CBPE and DLE workflow Exercise 12: Performance estimation for tip prediction

In this chapter, you will be introduced to realized performance calculators used when ground truth becomes available. You will learn about the more advanced methods for handling results, including filtering, plotting, converting them to data frames, chunking, and establishing custom thresholds. Lastly, you'll apply this knowledge to calculate the business value of a model trained on the hotel booking dataset.

Exercise 1: When labels are available Exercise 2: When performance estimation is off Exercise 3: Comparing estimated and realized performance Exercise 4: Working with calculated and estimated results Exercise 5: Different chunking methods Exercise 6: Modifying the thresholds Exercise 7: Interacting with results Exercise 8: Business value calculation and estimation Exercise 9: Business value calculation Exercise 10: Drop in monetary value Exercise 11: Business calculation for hotel booking dataset

Having detected the performance degradation in the hotel booking model, you will now learn how to identify the underlying issue causing it. In this chapter, you will be introduced to multivariate and univariate drift detection methods. You will also learn how to identify data quality issues and how to address the underlying problems you detect.

Exercise 1: Multivariate driftdetectie Exercise 2: Relevante drifts identificeren Exercise 3: Drift in de hotelreserveringsgegevensset Exercise 4: Univariate driftdetectie Exercise 5: Univariate driftdetectie voor de hotelboekingen-gegevensset Exercise 6: De univariate resultaten rangschikken Exercise 7: Driftende features visualiseren Exercise 8: Datakwaliteit en statistische controles Exercise 9: Controles op datakwaliteit Exercise 10: Samenvattende statistieken Exercise 11: Probleemoplossing Exercise 12: Wat is de oplossing?Exercise 13: Moet je niets doen of niet?Exercise 14: Een monitoringworkflow implementeren

Huidige oefening

Exercise 15: Gefeliciteerd