Interpréter les résultats

Dans ce scénario, vous avez déployé avec succès votre algorithme d’estimation des performances en production. Vous disposez donc d’un graphique pour la métrique ROC AUC estimée.

Votre tâche est maintenant de sélectionner, à partir du graphique, les informations correctes concernant :

la période de lot (chunk) : un point de données unique dans les résultats de surveillance
le seuil supérieur
le seuil inférieur
le nom du mois où une alerte de performance est déclenchée

Cet exercice fait partie du cours

<cours>Surveiller le Machine Learning en Python</cours>

Exercice interactif pratique

Transformez la théorie en action avec l’un de nos exercices interactifs

Commencer l’exercice

Cet exercice fait partie du cours

<cours>Surveiller le Machine Learning en Python</cours>

AvancéNiveau de compétence

4.8+

Commencer le cours gratuitement

In this chapter, you will be introduced to the NannyML library and its fundamental functions. Initially, you will learn the process of preparing raw data to create reference and analysis sets ready for production monitoring. As a practical example, you will investigate predicting the tip amount for taxi rides in New York. Toward the end of the chapter, you will also discover how to estimate the performance of the tip prediction model using NannyML.

Exercise 1: Qu’est-ce que NannyML ?Exercise 2: Fonctionnalités clés de NannyML Exercise 3: Charger le jeu de données Exercise 4: Préparation des données pour NannyML Exercise 5: Période de référence ou d’analyse ?Exercise 6: Chargement et découpage des données Exercise 7: Création des ensembles de référence et d’analyse Exercise 8: Estimation des performances Exercise 9: Spécifier l’algorithme et le type de problème Exercise 10: Interpréter les résultats

Exercice actuel

Exercise 11: Workflow CBPE et DLE Exercise 12: Estimation des performances pour la prédiction du pourboire

In this chapter, you will be introduced to realized performance calculators used when ground truth becomes available. You will learn about the more advanced methods for handling results, including filtering, plotting, converting them to data frames, chunking, and establishing custom thresholds. Lastly, you'll apply this knowledge to calculate the business value of a model trained on the hotel booking dataset.

Exercise 1: When labels are available Exercise 2: When performance estimation is off Exercise 3: Comparing estimated and realized performance Exercise 4: Working with calculated and estimated results Exercise 5: Different chunking methods Exercise 6: Modifying the thresholds Exercise 7: Interacting with results Exercise 8: Business value calculation and estimation Exercise 9: Business value calculation Exercise 10: Drop in monetary value Exercise 11: Business calculation for hotel booking dataset

Having detected the performance degradation in the hotel booking model, you will now learn how to identify the underlying issue causing it. In this chapter, you will be introduced to multivariate and univariate drift detection methods. You will also learn how to identify data quality issues and how to address the underlying problems you detect.

Exercise 1: Multivariate drift detection Exercise 2: Identifying relevant drifts Exercise 3: Drift in hotel booking dataset Exercise 4: Univariate drift detection Exercise 5: Univariate drift detection for hotel booking dataset Exercise 6: Ranking the univariate results Exercise 7: Visualizing drifting features Exercise 8: Data quality and statistic checks Exercise 9: Data quality checks Exercise 10: Summary statistics Exercise 11: Issue resolution Exercise 12: What is the resolution?Exercise 13: Should you do nothing or not?Exercise 14: Implementing a monitoring workflow Exercise 15: Congratulations