Menerapkan alur kerja monitoring

Sepanjang kursus, Anda telah mempelajari alur kerja monitoring. Langkah pertama adalah memantau kinerja. Jika terjadi perubahan negatif, langkah berikutnya melibatkan deteksi drift multivariat untuk mengetahui apakah drift menyebabkan penurunan kinerja, diikuti deteksi drift univariat untuk menelusuri penyebabnya pada fitur individual. Setelah hasil investigasi diperoleh, Anda dapat mengambil langkah untuk menyelesaikan masalah tersebut.

Untuk memantapkan pemahaman ini, dalam latihan ini Anda akan menerapkan proses tersebut pada himpunan data US Consensus. Himpunan data referensi dan analisis sudah dimuat, dan Anda memiliki akses ke CBPE estimator, kalkulator univariat uv_calc, serta sebuah alert_count_ranker untuk pemeringkatan drift fitur.

Latihan ini adalah bagian dari kursus

Pemantauan Machine Learning dengan Python

Latihan interaktif praktis

Cobalah latihan ini dengan menyelesaikan kode contoh berikut.

# Estimate the performance
estimator.____(____)
estimated_results = estimator.____(____)
estimated_results.____.____

Edit dan Jalankan Kode

Latihan ini adalah bagian dari kursus

Pemantauan Machine Learning dengan Python

SkillTag.level.advancedSkillTag.label

4.8+

Mulai Kursus Gratis

In this chapter, you will be introduced to the NannyML library and its fundamental functions. Initially, you will learn the process of preparing raw data to create reference and analysis sets ready for production monitoring. As a practical example, you will investigate predicting the tip amount for taxi rides in New York. Toward the end of the chapter, you will also discover how to estimate the performance of the tip prediction model using NannyML.

Exercise 1: What is NannyML?Exercise 2: Key features of NannyML Exercise 3: Load the dataset Exercise 4: Data preparation for NannyML Exercise 5: Reference or analysis period?Exercise 6: Loading and splitting the data Exercise 7: Creating reference and analysis set Exercise 8: Performance estimation Exercise 9: Specify the algorithm and problem type Exercise 10: Interpreting results Exercise 11: CBPE and DLE workflow Exercise 12: Performance estimation for tip prediction

In this chapter, you will be introduced to realized performance calculators used when ground truth becomes available. You will learn about the more advanced methods for handling results, including filtering, plotting, converting them to data frames, chunking, and establishing custom thresholds. Lastly, you'll apply this knowledge to calculate the business value of a model trained on the hotel booking dataset.

Exercise 1: When labels are available Exercise 2: When performance estimation is off Exercise 3: Comparing estimated and realized performance Exercise 4: Working with calculated and estimated results Exercise 5: Different chunking methods Exercise 6: Modifying the thresholds Exercise 7: Interacting with results Exercise 8: Business value calculation and estimation Exercise 9: Business value calculation Exercise 10: Drop in monetary value Exercise 11: Business calculation for hotel booking dataset

Having detected the performance degradation in the hotel booking model, you will now learn how to identify the underlying issue causing it. In this chapter, you will be introduced to multivariate and univariate drift detection methods. You will also learn how to identify data quality issues and how to address the underlying problems you detect.

Exercise 1: Deteksi drift multivariat Exercise 2: Mengidentifikasi drift yang relevan Exercise 3: Drift pada himpunan data pemesanan hotel Exercise 4: Deteksi drift univariat Exercise 5: Deteksi drift univariat untuk himpunan data pemesanan hotel Exercise 6: Memeringkat hasil univariat Exercise 7: Memvisualisasikan fitur yang mengalami drift Exercise 8: Pemeriksaan kualitas data dan statistik Exercise 9: Pemeriksaan kualitas data Exercise 10: Statistik ringkasan Exercise 11: Penyelesaian masalah Exercise 12: Apa penyelesaiannya?Exercise 13: Haruskah Anda tidak melakukan apa pun atau tidak?Exercise 14: Menerapkan alur kerja monitoring

Latihan Saat Ini

Exercise 15: Selamat