Alerting

Come hai sentito in questa lezione, puoi monitorare i dati in produzione usando:

metodi deterministici: verificando che tutti gli attributi richiesti siano presenti e abbiano valori all’interno di insiemi rigidamente definiti; e
metodi statistici: verificando se la distribuzione dei dati osservata in produzione è significativamente diversa da quella osservata in fase di training.

Hai anche sentito che i metodi statistici possono essere molto sensibili anche a piccolissimi cambiamenti e generare un gran numero di allerte poco informative (non azionabili).

Ma perché questo è un problema?

Questo esercizio fa parte del corso

Deployment e ciclo di vita in MLOps

Visualizza il corso

Esercizio pratico interattivo

Passa dalla teoria alla pratica con uno dei nostri esercizi interattivi

Inizia esercizio

Questo esercizio fa parte del corso

Deployment e ciclo di vita in MLOps

AvançadoNível de habilidade

4.8+

Inizia il corso gratis

This chapter gives a high-level overview of MLOps principles and framework components important for deployment and life cycling.

Exercise 1: The modern MLOps framework Exercise 2: ML workflows Exercise 3: MLOps benefits Exercise 4: Life-cycling stages Exercise 5: App vs. model Exercise 6: Decommissioning Exercise 7: The model life cycle: recap Exercise 8: MLOps components Exercise 9: Automated sequence Exercise 10: Stores and registries Exercise 11: DevOps or MLOps?

This chapter is dedicated to all the considerations we need to make already in the development phase, in order to ensure a smooth ride when we reach the operations. Our ultimate goal is to explain how to train the model using MLOps best practices and build a model package that enables smooth deployment, reproducibility and post-deployment monitoring.

Exercise 1: Deployment-driven development Exercise 2: Testing your machine learning model Exercise 3: Best time to start deploying Exercise 4: Profiling, versioning, and feature stores Exercise 5: Feature store properties Exercise 6: Profiling and feature store benefits Exercise 7: Ensuring reproducibility Exercise 8: Model build pipelines in CI/CD Exercise 9: Different pipelines Exercise 10: Model build pipeline properties Exercise 11: CI/CD integration Exercise 12: Model packaging Exercise 13: Model formats Exercise 14: Full package

This chapter deals with critical model operations questions such as: - What are the different ways in which we can serve our models? - What is an API, and what are its key functionalities? - How do we thoroughly test our service before making it available to the end users? - How do we update models in production without service disturbance? You will learn about batch prediction, real-time prediction, input and output data validation, unit testing, integration testing, canary deployment, and much more.

Exercise 1: Serving modes Exercise 2: Offline or online?Exercise 3: When time matters - a bit Exercise 4: Building the API Exercise 5: Client-server Exercise 6: API functionalities Exercise 7: Deployment progression and testing Exercise 8: Which test is it?Exercise 9: Progression through environments Exercise 10: Tests per environment Exercise 11: Model deployment strategies Exercise 12: A fitting deployment strategy Exercise 13: Order of risk Exercise 14: Shadow of the shadow

This final chapter is dedicated to monitoring and maintaining ML services after they are deployed, as well as to model governance. You will cover crucial concepts such as verification latency, covariate shift, concept drift, human-in-the-loop systems, and more.

Exercise 1: Monitorare i servizi di ML Exercise 2: Shift vs drift Exercise 3: Latenza Exercise 4: Già?Exercise 5: Monitoraggio e avvisi Exercise 6: Il sistema di monitoraggio Exercise 7: Alerting

Esercizio in corso

Exercise 8: Manutenzione del modello Exercise 9: Data-centric vs Model-centric Exercise 10: Human-in-the-Loop Exercise 11: Governance del modello Exercise 12: Elementi della governance Exercise 13: Fasi della governance Exercise 14: Classificazione del rischio Exercise 15: Riepilogo