Trained model, prediction service, and performance monitoring
1. Trained model, prediction service, and performance monitoring
person: The output of model validation is a trained model that can be pushed to the model registry. The machine learning model registry is a centralized tracking system that stores linage, versioning, and related metadata for published machine learning models. A registry may capture governance data required for auditing purposes, such as who trained and published a model, which datasets were used for training, the values of metrics measuring predictive performance, and when the model was deployed to production. It's a place to find, publish, and use ML models or model pipeline components. Machine learning uses data to answer questions. So prediction, or inference, is the step where we get to answer the questions we posed, whether they are about a business problem or an academic research problem. The trained model is served as a prediction service to production. It's important to note that the process is concerned only with deploying the trained model as a prediction service, for example, a microservice with a REST API, instead of deploying the entire ML system. For example, Google's AI Platform Prediction service has an API for serving predictions from machine learning models. In this particular example, AI Platform Prediction retrieves the trained model and saves it as a PKL in Cloud Storage. PKL is the standard method of serializing objects in Python. Trained models deployed in AI Platform Prediction service are exposed as REST endpoints that can be invoked from any standard client that supports HTTP, such as JupyterLab Notebook. The AI Platform Prediction service can host models trained in popular machine learning frameworks, including TensorFlow, XGBoost, and Scikit-Learn. As a best practice, you need a way to actively monitor the quality of your model in production. Monitoring lets you detect model performance degradation or model staleness. The output of monitoring for these changes then feeds into the data analysis component, which can serve as a trigger to execute the pipeline or to execute a new experimental cycle. For example, monitoring should be designed to detect data skews, which occur when your model training data is not representative of the live data. That is, the data that you used to train the model in the research or production environment does not represent the data that you actually have in your live system, and this leads to model staleness. To understand other performance metrics, you can configure Google's Cloud monitoring to monitor your model's traffic patterns, error rates, latency, and resource utilization. This can help spot problems with your models and find the right machine type to optimize latency and cost.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.