Model serving
1. Model serving
SPEAKER: Let's focus on the third stage of the ML workflow, model serving. The recipes are ready, and now it's time to serve the meal. This represents the final stage of the machine learning workflow, model serving. Model serving consists of two steps-- first, model deployment, which you can compare to serving a meal to a hungry customer; and second, model monitoring, which is like checking with the waitstaff to ensure that the restaurant is operating efficiently. It's important to note that model management exists throughout this whole workflow to manage the underlying machine learning infrastructure. This lets data scientists focus on what to do and not on how to do it. Let's start with model deployment, which is the exciting time when the model is implemented and ready to serve. You have two primary options. Option one-- deploy the model to an endpoint for real-time predictions, or often called online predictions. This option is best when immediate results with low latency are needed, such as making instant recommendations based on a user's browsing habits whenever they're online. A model must be deployed to an endpoint before it can be used to serve real-time predictions. Option two-- request the prediction job directly from the model resource for batch prediction. This option is best when no immediate response is required, for example, sending out marketing campaigns every other week based on the user's recent purchasing behavior, and what's currently popular on the market. Batch prediction does not require deploying the model to an endpoint. You can deploy a model either using the UI on Vertex AI or using code by calling APIs. You'll practice building an endpoint later in the lab. Beyond making predictions in the Cloud, deploying the model off cloud with edge computing is also possible. This approach is generally adopted when the model needs to be deployed in a specific environment to mitigate latency, ensure privacy, or enable offline functionality. For instance, consider an IoT application like object detection that utilizes a camera feed in a manufacturing plant. In such a use case, the added latency of relying on the cloud can be impractical. Once the model is deployed and begins making predictions or generating contents, it's important to monitor its performance. The backbone of automating ML workflow on Vertex AI is a toolkit called Vertex AI Pipelines. It automates, monitors, and governs machine learning systems by orchestrating the workflow in a serverless manner. Imagine you're in a production control room, and Vertex AI Pipelines is displaying the production data onscreen. If something goes wrong, it automatically triggers and displays a warning based on a predefined threshold. With Vertex AI Workbench and Colab Enterprise, which are notebook tools, you can define your own pipeline using SDKs. You can do this with pre-built pipeline components, which means that you primarily need to specify how the pipeline is put together using components as building blocks. You'll explore more details about Vertex AI pipelines in the next lesson. And it's with these final two steps, model deployment and model monitoring, that you complete the exploration of the machine learning workflow. The restaurant is open and operating smoothly. Bon appetit.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.