How businesses can become fully MLOps ready
1. How businesses can become fully MLOps ready
We will now discuss what it means as a business to become fully MLOps mature and how to measure the success of the MLOps transformation. We will then look at best practices around MLOps, before concluding this course with a case study.2. Where are we with regard to MLOps?
On our way to becoming a fully MLOps-ready company, we need to measure and assess where we are to understand what we need to change next. How can we measure our level of MLOps-readiness?3. MLOps maturity models
Both Google and Microsoft have released assessments of MLOps maturity. Microsoft's model has five levels, is less steep, and focuses on more dimensions. Let's, therefore, take a closer look.4. Microsoft MLOps maturity model - level 0
The Microsoft model starts with level zero. Here we have isolated teams and no automation.5. Microsoft MLOps maturity model - level 1
The first step toward automation is usually the automation of data ingestion. We add some basic automated tests and code versioning so that different people can work on the code in parallel without interruptions.6. Microsoft MLOps maturity model - level 2
We see better collaboration on level 2, the third level of MLOps. The data scientists are now working closely with data engineers, but software engineers still tend to be siloed. We now have a much better infrastructure. This allows us to restore previous model versions easily. Deploying new model versions remains a manual task, but it is much smoother than earlier. However, the development still relies heavily on manual work by the data scientist.7. Microsoft MLOps maturity model - level 3
On level 3, we are already quite advanced. We now have the software engineers well-integrated into our MLOps team, and we can deploy new model versions easily. All code is tested automatically, and we can often easily trace the root cause if something goes wrong. Even models that never make it into production are tracked, and we rely much less on the data scientist during this step.8. Microsoft MLOps maturity model - level 4
Now we reached the final level 4. The difference compared to level 3 is not as big anymore. We further foster collaboration in our team and work especially on mastering the post-deployment tasks, such as automated re-training of our model depending on new monitoring metrics. As a result, our application now faces only very few downtimes, and we get a skyrocketing high user satisfaction score.9. Measuring our progress towards MLOps
Such a maturity model helps us to see where we are and what is still ahead. However, often we want to track our achievements with a single, measurable target or a KPI, a key performance indicator. How could such a KPI look for our journey towards MLOps maturity?10. Possible MLOps KPIs
Here we can look at DevOps KPIs again and modify them to our needs. We could measure how often we deploy new model versions or other features of our applications. That is, of course, a very rough indicator. Probably better would be to measure how fast we can respond to either incidences, for example, a wrong prediction or downtime, or new requests, for example, adding a new feature or input variable to our model. Such KPIs can help you to monitor progress toward robust MLOps.11. Let's practice!
Now that we know about MLOps maturity, let's practice!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.