Model governance

1. Model governance

Welcome to the last lesson of this chapter and this course. We will use it to introduce the topic of Model Governance.

2. Impact of ML today

Machine Learning models are everywhere nowadays. Small and large enterprises in all industries use them to make complex and frequent decisions with enormous impact.

3. Disaster scenario

Imagine a major bank giving out loans based on predictions from an ML model.

4. Risk underestimation

If that model constantly underestimates each client's risk of default

5. Bankruptcy

that can lead to the bankruptcy of the bank itself, endangering the savings of all of its clients and disrupting the whole economies of multiple countries.

6. Cost of decisions

Whether made by a man or a machine, every wrong estimate has a cost

7. Cost of decision 2

and ML models can make thousands of them per second. It quickly becomes apparent that we cannot let anybody deploy whatever model they want, whenever they want.

8. Governance

The people from DataRobot define model governance as "the overall process for how an organization controls access, implements policy, and tracks activity for models and their results", and add that "effective model governance is the bedrock for minimizing risk to both an organization's bottom line and to its brand".

9. Governance in all phases

Model governance spans all phases of an ML project.

10. During design

The design phase deals with questions such as: In which cases is it ethical to use ML and in which not? Which use cases may justify the usage of private, sensitive data, and how will that be controlled? How will we detect and eliminate potential unethical model bias?

11. During development

In the development phase, governance will define the requirements for documenting the model selection journey, training data preparation, data quality assurance and data and code versioning, reproducibility, etc. This is where having a metadata store, for example, will be of great value.

12. Before production

Then, before going to production, we will have to prove the API of our model is secure, that we have a monitoring and alerting system in place, and that we have understood and handled various ways our ML service can fail. And for all of those controls, it must be defined who exactly will perform them, when, and how we will document the outcomes for audit purposes.

13. Freedom to govern

In some industries, we are free to define our governance approaches ourselves. In others, like in the financial sector, model governance requirements are determined by local and international regulations, where a lack of compliance can lead to severe penalties. Of course, the strictness of governance is proportional to the risk associated with the model in question.

14. Spectrum extremes

Product recommendation engines will be much less scrutinized than models for detecting money laundering.

15. Risk categories

The key criteria for assigning a model a low, medium, or high-risk score are the impact of its decisions and the frequency at which it produces them. In such an analysis, not only financial but also reputational risks are estimated.

16. Summary

It's clear that all these extra steps may add a bit of friction to our project, which is why governance is not the most beloved word in the ML-vocabulary. But its opposite is anarchy, which we clearly don't want. Launching as many ML models in production as fast as possible is not a goal in itself. These models must ultimately generate business value, and we must refrain from doing harm through reckless, ungoverned practices. The more our MLOps framework matures and the more models we have in operation, the more obvious the value of a good governance framework will become.

17. Let's practice!

Let's practice what we have just learned!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.