Monitoring and model drift

Monitoring is an important piece of a full AI solution. It enables the team to know when the AI algorithm may be at risk of decreased performance, or model drift.

The example below shows customer satisfaction scores pre-pandemic (teal blue) and post-pandemic (dark orange). The average score pre-pandemic is 2 while the average post-pandemic is 3.

Based on the descriptions of model drift in the video, how would you describe what is happening to customer satisfaction?

Image Description

This exercise is part of the course

Implementing AI Solutions in Business

View Course

Hands-on interactive exercise

Turn theory into action with one of our interactive exercises

Start Exercise

This exercise is part of the course

Implementing AI Solutions in Business

BeginnerSkill Level

4.8+

1500 reviews

Start Course for Free

Learn what AI is, how it has evolved, and how it is different from other non-AI data solutions. In this chapter, you will begin building a framework for how to implement AI into your business, including key elements to an AI strategy, phases for implementation, and basic technical design. Finally, you'll learn about the importance of Responsible AI governance throughout the entire process.

Exercise 1: Overview of AI Exercise 2: Do you know AI?Exercise 3: The phases of an AI solution Exercise 4: 10,000 ft view: how does AI work?Exercise 5: Still know AI?Exercise 6: Generative AI vs. Other AI Exercise 7: Components of an AI solution Exercise 8: Responsible AI Exercise 9: Why is Responsible AI important?Exercise 10: Common Principles of Responsible AI Exercise 11: Assessing risk for a new AI solution

You now know what AI is and how to think about it responsibly. It's time to gain a better sense of its benefits, limitations, and use cases in business. In this chapter, you'll do just that! You'll understand the value AI can bring to an organization as well as the areas it lacks in. You'll go through some high-level use cases across different departments in a business. Finally, you'll learn about principles for choosing a great use case for an AI solution.

Exercise 1: The benefits of AI for a business Exercise 2: Pillars of AI Value Exercise 3: Business value or nice to have?Exercise 4: Limitations of AI Exercise 5: Identify limitations of AI Exercise 6: Limitation or benefit?Exercise 7: Human-in-the-loop Exercise 8: Use cases for AI Exercise 9: Common business use cases Exercise 10: AI advantage in a scenario Exercise 11: Identify a use case for AI Exercise 12: Principles of a good use case Exercise 13: Classify use cases Exercise 14: Deconstructing a task

With use cases in mind, it's time to build a proof of concept! This is a small project that will determine the feasibility and estimate further business value if fully implemented. In this chapter, you will learn the phases of a POC, what makes them successful, and the important components to think about regarding technical infrastructure and data. Finally, you'll learn about the types of roles required for your POC project team.

Exercise 1: Getting started with a POC Exercise 2: Phases of a POC Exercise 3: What makes a successful POC?Exercise 4: Questions to clarify goals Exercise 5: Data Exercise 6: Importance of data Exercise 7: Profiling and evaluating data Exercise 8: Biased data Exercise 9: Infrastructure Exercise 10: Questions about the POC infrastructure Exercise 11: The AI development environment Exercise 12: Buy or build?Exercise 13: Building the project team Exercise 14: Types of roles Exercise 15: Responsibilities

In this final chapter of the course, you will learn the final steps of an AI solution implementation - assessing the POC, scaling, and aligning culture and skills. You will understand the areas in which a POC can be evaluated as well as what are good indicators for moving forward with scaling the solution. You will go over the big components to focus on for scaling to a full implementation, including requirements for culture and upskilling folks in your business.

Exercise 1: Measuring the POC Exercise 2: What makes a POC successful?Exercise 3: Classifying metrics in e-commerce scenario Exercise 4: Assessing the POC Exercise 5: Components for scaling Exercise 6: Balancing value and feasibility Exercise 7: Make a decision about scaling Exercise 8: Considerations for full implementation Exercise 9: Focus areas for a full implementation Exercise 10: Monitoring and model drift

Current Exercise

Exercise 11: DevOps, MLOps, or compliance?Exercise 12: Adoption and skilling Exercise 13: Reasons for no financial benefits Exercise 14: Resistance to AI Exercise 15: Benefits for change management, culture, and upskilling Exercise 16: Wrap-up