This module covers the course objectives and  provides an overview of the course structure.

Welcome to the course

Welcome to the Machine Learning Operations (MLOps) with Vertex AI: Model Evaluation

This module introduces model evaluation challenges and solution offerings by Vertex AI.

Introduction to Model Evaluation

Model Evaluation within MLOps

Model Evaluation Challenges and Solutions offered by Vertex AI

Introduction to Model Evaluation Question 1

Introduction to Model Evaluation Question 2

Introduction to Model Evaluation Question 3

Introduction to Model Evaluation Question 4

Reading List

This module describes challenges of evaluating the Generative AI tasks and best practices to overcome these challenges. The module also covers the different types of model evaluation services available in Vertex AI and then  introduces Vertex AI Automatic Metrics, Automatic Side by Side  and  Safety Bias evaluation services.

Challenges of evaluating the generative AI tasks - Introduction

The Art and Science of Evaluating Large Language Models

Beyond Accuracy: Mastering Evaluation Metrics for Generative AI

Best Practices for LLM Evaluation

Solving Evaluation Challenges

Streamlining Model Evaluation with Computation-based Metrics

Comparing performance with Model based evaluation

Model Evaluation for Generative AI Question 1

Model Evaluation for Generative AI Question 2

Model Evaluation for Generative AI Question 3

Model Evaluation for Generative AI Question 4

Model Evaluation for Generative AI Question 5

Model Evaluation for Generative AI Question 6

Model Evaluation for Generative AI Question 7

Model Evaluation for Generative AI

This module provides a summary of the entire course by covering the most important concepts, tools, technologies, and products.

Course Summary

This course equips machine learning practitioners with the essential tools, techniques, and best practices for evaluating both generative and predictive AI models. Model evaluation is a critical discipline for ensuring that ML systems deliver reliable, accurate, and high-performing results in production.

Participants will gain a deep understanding of various evaluation metrics, methodologies, and their appropriate application across different model types and tasks. The course will emphasize the unique challenges posed by generative AI models and provide strategies for tackling them effectively. By leveraging Google Cloud's Vertex AI platform, participants will learn how to implement robust evaluation processes for model selection, optimization, and continuous monitoring.

Machine Learning Operations (MLOps) with Vertex AI: Model Evaluation


Gain a deep understanding of various evaluation metrics, methodologies, and their appropriate application across different model types and tasks.

Reading List

Machine Learning Operations (MLOps) with Vertex AI: Model Evaluation

Практическое интерактивное упражнение