Understand why Great Expectations (GX) is such a powerful tool for monitoring data quality. Get familiar with the basics of GX, including how to start a session using a Data Context, and how to load in a pandas dataframe using a Data Source, Data Asset, and Batch Definition. 

Create a Data Context

Create your Data Context

Sneak Peek into GX

Options

Great Expectations

Data Context

Expectation

Keyword Definitions

Connect to Data

`name` parameter

Data Source

Data Asset

Concept Review

Create a Data Source and Data Asset

Read Data in Batches

Create a Batch Definition and Batch

Inspect your Batch

Connecting to Data

Create and evaluate basic shape and schema Expectations. Validate your Expectations either individually, as part of an Expectation Suite with a Batch Definition, or using a Validation Definition.

Create Expectations

Establish a Column Count Expectation

Evaluate Validation Results

Schema Expectations

Establish a Column Name Expectation

Establish a Column Range Expectation

Create a Suite of Expectations

Create an Expectation Suite

Validate your Expectation Suite

Validate Expectation Suites

Create and Run a Validation Definition

Validation Definition vs. Batch Definition

Establishing Expectations

Learn practical skills that will help you dominate the dynamic nature of Expectations in the real world. Deploy Validation Definitions using Checkpoints; update your Expectation Suites; and learn how to add, retrieve, list, and delete key GX components.

Deploy Validation Definitions

Concept Check

Create a Checkpoint

Update Expectation Suites

Copy an Expectation

Delete an Expectation

Update an Expectation

Manage Components

Add and list components

Manage a Checkpoint

GX in Practice

Dive head-first into the world of Expectations. Practice creating basic column Expectations, row- and aggregate-level numeric Expectations, string and string parseability Expectations, and more. Learn how to apply Expectations to only some rows of a dataframe.

Basic Column Expectations

Establish row-level Expectations

Establish aggregate-level Expectations

Type-Specific Expectations

Establish numeric Expectations

Establish string Expectations

Conditional Expectations

Write up a Conditional Expectation

Invert a Conditional Expectation

Wrap-Up

All About Expectations

weather_new.csv

movie_dataset.csv

life_expectancy.csv

renewable_new.csv

shein_footwear_clean.csv

Great Expectations is a powerful Python library for monitoring data quality in data science and engineering workflows. Learn to connect real-world datasets, apply, edit, and delete Expectations, and build pipelines to validate new datasets in production. You'll explore Expectations for numeric and string columns and write validations across columns. By the end, you'll confidently ensure your data meets quality standards.

Great Expectations is a powerful tool for monitoring data quality in data science and data engineering workflows. The platform can be easily integrated into Python, making it a useful library for Python users to master.

At the core of Great Expectations are Expectations, or assertions that you'd like to verify about your data. You'll begin this course by learning how to connect to real-world datasets and apply Expectations to them. You'll then learn how to retrieve, edit, delete Expectations, and build pipelines for applying Expectations to new datasets in a production deployment.

Finally, you'll learn about specific types of Expectations, such as for numeric and string columns, and how to write Expectations of one column conditional on the values of other columns.

By the end of this course, you'll have a strong foundation in the Great Expectations Python library. You'll be able to use the platform's core functionalities to monitor the quality of your data, and you'll be able to use your data with confidence that it meets your data quality standards.

Data Manipulation with pandas

Ensure high data quality in data science and data engineering workflows with Python's Great Expectations library.

Introduction to Data Quality with Great Expectations

Machine Learning Engineer

The string name of a component stored in GX

An object that tells GX how to connect to a specific source of external data

A collection of records within a Data Source

Concept Review

Introduction to Data Quality with Great Expectations

Praktische interactieve oefening