Establish aggregate-level Expectations

Time to practice writing some column-specific Expectations at the aggregate level. The Expectation Suite and Batch have already been assigned to the variables suite and batch, respectively, and loaded with the Shein Footwear dataset. Great Expectations and pandas are available as gx and pd, respectively.

This exercise is part of the course

Introduction to Data Quality with Great Expectations

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# "colour" should be in the set "Khaki", "Purple", or "Grey"
colour_expectation = gx.expectations.ExpectColumnDistinctValuesToBeInSet(
    ____="colour", value_set={"Khaki", "Purple", "Grey"}
)

# "seller_name" should have 7 to 10 distinct values
seller_expectation = gx.expectations.ExpectColumnUniqueValueCountToBeBetween(
    column="seller_name", ____=7, ____=10
)

# "link" should have all unique values
link_expectation = gx.expectations.____(
    column="link"
)

# "review_count" should have a most common value in the set "0" or "100+"
review_count_expectation = gx.expectations.ExpectColumnMostCommonValueToBeInSet(
    column=____, value_set=____
)

Edit and Run Code

This exercise is part of the course

Introduction to Data Quality with Great Expectations

IntermediateSkill Level

4.8+

Start Course for Free

Understand why Great Expectations (GX) is such a powerful tool for monitoring data quality. Get familiar with the basics of GX, including how to start a session using a Data Context, and how to load in a pandas dataframe using a Data Source, Data Asset, and Batch Definition.

Exercise 1: Create a Data Context Exercise 2: Create your Data Context Exercise 3: Sneak Peek into GX Exercise 4: Keyword Definitions Exercise 5: Connect to Data Exercise 6: Concept Review Exercise 7: Create a Data Source and Data Asset Exercise 8: Read Data in Batches Exercise 9: Create a Batch Definition and Batch Exercise 10: Inspect your Batch

Create and evaluate basic shape and schema Expectations. Validate your Expectations either individually, as part of an Expectation Suite with a Batch Definition, or using a Validation Definition.

Exercise 1: Create Expectations Exercise 2: Establish a Column Count Expectation Exercise 3: Evaluate Validation Results Exercise 4: Schema Expectations Exercise 5: Establish a Column Name Expectation Exercise 6: Establish a Column Range Expectation Exercise 7: Create a Suite of Expectations Exercise 8: Create an Expectation Suite Exercise 9: Validate your Expectation Suite Exercise 10: Validate Expectation Suites Exercise 11: Create and Run a Validation Definition Exercise 12: Validation Definition vs. Batch Definition

Learn practical skills that will help you dominate the dynamic nature of Expectations in the real world. Deploy Validation Definitions using Checkpoints; update your Expectation Suites; and learn how to add, retrieve, list, and delete key GX components.

Exercise 1: Deploy Validation Definitions Exercise 2: Concept Check Exercise 3: Create a Checkpoint Exercise 4: Update Expectation Suites Exercise 5: Copy an Expectation Exercise 6: Delete an Expectation Exercise 7: Update an Expectation Exercise 8: Manage Components Exercise 9: Add and list components Exercise 10: Manage a Checkpoint

Dive head-first into the world of Expectations. Practice creating basic column Expectations, row- and aggregate-level numeric Expectations, string and string parseability Expectations, and more. Learn how to apply Expectations to only some rows of a dataframe.

Exercise 1: Basic Column Expectations Exercise 2: Establish row-level Expectations Exercise 3: Establish aggregate-level Expectations

Current Exercise

Exercise 4: Type-Specific Expectations Exercise 5: Establish numeric Expectations Exercise 6: Establish string Expectations Exercise 7: Conditional Expectations Exercise 8: Write up a Conditional Expectation Exercise 9: Invert a Conditional Expectation Exercise 10: Wrap-Up