1. Validate Expectation Suites
So far, we've been validating our Expectation Suites directly from our Batch. Now, we'll discover another method of validation, using Validation Definitions.
2. Validation Definitions
A Validation Definition is an object that takes in an Expectation Suite and a Batch Definition, and runs the Expectations against the data. You can think of this like an auditor that receives the Expectation Suite and goes through the dataset to make sure it meets the listed Expectations.
3. Creating a Validation Definition
To create a Validation Definition, we use GX's ValidationDefinition class, providing a name, a Batch Definition, and an Expectation Suite. Note that we didn't pass in the DataFrame itself; we'll get back to this later.
4. Viewing a Validation Definition
As we can see, the Validation Definition contains information about the data in the `"data"` field, which is populated by the Batch Definition, and about the Expectations in the `"suite"` field, which is populated by the Expectation Suite. It also contains fields for name and ID.
5. Viewing a Validation Definition
We can access each of these fields individually, as attributes: name, data, suite, and id.
6. Viewing a Validation Definition
We can also view the Data Source associated with the Validation Definition using the `.data_source` attribute.
7. Running a Validation Definition
To actually run our validation, we simply use the Validation Definition's `.run()` method. Now is when we pass in our DataFrame, with the exact same syntax we used when creating a Batch: we assign to `batch_parameters` a dictionary with `"dataframe"` as the key and our DataFrame as the value.
8. Validation Definition errors
Here, when we try to run our Validation Definition, we get an error. This exception is pretty straightforward. It's basically telling us that we need to add our Expectation Suite to the Data Context before we can run it through a Validation Definition.
9. Adding an Expectation Suite
To add our Expectation Suite to the Data Context, we use the `.add()` method of the Context's `.suites` attribute, passing in our Expectation Suite object to the `suite` parameter.
10. Assessing a Validation Definition
Let's try that again. Now our Validation Definition runs without error. By using the `.success` attribute, we can see that our validation failed. We can also take a look at our Validation Results using the same `.describe()` method we used previously.
11. Assessing a Validation Definition
This looks almost identical to the Validation Results we got from running the `batch.validate()` method.
12. A note about GX
At this point, I'd like to note that, in Great Expectations, there are several ways to accomplish the same task. Batch Definitions and Validation Definitions are one example of this: they are two ways of applying the same Expectation Suite to a Batch of data. While I'll try to give you a broad understanding of GX in this course, we won't be able to cover everything. Just keep this in mind if you come across implementations of GX that use different workflows.
13. Cheat sheet
To summarize, a Validation Definition is the link that ties together our Expectations (via the Expectation Suite) and our data (via the Batch Definition). Before running a Validation Definition, we need to add our Expectation Suite to the Data Context using the `context.suites.add()` method. We create a Validation Definition using the `ValidationDefinition` class and run it using its `.run()` method. We can view a summary of the Validation Results with the same `.describe()` method we've been using.
14. Let's practice!
Time to practice creating your own Validation Definitions to validate your Expectation Suites!