Watching partitioned assets trigger a downstream Dag
The upstream sales_pipeline_v3 and downstream sales_report Dags from the previous exercise are deployed and ready to run. The sales pipeline runs every minute and emits a partitioned daily_sales event on every run, while the report only cares about the date; that's exactly what StartOfDayMapper is for. Time to watch them work together.
Follow these steps in order:
- In the Dags view, activate
sales_reportfirst so it is ready to react when the asset event arrives. - Then activate
sales_pipeline_v3; it runs every minute and emits a partitioneddaily_salesevent on each run. - Wait about a minute for
sales_pipeline_v3to complete —sales_reportthen triggers automatically from the asset event. - Open
sales_report, go to the Runs tab, open the latest run, and find the Mapped Partition key at the top of the run details.
What is the date format of Mapped Partition key?
This exercise is part of the course
Building Data Pipelines with Airflow
Hands-on interactive exercise
Turn theory into action with one of our interactive exercises
Start Exercise