Creating a partitioned pipeline
A plain asset event tells downstream consumers that "something changed," but not which slice of data. In practice, reporting and analytics teams usually only need to reprocess the partition that actually changed, like a single day, instead of rerunning everything. Asset partitions carry that partition key downstream so consumers can react to exactly the right slice. In this exercise, you'll schedule your sales pipeline on a daily partition timetable and emit a partitioned daily_sales event on every run.
Deze oefening maakt deel uit van de cursus
Building Data Pipelines with Airflow
Interactieve oefening met praktijkervaring
Zet theorie om in actie met een van onze interactieve oefeningen
Begin oefening