CommencerCommencez gratuitement

Making a pipeline idempotent

A colleague's pipeline appends sales data on every run, so re-runs and backfills produce duplicates. Both the staging and sales tables have a date column (format YYYY-MM-DD).

Your team needs to make the pipeline idempotent: filter the staging query by the logical date, and add a preoperator to delete existing rows from the sales table.

Cet exercice fait partie du cours

<cours>Building Data Pipelines with Airflow</cours>
Voir le cours

Exercice interactif pratique

Transformez la théorie en action avec l’un de nos exercices interactifs

Commencer l’exercice