Making a pipeline idempotent
A colleague's pipeline appends sales data on every run, so re-runs and backfills produce duplicates. Both the staging and sales tables have a date column (format YYYY-MM-DD).
Your team needs to make the pipeline idempotent: filter the staging query by the logical date, and add a preoperator to delete existing rows from the sales table.
Diese Übung ist Teil des Kurses
<Kurs>Building Data Pipelines with Airflow</Kurs>Interaktive praktische Übung
Verwandle Theorie mit einer unserer interaktiven Übungen in die Praxis
Übung starten