When to use poke, reschedule, or deferrable
Your team is reviewing sensor configurations across three pipelines:
- Pipeline A: A sensor checks for a file that usually arrives within 10 seconds of the Dag starting.
- Pipeline B: A sensor waits for a partner data feed that arrives sometime within a 4-hour window. There are 20 similar sensors running concurrently.
- Pipeline C: A sensor checks an API endpoint every 2 minutes and typically waits 15 minutes.
Which combination of sensor modes is the best fit?
Deze oefening maakt deel uit van de cursus
Building Data Pipelines with Airflow
Interactieve oefening met praktijkervaring
Zet theorie om in actie met een van onze interactieve oefeningen
Begin oefening