When to use poke, reschedule, or deferrable
Your team is reviewing sensor configurations across three pipelines:
- Pipeline A: A sensor checks for a file that usually arrives within 10 seconds of the Dag starting.
- Pipeline B: A sensor waits for a partner data feed that arrives sometime within a 4-hour window. There are 20 similar sensors running concurrently.
- Pipeline C: A sensor checks an API endpoint every 2 minutes and typically waits 15 minutes.
Which combination of sensor modes is the best fit?
Este exercicio faz parte do curso
Building Data Pipelines with Airflow
exercicio interativo prático
Transforme teoria em prática com um dos nossos exercicio interativos
Iniciar exercicio