Configuring a sensor for production
Your pipeline uses a FileSensor to wait for an incoming CSV report before processing it. In the default poke mode, the sensor holds a worker slot the entire time it waits, even when the file takes several minutes to arrive.
For production, a sensor should release the slot between checks and have a safety timeout so it doesn't wait indefinitely.
Это упражнение является частью курса
Building Data Pipelines with Airflow
Практическое интерактивное упражнение
Превратите теорию в практику с помощью одного из наших интерактивных упражнений
Начать упражнение