Human-in-the-loop workflows

1. Human-in-the-loop workflows

Sometimes a pipeline needs a human decision before it can continue. You've seen this idea before, and now let's explore two operators that give reviewers more control.

2. HITL use cases

When would we need a human in the loop? We need one any time a pipeline produces output that requires judgment before proceeding. For example, approving an AI-generated response before sending it to a customer, routing a support ticket to the right team, or reviewing a data quality report before publishing.

3. The HITL operator family

As a quick recap, Airflow provides four HITL operators in the standard provider. You already used ApprovalOperator for simple approve-or-reject decisions. The other three handle more complex interactions: HITLBranchOperator lets a human choose which downstream tasks to run, HITLOperator combines custom options with a form, and HITLEntryOperator collects pure form input without presenting options. All four are deferrable, meaning they release the worker slot while waiting for the human to respond. In this video, we'll focus on HITLEntryOperator and HITLBranchOperator.

4. HITLEntryOperator

Let's walk through the HITLEntryOperator. We import it from the standard HITL provider, along with Param from the Airflow SDK. The operator takes several parameters. The task_id identifies it in the Dag. The subject is the title the reviewer sees in the Airflow UI. The body uses a Jinja template to pull in the AI-generated draft from a previous task via XCom. And params defines the form fields: here, a feedback field that starts empty and an urgency field that defaults to "p3". When this task runs, it pauses the pipeline and presents a form with these fields. The reviewer fills in their feedback, sets the urgency, and submits.

5. Accessing the response

Downstream tasks access the reviewer's input through the operator's XCom output. Here, send_response receives the full HITL output and extracts feedback and urgency from the params_input dictionary. At the bottom, we use the chain function to set the execution order. Here, generate_draft runs first, then review pauses for human input, and finally send_response processes the result. Form values always live under the params_input key, and any selected options live under chosen_options.

6. HITLBranchOperator

The HITLBranchOperator lets a human decide the execution path. We import it from the same standard HITL provider. The key parameters are options, which define the choices the reviewer sees, and options_mapping, which maps each choice to a downstream task ID. The body provides the reviewer with context for their decision. The defaults parameter specifies which option is pre-selected if no one responds before the timeout. When the reviewer selects "Billing Issue", Airflow runs the handle_billing task and skips handle_technical and handle_general. It works like any Airflow branching operator: the selected path runs, all others are skipped.

7. Required Actions in the UI

When a HITL task runs, it creates a Required Action visible in the Airflow UI. Reviewers can find it in the Dag Run view by clicking on the deferred task. They respond directly in the task instance view. All HITL operators are deferrable, which is important to understand: when the task defers, it releases its worker slot completely. The Triggerer, a separate Airflow process, takes over and monitors for the human response using async I/O. This means waiting for human input does not block other tasks from running. When the reviewer submits their response, the Triggerer wakes the task up, it resumes execution on a worker, and downstream tasks continue.

8. Let's practice!

Let's build a pipeline with human input.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.