Understanding parameter options
You've used a few different methods to add templates to your workflows. Considering the differences between options, why would you want to create individual tasks (ie, BashOperators) with specific parameters vs a list of files?
For example, why would you choose
t1 = BashOperator(task_id='task1', bash_command=templated_command, params={'filename': 'file1.txt'}, dag=dag)
t2 = BashOperator(task_id='task2', bash_command=templated_command, params={'filename': 'file2.txt'}, dag=dag)
t3 = BashOperator(task_id='task3', bash_command=templated_command, params={'filename': 'file3.txt'}, dag=dag)
over using a loop form such as
t1 = BashOperator(task_id='task1',
bash_command=templated_command,
params={'filenames': ['file1.txt', 'file2.txt', 'file3.txt']},
dag=dag)
Deze oefening maakt deel uit van de cursus
Introduction to Apache Airflow in Python
Praktische interactieve oefening
Zet theorie om in actie met een van onze interactieve oefeningen.
Begin met trainen