More PythonOperators
To continue implementing your workflow, you need to add another step to parse and save the changes of the downloaded file. The DAG process_sales_dag
is defined and has the pull_file
task already added. In this case, the Python function is already defined for you, parse_file(inputfile, outputfile)
.
Note that often when implementing Airflow tasks, you won't necessarily understand the individual steps given to you. As long as you understand how to wrap the steps within Airflow's structure, you'll be able to implement a desired workflow.
This exercise is part of the course
Introduction to Apache Airflow in Python
Exercise instructions
- Define the Python task to the variable
parse_file_task
with the idparse_file
. - Add the
parse_file(inputfile, outputfile)
to the Operator. - Define the arguments to pass to the callable.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Add another Python task
____ = ____(
task_id=____,
# Set the function to call
____
# Add the arguments
____={'inputfile':'latestsales.json', 'outputfile':'parsedfile.json'},
)