More PythonOperators

To continue implementing your workflow, you need to add another step to parse and save the changes of the downloaded file. The DAG process_sales_dag is defined and has the pull_file task already added. In this case, the Python function is already defined for you, parse_file(inputfile, outputfile).

Note that often when implementing Airflow tasks, you won't necessarily understand the individual steps given to you. As long as you understand how to wrap the steps within Airflow's structure, you'll be able to implement a desired workflow.

Deze oefening maakt deel uit van de cursus

Introduction to Apache Airflow in Python

Cursus bekijken

Oefeninstructies

Define the Python task to the variable parse_file_task with the id parse_file.
Add the parse_file(inputfile, outputfile) to the Operator.
Define the arguments to pass to the callable.

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

# Add another Python task
____ = ____(
    task_id=____,
    # Set the function to call
    ____
    # Add the arguments
    ____={'inputfile':'latestsales.json', 'outputfile':'parsedfile.json'},
)

Code bewerken en uitvoeren