Using the PythonOperator
You've implemented several Airflow tasks using the BashOperator but realize that a couple of specific tasks would be better implemented using Python. You'll implement a task to download and save a file to the system within Airflow.
The requests
library is imported for you, and the DAG process_sales_dag
is already defined.
This exercise is part of the course
Introduction to Apache Airflow in Python
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Define the method
def ____(____, ____):
r = requests.get(URL)
with open(savepath, 'wb') as f:
f.write(r.content)
# Use the print method for logging
print(f"File pulled from {____} and saved to {____}")