Creating a templated BashOperator
You've successfully created a BashOperator that cleans a given data file by executing a script called cleandata.sh
. This works, but unfortunately requires the script to be run only for the current day. Some of your data sources are occasionally behind by a couple of days and need to be run manually.
You successfully modify the cleandata.sh
script to take one argument - the date in YYYYMMDD format. Your testing works at the command-line, but you now need to implement this into your Airflow DAG. For now, use the term {{ ds_nodash }}
in your template - you'll see exactly what this means later on.
This exercise is part of the course
Introduction to Apache Airflow in Python
Hands-on interactive exercise
Turn theory into action with one of our interactive exercises
