Templating with Jinja

1. Templating with Jinja

Welcome back! In this lesson, we'll discuss a templating language called Jinja and how we can use it to simplify working with Dags in Airflow.

2. What is a template?

So what is a template? For our purposes it's easiest to define a template as a defined format that allows Airflow to substitute information automatically. Most often, templates are simply text that can be modified in most editors. We'll go over the details more in a moment, but a template is designed to simplify access and re-use.

3. What is Jinja?

We'll be using a specific templating language with Airflow, called Jinja. Jinja is a simple text-based templating system used in many tools beyond just Airflow, such as Django and Flask. To define a Jinja template, simply put the desired content between two opening and closing curly braces within your text files. When a Dag is run, it will replace the contents of the braces with the appropriate content. This behaves like a Python f-string, but in the Airflow environment. Airflow has many Jinja functions available for use in Dags. This allows for more dynamic usage of Airflow, such as different source and destination data locations. In addition, Jinja also supports using for loops to simplify coding.

4. Airflow Jinja functions

There are many Jinja substitutions in Airflow. This includes the logical_date, which is when a Dag run occurs. There's also ds, which is a specific formatting of the logical_date in YYYY-MM-DD format. Accessing logical_date.year, .month, or .day gives access to the date components. Params provides access to the runtime parameters of the current Dag run. Jinja can also be used for calculations, defining tasks in a loop, and more. The macros.ds_add option allows date calculations in the template.

5. Example using SmtpNotifier

Let's take a look at an example where Jinja can be useful. We've recently seen how the SmtpNotifier works and that we can pass in a subject field. Previously, those messages were static, meaning the same message would be received each time. Using Jinja, we can modify the content of the field easily. Here's the initial portion of our Dag with the on_failure_callback. For the subject, we'll use the logical_date Jinja substitution and we'd now receive a subject that updates to include the date in the alert email.

6. Example using FileSensor

Let's try a FileSensor example and configure the filepath dynamically. Consider a scenario where we need to grab the input.csv file that exists the day prior to the logical_date. We can use the Jinja macros.ds_add substitution. If we pass in the ds variable and -1 for one less day, we'll get the previous day. For example, if logical_date is April 30, 2026, the sensor will look for the file at /data/2026-04-29/input.csv.

7. Jinja notes

It should be noted that you can't use Jinja everywhere. It's best to refer to the Airflow component documentation and look for templated attributes, which indicate that Jinja substitutions should work. You can also view the Python code of any Dag in the Airflow UI by clicking on the Dag name and selecting the Code tab. This is useful for inspecting which fields are configured and how they're set up. If you find Jinja is not supported in a component, we'll cover some other options shortly.

8. Let's practice!

We've covered some different concepts in this lesson. Let's try a few exercises and solidify what we've learned.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.