1. Airflow web interface
Hello again! Now that we've looked at some basic DAG functionality and interacting with them, it's time to introduce the Airflow web UI.
2. DAGs view
The Airflow web UI is made up of several primary page groups useful in developing and administering workflows on the Airflow platform.
Note that for this course, we'll only be focusing on a few pages but it's helpful to click around the various options and get familiar with what's available.
The DAGs view of the Airflow UI is the page we'll spend most of our time.
3. DAGs view DAGs
It provides a quick status of the number of DAGs / workflows available.
4. DAGs view owner
We can see the owner of the DAG
5. DAGs view runs
and the last several DAG runs.
6. DAGs view schedule
It shows us the schedule for the DAG (in date or cron format).
7. DAGs view last run
when the last run started,
8. DAGs view next run
and when the next DAG run is scheduled.
9. DAGs view recent tasks
We can also see which of the most recent tasks have run.
10. DAGs view example_dag
Don't worry about those for now - instead we'll click on the "example_dag" link which takes us to our DAG detail page.
11. DAG detail view
The DAG detail view gives us specific access to information about the DAG itself, including several views of information (Grid, Graph, and Code) illustrating the tasks and dependencies in the code. We also get access to the Task duration, task tries, timings, a Gantt chart view, and specific details about the DAG. We have the ability to trigger the DAG (to start), refresh our view, and delete the DAG if we desire.
The detail view defaults to the Grid view, showing the specific named tasks, which operators are in use, and any dependencies between tasks.
In the case of our specific DAG, we see that we have one task called generate_random_number.
12. DAG graph view
The DAG graph view arranges the tasks and dependencies in a chart format - this provides another view into the flow of the DAG. You can see the operators in use and the state of the tasks at any point in time.
The tree and graph view provide different information depending on what you'd like to know. Try moving between them when examining a DAG to obtain further details.
For this view we again see that we have a task called generate_random_number. We can also see that it is of the type BashOperator in the middle left of the image.
13. DAG code view
The DAG code view does exactly as it sounds - it provides a copy of the Python code that makes up the DAG. The code view provides easy access to exactly what defines the DAG without clicking in various portions of the UI. As you use Airflow, you'll determine which tools work best for you.
It is worth noting that the code view is read-only. Any DAG code changes must be done via the actual DAG script.
In this view, we can finally see the code making up the generate_random_number task and that it runs the bash command echo $RANDOM.
14. Audit logs
The audit logs page, under the Browse menu option, provides troubleshooting and audit ability while using Airflow. This includes items such as starting the Airflow webserver, viewing the graph or code nodes, creating users, starting DAGs, etc. When using Airflow, look at the logs often to become more familiar with the types of information included, and also what happens behind the scenes of an Airflow install. Note that you'll often refer to the Event type present on the Logs view when searching (such as graph, cli scheduler, etc).
15. Web UI vs command line
In most circumstances, you can choose between using the Airflow web UI or the command line tool based on your preference. The web UI is often easier to use overall. The command line tool may be simpler to access depending on settings (via SSH, etc.)
16. Let's practice!
Now that we've covered some of the most important pages of the Airflow UI, let's practice examining some workflows using it.