Airflow web interface
1. Airflow web interface
Hello again! Now that we've looked at some basic Dag functionality and interaction, it's time to learn the Airflow web UI.2. Home View
The Airflow web UI is made up of several primary page groups that are useful for developing and administering workflows on the Airflow platform. The Home view is the default Airflow page. It provides a dashboard showing the general state of the system. The navigation links to the left provide access to the individual sections as required. For this course, we'll only be focusing on a few pages, but it's helpful to click around the various options and get familiar with what's available.3. Dags view
The Dags view of the Airflow UI is the page where we'll spend most of our time. Let's explore it further.4. Dags view Dags
The Dags view provides a quick status of the number of Dags available,5. Dags view Latest Run
shows the latest run for each Dag,6. Dags view runs
and shows the state of the last several Dag runs.7. Dag detail navigation
If we click on one of the Dags, in this case update_state, we can access the detail view for the Dag.8. Dags view update_state (Grid)
The Dag detail view provides specific information about the Dag itself, including several views illustrating the tasks and dependencies in the code: Grid, Graph, and Code. We also get access to the latest runs, tasks, calendar, and details about the Dag.9. Dags view update_state tasks
The detail view defaults to the Grid view, showing the specific named tasks. In this specific Dag, we have three tasks.10. Dags view update_state (Graph)
The Dag graph view arranges the tasks and dependencies in a chart format which shows the flow of the Dag. You can see the task types and their state at any point in time.11. Dags view update_state (Code)
The Dag detail can also show the Python code that defines the Dag. The code link provides easy access to exactly what defines the Dag without clicking in the UI. As you use Airflow, you'll determine which tools work best for you. Note the code view is read-only. Any Dag code changes must be made through the actual Dag script.12. Dag Runs view
In addition to the information on Dags, we can also see the most recent Dag Runs for all Dags.13. Task instances
Also, we can choose to see the instances of all tasks from all Dags that have run.14. Dag Import Errors
Sometimes there are issues importing a Dag. If so, this button will appear on the Dag page.15. Dag Import Errors Detail
Clicking it provides the details of the error.16. Audit Log
The audit logs page, under the Browse menu, provides troubleshooting and audit ability while using Airflow.17. Audit Log
This includes items such as starting the Airflow webserver, viewing the graph or code nodes, creating users, starting Dags, and more. When using Airflow, examine the logs to gain familiarity with the types of information included, and what happens behind the scenes of an Airflow install. You'll often refer to the Event type present on the Logs view when searching, such as graph or cli scheduler.18. Web UI vs command line
You can usually choose between using the Airflow web UI or the command line tool based on your preference. Both are equally powerful and use the Airflow API-Server. The web UI is often easier to use, and the command line tool may be simpler to access depending on settings like SSH.19. Let's practice!
Now that we've covered some of the most important pages of the Airflow UI, let's practice examining some workflows using it.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.