Job List
1. Job List
Omar: Hey, there, my name is Omar Ismail, a solutions developer at Google Cloud. You have launched your Apache Beam pipeline onto Dataflow, and now you want to view its usage. What monitoring resources exist? There is a Dataflow monitoring page which you will learn to use for both batch and streaming jobs. Dataflow also integrates with cloud monitoring. Cloud monitoring is mostly used as a back end to build the Dataflow UI, and you can use it as well to build your own custom dashboards. Let's start by looking at the job list page in Dataflow. This is the first page you see when you visit Dataflow. The job list page shows all the jobs that have run over the last 30 days. If we click on the running button... We see only the jobs that are running. In my project, I currently have two. if we toggle back to show all jobs and enable sorting... the columns become sortable. For example, I can sort by the start time... or by pipeline status. Let's switch off sorting... and view how filtering works. Click into the filter text box, and you can choose any of the columns to filter by. I'm going to select status. And that shows me all the available statuses. Let me select all the ones that failed. I can use more than one filter and set its relationship to the previous one. "And" is implied by default. So if you want your condition to be an "or," you'll need to select the "or" option. I am not going to set an "or." Instead, I'm going to filter by name... And find jobs that have the word "Wikipedia" in them. Imagine a use case where you want to check the status of all of your failed jobs that ran with the word "Wikipedia" in them. Instead of having to add that filter every time, you can bookmark that link and revisit those jobs every time. If I take the link in the URL... Copy it... Open a new tab... And then paste in go... We can see that it takes us back to the filtered view that we had. We can then go ahead and add the link as a bookmark.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.