Jobs and Cronjobs
1. Jobs and Cronjobs
Like a Deployment, a Job is a Kubernetes resource. Jobs create one or more Pods to execute a specific task, and then automatically terminate the pods once the task is finished. At its simplest, a Job creates a single Pod, and tracks the task to completion within that Pod. Jobs are also useful if a Pod fails. Unlike Kubernetes controllers, a Job manages a task to completion, rather than to a desired state. Let’s explore this concept with an example–like when a user uploads a video file to a web server for transcoding. In this instance, transcoding would be the task. To begin, the web server would create a Job manifest in a YAML file for the task, and then a Job would be created on the cluster. From there, the Job controller would schedule a Pod on a node to complete the task, and then monitor the Pod. If a node fails, the Pod is lost. The Job controller, which monitors the Pod, would be aware that the task has not been completed. As a result, the Job controller would then reschedule the Pod to run on a different node, and continue to monitor the Pod until the task is completed. After the task is complete, the Job controller would remove the finished Job and any Pods associated with it. It’s at this point that the video would be successfully transcoded! There are two main types of Jobs: non-parallel and parallel. Non-parallel Jobs are the simplest type, used to run a single task one time and ensure its completion. Examples include image processing and data migration. Kubernetes creates one Pod to execute the task, and the Job is considered finished when the Pod exits successfully, or when the required number of completions is reached. If the Pod fails, it will be recreated. Alternatively, parallel Jobs schedule multiple Pods to work on the Job concurrently, but with a predefined limit on the number of completions. This means that each Pod works independently on its assigned task and the Job finishes when the specified number of tasks are completed successfully. Parallel Jobs are useful for situations where tasks need to be completed more than once, like bulk image resizing or parallel scientific computations. Other types of parallel Jobs exist such as work queues and indexed completion jobs, but these are beyond the scope of this course. Let's explore another example, this time how a parallel Job can be used to calculate pi to 2000 decimal places. In a manifest file, the specific type of a Job object is identified through the value of its "Kind" property. This property acts as a label that tells the Kubernetes system what type of batch process the Job represents and how it should be handled. Details about how the job should perform its tasks are specified in the Job spec. Nested within the job spec is a Pod template, which acts as a blueprint for the Pods that the Job will create. Also, this is where its restartPolicy is defined. The restartPolicy can either be set to Never or onFailure. If the restartPolicy is set to Never, it means that any container failure within a Pod will cause the entire Pod to fail permanently. In response, the Job controller will automatically create a new Pod to continue the task. If set to OnFailure, the Pod remains on the node, but the Container restarts. And then there is a backOffLimit field that controls how many times the Job controller should attempt to restart failed Pods before considering the Job itself as failed. Setting spec.parallelism greater than 1 signals to the Job controller that this is a parallel job and that it should create and run multiple Pods concurrently to execute the Job's tasks. To run multiple Pods concurrently for a Job, you need to identify both the completions and parallelism values. The Job controller will initially create as many Pods as specified by parallelism. It will then keep launching new Pods to replace any that finish, up until the total successful completions reach the completions count. Like other Kubernetes objects, Jobs can be inspected by using the kubectl ‘describe’ command. You can use the kubectl get command and label selector to filter Pods. Job details can also be viewed from the Google Cloud console. You can delete a Job by using a kubectl delete command. When you delete a Job, the Job’s Pods will also be deleted, but you can retain the Pods by setting the cascade flag to false. Alternatively, you can delete jobs directly in the Google Cloud console. Let’s finish this lesson by exploring Cronjobs. For scheduling Jobs, the Cron format offers a powerful and flexible way to specify precise dates and times for repeated execution. Cron uses a straightforward text format, which is like a mini-language with specific rules: you write a string with different parts separated by spaces, and each part controls a different aspect of when and what to execute. You can use an asterisk to represent an entire range of possible values, for example, each minute or each hour. Cron lets you schedule your jobs precisely by specifying individual times or ranges. You can do this by listing specific values separated by commas (like 1, 3, 7) or using a hyphen to define a range (like 1-5). To make a value repeat at regular intervals, add a slash followed by the interval number. This works for both asterisks (to repeat every interval) and ranges (to repeat the entire range at the specified interval).2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.