1. Scheduling your scripts with Cron
Now you have mastered creating Bash scripts, let's learn how to program them to run on a schedule. You'll never need to lift a finger to execute code again!
2. Why schedule scripts?
There are many situations where being able to schedule scripts is helpful.
Firstly, you may have regular tasks to be done. Such as daily, weekly or multiple times per day.
You could try to remind yourself - but what if you forget or are away?
Secondly, you can better use resources. Such as running scripts early in the morning when server usage is low.
Learning to schedule with cron is an essential skill in modern data infrastructures.
3. What is cron?
The program 'cron' has been part of unix-like systems for a long time. Since the 70's in fact. We humans have been lazy for that long!
The name comes from the Greek word for time, chronos.
This program is driven by something called a 'crontab'. This file contains cron jobs which tells crontab what code to run and when.
4. Crontab - the driver of cronjobs
You can see what schedules, called 'cronjobs' you have currently programmed by running the following in the terminal.
We see that this user has no cronjobs.
We better make a schedule then!
5. Crontab and cronjob structure
This image from wikipedia demonstrates how to construct a cronjob. You can have as many cronjobs as you like, one per line in the crontab file.
There are 5 stars, one for each time unit. There are minutes, hours, days of the month, months of the year and days of the week.
The default (just a star) means to run at every interval.
6. Cronjob example
Let's walk through some example cronjobs.
In this first example.
The minutes star is 5 (meaning 5 minutes past the hour). Which hour? Well, the hours star is 1 (meaning after 1am). The last three are all stars, meaning 'every'.
So overall, we run myscript.sh every day at 1:05am.
In this second example.
The minutes star is 15 (meaning 15 minutes past the hour), the hours star is 14 (meaning after 2pm). The stars for day-of-month and month-of-year are 'every'. The last star is day 7 (on Sundays).
So overall we run myscript.sh at 2:15pm every Sunday.
7. Advanced cronjob structure
You can also run programs multiple times per day (or other time units) or even every X time increments.
For specific intervals, you can use a comma.
For example, this cronjob would run at 15, 30 and 45 minutes past the hour. For every hour specified in the next star. Here it is every hour, every day etc.
You can use a slash for every X increment.
In another example, this cronjob runs every 15 minutes. Again, for every hour, every day etc.
8. Your first cronjob
Let's schedule a script called extract_data to run every morning at 1.30am. The steps are as follows.
Firstly type crontab with an 'e' flag to edit your list of cronjobs.
If this is the first time, it may ask which editor you want to use. Nano is an easy option and has a less-steep learning curve than vi (or vim).
Create the cronjob on a blank line.
We want it to run at 30 minutes past the first hour, every day, week, month etc. So we put 30 as the first star, 1 as the second star and leave the rest as 'every'.
9. Your first cron job
Finally you need to exit the editor, saving the crontab.
If you were using nano on a mac, this is how you exit and save. You may need to google the commands for your system and editor.
You will see a note in the terminal that a crontab has been installed.
Let's check that the cronjob was correctly programmed by running 'crontab' with the l flag.
The crontab was returned - great stuff!
10. Let's practice!
Let's practice scheduling some Bash scripts with cron.