Scheduling cronjobs with crontab

As the resident data scientist, you have been asked to automate part of a data processing pipeline. You will use one of the cronjob schedules created in a previous exercise.

You will use the built in crontab functionality of this workspace to schedule a data pipeline script, extract_data.sh to run at 30 minutes past 2am every day. Running early in the morning saves a great amount of cost by utilizing cheaper server power.

You will end up in the nano text editor where you save a file with ctrl + o (press enter) and exit with ctrl + x. Useful Nano documentation is here.

At any stage you can refresh your browser window and you will get a fresh workspace to try again if you accidentally make a mistake in the workspace.

This exercise is part of the course

Introduction to Bash Scripting

View Course

Hands-on interactive exercise

Turn theory into action with one of our interactive exercises

Start Exercise

This exercise is part of the course

Introduction to Bash Scripting

IntermediateSkill Level

4.7+

68 reviews

Start Course for Free

Save yourself time when performing complex repetitive tasks. You’ll begin this course by refreshing your knowledge of common command-line programs and arguments before quickly moving into the world of Bash scripting. You’ll create simple command-line pipelines and then transform these into Bash scripts. You’ll then boost your skills and learn about standard streams and feeding arguments to your Bash scripts.

Exercise 1: Introduction and refresher Exercise 2: Extracting scores with shell Exercise 3: Searching a book with shell Exercise 4: Your first Bash script Exercise 5: A simple Bash script Exercise 6: Shell pipelines to Bash scripts Exercise 7: Extract and edit using Bash scripts Exercise 8: Standard streams & arguments Exercise 9: Using arguments in Bash scripts Exercise 10: Using arguments with HR data

Master the bread and butter of Bash scripts—variables! In this chapter, you’ll learn how to create basic string and numeric variables, and perform calculations on these variables. You’ll also learn about the magic of a shell-within-a-shell (shell-ception), opening up huge opportunities for advanced scripting.

Exercise 1: Basic variables in Bash Exercise 2: Using variables in Bash Exercise 3: Shell within a shell Exercise 4: Numeric variables in Bash Exercise 5: Converting Fahrenheit to Celsius Exercise 6: Extracting data from files Exercise 7: Arrays in Bash Exercise 8: Creating an array Exercise 9: Creating associative arrays Exercise 10: Climate calculations in Bash

Life isn't always linear and neither should your Bash scripts. In this chapter, you’ll take your Bash scripts to the next level by learning control statements. You’ll learn the differences between FOR, WHILE, IF, and CASE statements and how to use them in your Bash scripts. Armed with these new tools, you’ll be ready to create more advanced Bash scripts with conditional logic.

Exercise 1: IF statements Exercise 2: Sorting model results Exercise 3: Moving relevant files Exercise 4: FOR loops & WHILE statements Exercise 5: A simple FOR loop Exercise 6: Correcting a WHILE statement Exercise 7: Cleaning up a directory Exercise 8: CASE statements Exercise 9: Days of the week with CASE Exercise 10: Moving model results with CASE Exercise 11: Finishing a CASE statement

Give your Bash scripts a powerful new addition by building useful functions to process data and variables. In this last chapter, you will learn the structure of functions in Bash, how to use functions to help automate frequent tasks, and program your scripts to run on schedule without needing to even lift a finger.

Exercise 1: Basic functions in Bash Exercise 2: Uploading model results to the cloud Exercise 3: Get the current day Exercise 4: Arguments, return values, and scope Exercise 5: A percentage calculator Exercise 6: Sports analytics function Exercise 7: Summing an array Exercise 8: Scheduling your scripts with Cron Exercise 9: Analyzing a crontab schedule Exercise 10: Creating cronjobs Exercise 11: Scheduling cronjobs with crontab

Current Exercise

Exercise 12: Thanks and wrap up