Save yourself time when performing complex repetitive tasks. You’ll begin this course by refreshing your knowledge of common command-line programs and arguments before quickly moving into the world of Bash scripting. You’ll create simple command-line pipelines and then transform these into Bash scripts. You’ll then boost your skills and learn about standard streams and feeding arguments to your Bash scripts.

Introduction and refresher

Extracting scores with shell

Searching a book with shell

Your first Bash script

A simple Bash script

Shell pipelines to Bash scripts

Extract and edit using Bash scripts

Standard streams & arguments

Using arguments in Bash scripts

Using arguments with HR data

From Command-Line to Bash Script

Master the bread and butter of Bash scripts—variables! In this chapter, you’ll learn how to create basic string and numeric variables, and perform calculations on these variables. You’ll also learn about the magic of a shell-within-a-shell (shell-ception), opening up huge opportunities for advanced scripting.

Basic variables in Bash

# Create the required variable
____="____"

# Print out the assigned name (Help fix this error!)
echo "Hi there yourname, welcome to the website!"


# Create the required variable
yourname="Sam"

# Print out the assigned name (Help fix this error!)
echo "Hi there $yourname, welcome to the website!"


Using variables in Bash

Shell within a shell

Numeric variables in Bash

# Get first ARGV into variable
temp_f=____

# Subtract 32
temp_f2=$(echo "scale=2; $temp_f - ____" | bc)

# Multiply by 5/9
temp_c=$(echo "scale=2; $temp_f2 * ____ / 9" | bc)

# Print the celsius temp
echo ____


# Get first ARGV into variable
temp_f=$1

# Subtract 32
temp_f2=$(echo "scale=2; $temp_f - 32" | bc)

# Multiply by 5/9
temp_c=$(echo "scale=2; $temp_f2 * 5 / 9" | bc)

# Print the celsius temp
echo $temp_c


Converting Fahrenheit to Celsius

# Create three variables from the temp data files' contents
temp_a=$(____ temps/____)
temp_b=$(____ temps/____)
temp_c=$(____ temps/____)

# Print out the three variables
echo "The three temperatures were ____, $temp_b, and $temp_c"

# Create three variables from the temp data files' contents
temp_a=$(cat temps/region_A)
temp_b=$(cat temps/region_B)
temp_c=$(cat temps/region_C)

# Print out the three variables
echo "The three temperatures were $temp_a, $temp_b, and $temp_c"


Extracting data from files

Arrays in Bash

Creating an array

Creating associative arrays

# Create variables from the temperature data files
temp_b="$(cat temps/region_B)"
temp_c="$(cat temps/region_C)"

# Create an array with these variables as elements
region_temps=(____ ____)

# Call an external program to get average temperature
average_temp=$(echo "scale=2; (${____[0]} + ${____[1]}) / 2" | ____)

# Append average temp to the array
region_temps+=($____)

# Print out the whole array
echo ${region_temps[____]}


# Create variables from the temperature data files
temp_b="$(cat temps/region_B)"
temp_c="$(cat temps/region_C)"

# Create an array with these variables as elements
region_temps=($temp_b $temp_c)

# Call an external program to get average temperature
average_temp=$(echo "scale=2; (${region_temps[0]} + ${region_temps[1]}) / 2" | bc)

# Append to array
region_temps+=($average_temp)

# Print out the whole array
echo ${region_temps[@]}


Climate calculations in Bash

Variables in Bash Scripting

Life isn't always linear and neither should your Bash scripts. In this chapter, you’ll take your Bash scripts to the next level by learning control statements. You’ll learn the differences between FOR, WHILE, IF, and CASE statements and how to use them in your Bash scripts. Armed with these new tools, you’ll be ready to create more advanced Bash scripts with conditional logic. 

IF statements

Sorting model results

Moving relevant files

FOR loops & WHILE statements

A simple FOR loop

Correcting a WHILE statement

Cleaning up a directory

CASE statements

Days of the week with CASE

Moving model results with CASE

Finishing a CASE statement

Control Statements in Bash Scripting

Give your Bash scripts a powerful new addition by building useful functions to process data and variables. In this last chapter, you will learn the structure of functions in Bash, how to use functions to help automate frequent tasks, and program your scripts to run on schedule without needing to even lift a finger.

Basic functions in Bash

Uploading model results to the cloud

Get the current day

Arguments, return values, and scope

A percentage calculator

Sports analytics function

Summing an array

Scheduling your scripts with Cron

Analyzing a crontab schedule

Creating cronjobs

Scheduling cronjobs with crontab

Thanks and wrap up

Functions and Automation

New Hires

Soccer Scores

Hire Data

Model Results

Rob's Files

Model Output

Inherited Folder

Bash is a concise, superfast, and robust scripting language for data and file manipulation. It’s a vital skill for building analytics pipelines in the cloud, favored by Linux users to work with data stored across multiple files.

In this course, we’ll guide you through the basics of Bash scripting. We begin with an introduction to Bash script structures, including inputting arguments and outputting results. You’ll then work through data structures, such as variables and arrays, and control statements, including loops and conditionals. You’ll then put what you’ve learned into practice, by creating your own Bash functions and scheduling automated scripts to run like clockwork with cron.

Introduction to Shell

Learn how you can use Bash scripting to build automated scripts, create analytics pipelines in the cloud, and work with data stored across multiple files.

Introduction to Bash Scripting

Bash scripting allows you to build analytics pipelines in the cloud and work with data stored across multiple files.

Shell within a shell

Introduction to Bash Scripting

Hands-on interactive exercise