Save yourself time when performing complex repetitive tasks. You’ll begin this course by refreshing your knowledge of common command-line programs and arguments before quickly moving into the world of Bash scripting. You’ll create simple command-line pipelines and then transform these into Bash scripts. You’ll then boost your skills and learn about standard streams and feeding arguments to your Bash scripts.

Introduction and refresher

Extracting scores with shell

Searching a book with shell

Your first Bash script

#!/bin/bash

# Concatenate the file
____ ____.txt



#!/bin/bash

# Concatenate the file
cat server_log_with_todays_date.txt



A simple Bash script

#!/bin/bash

# Create a single-line pipe
cat ____.csv | cut -d "____" -f ____ | tail -n +2 | sort | ____ -c

# Now save and run!


#!/bin/bash

# Create a single-line pipe
cat soccer_scores.csv | cut -d "," -f 2 | tail -n +2 | sort | uniq -c

# Now save and run!


Shell pipelines to Bash scripts

#!/bin/bash

# Create a sed pipe to a new file
cat ____.csv | sed 's/____/____/g' | sed '____' ____ soccer_scores_edited.csv

# Now save and run!


#!/bin/bash

# Create a sed pipe to a new file
cat soccer_scores.csv | sed 's/Cherno/Cherno City/g' | sed 's/Arda/Arda United/g' > soccer_scores_edited.csv

# Now save and run!


Extract and edit using Bash scripts

Standard streams & arguments

# Echo the first and second ARGV arguments
echo ____ 
echo ____

# Echo out the entire ARGV array
echo ____

# Echo out the size of ARGV
echo ____


# Echo the first and second ARGV arguments
echo $1 
echo $2

# Echo out the entire ARGV array
echo $@

# Echo out the size of ARGV
echo $#


Using arguments in Bash scripts

# Echo the first ARGV argument
echo ____ 

# Cat all the files
# Then pipe to grep using the first ARGV argument
# Then write out to a named csv using the first ARGV argument
cat hire_data/____ | grep "____" > "____".csv


# Echo the first ARGV argument
echo $1 

# Cat all the files
# Then pipe to grep using the first ARGV argument
# Then write out to a named csv using the first ARGV argument
cat hire_data/* | grep "$1" > "$1".csv


Using arguments with HR data

From Command-Line to Bash Script

Master the bread and butter of Bash scripts—variables! In this chapter, you’ll learn how to create basic string and numeric variables, and perform calculations on these variables. You’ll also learn about the magic of a shell-within-a-shell (shell-ception), opening up huge opportunities for advanced scripting.

Basic variables in Bash

Using variables in Bash

Shell within a shell

Numeric variables in Bash

Converting Fahrenheit to Celsius

Extracting data from files

Arrays in Bash

Creating an array

Creating associative arrays

Climate calculations in Bash

Variables in Bash Scripting

Life isn't always linear and neither should your Bash scripts. In this chapter, you’ll take your Bash scripts to the next level by learning control statements. You’ll learn the differences between FOR, WHILE, IF, and CASE statements and how to use them in your Bash scripts. Armed with these new tools, you’ll be ready to create more advanced Bash scripts with conditional logic. 

IF statements

Sorting model results

Moving relevant files

FOR loops & WHILE statements

A simple FOR loop

Correcting a WHILE statement

Cleaning up a directory

CASE statements

Days of the week with CASE

Moving model results with CASE

Finishing a CASE statement

Control Statements in Bash Scripting

Give your Bash scripts a powerful new addition by building useful functions to process data and variables. In this last chapter, you will learn the structure of functions in Bash, how to use functions to help automate frequent tasks, and program your scripts to run on schedule without needing to even lift a finger.

Basic functions in Bash

Uploading model results to the cloud

Get the current day

Arguments, return values, and scope

A percentage calculator

Sports analytics function

Summing an array

Scheduling your scripts with Cron

Analyzing a crontab schedule

Creating cronjobs

Scheduling cronjobs with crontab

Thanks and wrap up

Functions and Automation

New Hires

Soccer Scores

Hire Data

Model Results

Rob's Files

Model Output

Inherited Folder

Bash is a concise, superfast, and robust scripting language for data and file manipulation. It’s a vital skill for building analytics pipelines in the cloud, favored by Linux users to work with data stored across multiple files.

In this course, we’ll guide you through the basics of Bash scripting. We begin with an introduction to Bash script structures, including inputting arguments and outputting results. You’ll then work through data structures, such as variables and arrays, and control statements, including loops and conditionals. You’ll then put what you’ve learned into practice, by creating your own Bash functions and scheduling automated scripts to run like clockwork with cron.

Introduction to Shell

Learn how you can use Bash scripting to build automated scripts, create analytics pipelines in the cloud, and work with data stored across multiple files.

Introduction to Bash Scripting

Bash scripting allows you to build analytics pipelines in the cloud and work with data stored across multiple files.

Extract and edit using Bash scripts

Introduction to Bash Scripting

Hands-on interactive exercise