Save yourself time when performing complex repetitive tasks. You’ll begin this course by refreshing your knowledge of common command-line programs and arguments before quickly moving into the world of Bash scripting. You’ll create simple command-line pipelines and then transform these into Bash scripts. You’ll then boost your skills and learn about standard streams and feeding arguments to your Bash scripts.

Introduction and refresher

Extracting scores with shell

Searching a book with shell

Your first Bash script

A simple Bash script

Shell pipelines to Bash scripts

Extract and edit using Bash scripts

Standard streams & arguments

Using arguments in Bash scripts

Using arguments with HR data

From Command-Line to Bash Script

Master the bread and butter of Bash scripts—variables! In this chapter, you’ll learn how to create basic string and numeric variables, and perform calculations on these variables. You’ll also learn about the magic of a shell-within-a-shell (shell-ception), opening up huge opportunities for advanced scripting.

Basic variables in Bash

Using variables in Bash

Shell within a shell

Numeric variables in Bash

Converting Fahrenheit to Celsius

Extracting data from files

Arrays in Bash

Creating an array

Creating associative arrays

Climate calculations in Bash

Variables in Bash Scripting

Life isn't always linear and neither should your Bash scripts. In this chapter, you’ll take your Bash scripts to the next level by learning control statements. You’ll learn the differences between FOR, WHILE, IF, and CASE statements and how to use them in your Bash scripts. Armed with these new tools, you’ll be ready to create more advanced Bash scripts with conditional logic. 

IF statements

Sorting model results

Moving relevant files

FOR loops & WHILE statements

A simple FOR loop

Correcting a WHILE statement

Cleaning up a directory

CASE statements

Days of the week with CASE

Moving model results with CASE

Finishing a CASE statement

Control Statements in Bash Scripting

Give your Bash scripts a powerful new addition by building useful functions to process data and variables. In this last chapter, you will learn the structure of functions in Bash, how to use functions to help automate frequent tasks, and program your scripts to run on schedule without needing to even lift a finger.

Basisfuncties in Bash

# Create function
____ upload_to_cloud () ____
  # Loop through files with glob expansion
  for file in output_dir/____results____
  do
    # Echo that they are being uploaded
    ____ "Uploading $file to cloud"
  done
____

# Call the function
upload_to_cloud


# Create function
function upload_to_cloud () {
  # Loop through files with glob expansion
  for file in output_dir/*results*
  do
    # Echo that they are being uploaded
    echo "Uploading $file to cloud"
  done
}

# Call the function
upload_to_cloud


Modelresultaten uploaden naar de cloud

# Create function
_____ _____ {

  # Parse the results of date
  ____=$(date | cut -d " " -f1)

  # Echo the result
  echo ____
}

# Call the function
____


# Create function
what_day_is_it () {

  # Parse the results of date
  current_day=$(date | cut -d " " -f1)

  # Echo the result
  echo $current_day
}

# Call the function
what_day_is_it


Haal de huidige dag op

Argumenten, returnwaarden en scope

# Create a function 
____ return_percentage () ____

  # Calculate the percentage using bc
  percent=$(echo "scale=2; 100 * ____ / ____" | ____)

  # Return the calculated percentage
  echo $____
}

# Call the function with 456 and 632 and echo the result
return_test=$(____ ____ 632)
echo "456 out of 632 as a percent is $____%"


# Create a function 
function return_percentage () {

  # Calculate the percentage using bc
  percent=$(echo "scale=2; 100 * $1 / $2" | bc)

  # Return the calculated percentage
  echo $percent
}

# Call the function with 456 and 632 and echo the result
return_test=$(return_percentage 456 632)
echo "456 out of 632 as a percent is $return_test%"


Een procenten-rekenhulp

# Create a function
____ get_number_wins () ____

  # Filter aggregate results by argument
  ____=$(cat soccer_scores.csv | cut -d "," -f2 | egrep -v 'Winner'| sort | uniq -c | egrep "____")

____

# Call the function with specified argument
____ "____"

# Print out the global variable
echo "The aggregated stats are: ____"


# Create a function
function get_number_wins () {

  # Filter aggregate results by argument
  win_stats=$(cat soccer_scores.csv | cut -d "," -f2 | egrep -v 'Winner'| sort | uniq -c | egrep "$1")

}

# Call the function with specified argument
get_number_wins "Etar"

# Print out the global variable
echo "The aggregated stats are: $win_stats"


Sportanalyses-functie

# Create a function with a local base variable
function sum_array () {
  ____ sum=0
  # Loop through, adding to base variable
  for number in "$____"
  do
    sum=$(echo "$sum + $____" | bc)
  done
  # Echo back the result
  echo $____
  }
# Call function with array
test_array=(14 12 23.5 16 19.34)
total=$(sum_array "${____[@]}")
echo "The total sum of the test array is $____"


# Create a function with a local base variable
function sum_array () {
  local sum=0
  # Loop through, adding to base variable
  for number in "$@"
  do
    sum=$(echo "$sum + $number" | bc)
  done
  # Echo back the result
  echo $sum
  }
# Call function with array
test_array=(14 12 23.5 16 19.34)
total=$(sum_array "${test_array[@]}")
echo "The total sum of the test array is $total"


Een array optellen

Je scripts inplannen met Cron

Een crontab-schema analyseren

Cronjobs maken

Cronjobs plannen met crontab

Bedankt en afronden

Functions and Automation

New Hires

Soccer Scores

Hire Data

Model Results

Rob's Files

Model Output

Inherited Folder

Bash is een beknopte, supersnelle en robuuste scripttaal voor data- en bestandsbewerking. Het is een essentiële vaardigheid voor het bouwen van analysepijplijnen in de cloud, geliefd bij Linux-gebruikers om met data te werken die verspreid is over meerdere bestanden.

In deze cursus nemen we je stap voor stap mee door de basis van Bash-scripting. We beginnen met de structuur van Bash-scripts, inclusief het meegeven van argumenten en het tonen van resultaten. Daarna ga je aan de slag met datastructuren, zoals variabelen en arrays, en besturingsconstructies zoals lussen en voorwaarden. Tot slot pas je alles toe in de praktijk door je eigen Bash-functies te schrijven en geautomatiseerde scripts in te plannen die als een klok lopen met cron.

Introduction to Shell

Leer hoe je Bash-scripting gebruikt voor geautomatiseerde scripts en data-analyse in de cloud.

Introductie tot Bash-scripting

Met Bash-scripting kun je analytische pijplijnen in de cloud bouwen en werken met gegevens die in meerdere bestanden zijn opgeslagen.

Een crontab-schema analyseren

Introductie tot Bash-scripting

Praktische interactieve oefening