Uploading model results to the cloud

You are working as a data scientist on building part of your machine learning cloud infrastructure. Your team has many machine learning experiments running all the time. When an experiment is finished, it will output the results file and a configuration file into a folder called output_dir/.

These results need to be uploaded to your cloud storage environment for analysis. Your task is to write a Bash script that contains a function which will loop through all the files in output_dir/ and upload the result files to your cloud storage.

For technical reasons, no files will be uploaded; we will simply echo out the file name. Though you could easily replace this section with code to upload to Amazon S3, Google Cloud or Microsoft Azure!

This exercise is part of the course

Introduction to Bash Scripting

View Course

Hands-on interactive exercise

Turn theory into action with one of our interactive exercises

Start Exercise

This exercise is part of the course

Introduction to Bash Scripting

IntermediateSkill Level

4.7+

68 reviews

Start Course for Free

Save yourself time when performing complex repetitive tasks. You’ll begin this course by refreshing your knowledge of common command-line programs and arguments before quickly moving into the world of Bash scripting. You’ll create simple command-line pipelines and then transform these into Bash scripts. You’ll then boost your skills and learn about standard streams and feeding arguments to your Bash scripts.

Exercise 1: Introduction and refresher Exercise 2: Extracting scores with shell Exercise 3: Searching a book with shell Exercise 4: Your first Bash script Exercise 5: A simple Bash script Exercise 6: Shell pipelines to Bash scripts Exercise 7: Extract and edit using Bash scripts Exercise 8: Standard streams & arguments Exercise 9: Using arguments in Bash scripts Exercise 10: Using arguments with HR data

Master the bread and butter of Bash scripts—variables! In this chapter, you’ll learn how to create basic string and numeric variables, and perform calculations on these variables. You’ll also learn about the magic of a shell-within-a-shell (shell-ception), opening up huge opportunities for advanced scripting.

Exercise 1: Basic variables in Bash Exercise 2: Using variables in Bash Exercise 3: Shell within a shell Exercise 4: Numeric variables in Bash Exercise 5: Converting Fahrenheit to Celsius Exercise 6: Extracting data from files Exercise 7: Arrays in Bash Exercise 8: Creating an array Exercise 9: Creating associative arrays Exercise 10: Climate calculations in Bash

Life isn't always linear and neither should your Bash scripts. In this chapter, you’ll take your Bash scripts to the next level by learning control statements. You’ll learn the differences between FOR, WHILE, IF, and CASE statements and how to use them in your Bash scripts. Armed with these new tools, you’ll be ready to create more advanced Bash scripts with conditional logic.

Exercise 1: IF statements Exercise 2: Sorting model results Exercise 3: Moving relevant files Exercise 4: FOR loops & WHILE statements Exercise 5: A simple FOR loop Exercise 6: Correcting a WHILE statement Exercise 7: Cleaning up a directory Exercise 8: CASE statements Exercise 9: Days of the week with CASE Exercise 10: Moving model results with CASE Exercise 11: Finishing a CASE statement

Give your Bash scripts a powerful new addition by building useful functions to process data and variables. In this last chapter, you will learn the structure of functions in Bash, how to use functions to help automate frequent tasks, and program your scripts to run on schedule without needing to even lift a finger.

Exercise 1: Basic functions in Bash Exercise 2: Uploading model results to the cloud

Current Exercise

Exercise 3: Get the current day Exercise 4: Arguments, return values, and scope Exercise 5: A percentage calculator Exercise 6: Sports analytics function Exercise 7: Summing an array Exercise 8: Scheduling your scripts with Cron Exercise 9: Analyzing a crontab schedule Exercise 10: Creating cronjobs Exercise 11: Scheduling cronjobs with crontab Exercise 12: Thanks and wrap up