Creating wait time using Wget
For downloading smaller files, enforcing a mandatory wait time between file downloads makes sure we don't overload the server with too many requests. Here, we will be using the built in option flag with wget to create a mandatory wait time (in seconds) between downloading each file stored in the URL list file.
This exercise is part of the course
Data Processing in Shell
Exercise instructions
- Create a mandatory 1 second pause between downloading all files in
url_list.txt.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# View url_list.txt to verify content
cat url_list.txt
# Create a mandatory 1 second pause between downloading all files in url_list.txt
wget ___ -i url_list.txt
# Take a look at all files downloaded
ls