Creating wait time using Wget
For download smaller files, enforcing a mandatory wait time between file downloads makes sure we don't overload the server with too many requests. Here, we will using the built in option flag with wget to create a mandatory wait time (in seconds) between downloading each file stored in the URL list file.
This exercise is part of the course
Data Processing in Shell
Exercise instructions
- Create a mandatory 1 second pause between downloading all files in
url_list.txt.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# View url_list.txt to verify content
cat url_list.txt
# Create a mandatory 1 second pause between downloading all files in url_list.txt
wget ___ -i url_list.txt
# Take a look at all files downloaded
ls