Diving into buckets
1. Diving into buckets
In the last lesson, we learned about different AWS services and how to create our first Boto3 client. Now, let's dive into cloud storage with AWS S3. S3 lets us put any file in the cloud and make it accessible anywhere in the world through a URL. Managing cloud storage is a key component of data pipelines. Many services we will learn will depend on an object being uploaded to S3.2. S3 Components - Buckets
The main components of S3 are Buckets and Objects. Buckets are like folders on our desktop. Objects are like files within those folders. But there's a lot of power hidden underneath. Buckets have their own permissions policies. They can be configured to act as folders for a static website They can generate logs about their own activity and write them to a different bucket.3. S3 Components - Objects
The most important thing that buckets do - they contain objects. An object can be anything - an image, a video file, CSV or a log file. There are plenty of operations we can do with objects, but for now, let's focus on what we can do with buckets.4. What can we do with buckets?
What can we do with buckets using boto3? We can create a bucket. List buckets that we have in our account. And we can delete a bucket We can only store objects in buckets, so knowing how to work with buckets is a crucial component of S3 knowledge. Let's dive into some buckets!5. Creating a Bucket
Let's start off by making a new bucket called gid-requests. We create a boto3 client that lets us interact with AWS S3. Then, we call the client's create_bucket method, passing the bucket name as the argument.6. Bang!
Tada! We have a shiny new bucket.7. Our bucket in the console
We can see it in the console as well. Keep in mind that bucket names have to be unique across all of S3, otherwise we will get an error when trying to create one.8. Listing buckets
Now, that we can create a bucket, let's get a list of all the buckets we have in S3. Once again, we create a boto3 s3 client. Then we call the list_buckets method on the client.9. Listing Buckets
When S3 responds, it will give us some additional response metadata. But it will include a dictionary under the Buckets key. Let's get that dictionary, and print it out.10. Listing Buckets
We can see our new bucket name and the time that it was created. Now that we have this dictionary, we can run it through a for loop and perform an operation on multiple buckets.11. Deleting buckets
Let's say we don't need the gid-requests bucket anymore. Let's delete it. Once again, we create the boto3 s3 client. Then we call the delete_bucket method.12. Bye Bye Bucket
Alas, our bucket is gone. If we tried to delete it, and it didn't exist, we would've gotten an error.13. Bye Bye Bucket
It's nowhere to be found in the console either.14. Other operations
We will learn more operations on buckets as we get further in the course, but we won't learn them all. Get in the habit of reading boto3 documentation for all the methods we can do on an Amazon Web Service.15. Summary
In this lesson we learned about how to work with a key components of S3 - Buckets. We learned that buckets contain objects. How to create buckets. How to list buckets. And how to delete buckets. We also learned that there are more operations that we can read about in the boto3 docs.16. Let's practice!
Now we're ready to dive into buckets. Let's help Sam make some!Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.