DynamoDB indexes, streams, and optimization

1. DynamoDB indexes, streams, and optimization

Welcome back! In this video, we're exploring DynamoDB features that unlock more flexibility. You'll learn to create Global and Local Secondary Indexes for flexible querying, use DynamoDB Streams to capture data changes, set up Time-to- Live for automatic expiration, and leverage batch operations for efficiency. Let's get started.

2. The query that couldn't be done

You've designed an Orders table with the userId as partition key: perfect for user queries. But customer support needs to search by email. You can't - DynamoDB only queries using partition keys. Scan is expensive. The solution? Create a Global Secondary Index on email. This enables email queries without changing your table. Indexes enable multiple access patterns. Let's learn how.

3. Global Secondary Index (GSI)

A Global Secondary Index lets you query using a different partition key. It's 'global' because it spans all partitions. Create GSIs anytime - even years later. The trade-off? GSIs only support eventually consistent reads. This works for most use cases like email lookups or status queries. GSIs cost money because they're a copy of your data organized differently.

4. Local Secondary Index (LSI)

Local Secondary Indexes use the same partition key but let you sort by a different attribute. They're 'local' because they exist within a single partition. The catch? You must create LSIs at table creation - you can't add them later. The benefit? LSIs support strongly consistent reads, unlike GSIs. Use LSIs for multiple sort orders within a partition, like sorting orders by date, status, or total.

5. GSI vs LSI - when to use each

When do you use each? GSIs are more common - add them anytime and query on any attribute. Use GSIs to access data by different keys, like email or status. LSIs are less common because you must plan them upfront, but they're valuable for strongly consistent reads or multiple sort orders within a partition. Most applications use GSIs for additional access patterns.

6. Data serialization for DynamoDB

DynamoDB only stores strings, numbers, and binary data: not Python objects. To store complex data like user preferences, serialize it first. For dictionaries and lists, use JSON with json.dumps() and json.loads(). For binary data like images or tokens, use base64 encoding. Always deserialize when reading data back. This pattern applies to any database: convert application objects into storable formats.

7. DynamoDB Streams for change data capture

DynamoDB Streams captures every table change in an ordered log. Each record contains old and new item values. Use streams for audit logging, data replication, or triggers. When an order is placed, automatically send confirmation via Lambda. Streams enable event-driven architectures where applications react to data changes in real-time. Enable streams at table creation and process with Lambda.

8. Time-to-Live for automatic expiration

Time-to-Live automatically deletes items after a specified time: perfect for session data or tokens. Add a TTL attribute with a Unix timestamp, and DynamoDB deletes the item when that time passes. No code needed, no cost. For example, session tokens expire after 24 hours: set TTL to current time plus 86400 seconds. DynamoDB handles cleanup within 48 hours, reducing storage costs and improving security.

9. Index projection types and query performance

Index projection determines which attributes are copied to your index. KEYS_ONLY includes only keys: smallest storage but requires fetching from base table. INCLUDE projects specified attributes: balances size and performance. ALL projects every attribute: largest storage but no fetches needed. Choose based on access patterns. Storage costs scale with projection size.

10. Batch operations and transactions

Batch operations process multiple items in one request. BatchGetItem retrieves up to 100 items: faster than individual calls. BatchWriteItem writes up to 25 items. These are efficient but don't guarantee all-or-nothing execution. For ACID guarantees, use TransactWriteItems - all succeed or all fail. Perfect for consistency workflows like money transfers or inventory updates. Transactions cost twice as much.

11. Let's practice!

Now let's practice what we have learned!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.