Decoupling using AWS Messaging Services
1. Decoupling using AWS Messaging Services
In this video we cover how to develop applications using messaging services.2. Amazon SQS fundamentals
Amazon Simple Queue Service (SQS) is a managed message queue service that allows application components to exchange messages asynchronously. Producers send messages into a queue and consumers retrieve messages. Messages are retained in SQS until a consumer picks it up. The default retention is 4 days, maximum is 14 days. Consumers poll the queue and process messages at their own rate. This helps applications handle bursts of traffic and process workloads independently of message producers.3. Message visibility
When a consumer picks up a message, it becomes invisible to other consumers. The Visibility timeout determines how long a message remains hidden after being read by the consumer. If processing fails and the message is not deleted, it becomes visible again for reprocessing. Visibility timeout default is 30 seconds, maximum is 12 hours4. Message size
The max size for a message is 256KB. SQS Extended Client library allows you to send larger messages by storing the payload in S3 and sending a pointer in the SQS message.5. Short polling
SQS supports two polling strategies that directly impact both cost and efficiency of collecting messages. Short Polling (the default) queries a subset of SQS servers and returns immediately. It may return empty responses even when messages are available. It results in more API calls and higher cost.6. Long polling
Long Polling queries all SQS servers and waits up to 20 seconds for a message. It eliminates unnecessary empty responses, and reduces API calls and lowers cost.7. Standard queues
Queues come in two flavors. Standard Queue provide at-least-once delivery, best-effort ordering, and nearly unlimited throughput.8. FIFO queues
FIFO Queue provide exactly-once processing, strict message ordering, and de-duplication support. The trade-off is that FIFO queues support 300 TPS by default. You can increase this in two ways. Using message batching, you can reach 3000 TPS. You can enable FIFO high-throughput mode to get much higher TPS depending on AWS region. FIFO queues use message group IDs to preserve ordering and de-duplication IDs to avoid processing duplicate messages. Standard queues prioritize scale and throughput, while FIFO queues prioritize ordering and duplicate prevention.9. Handling poison messages
A dead letter queue is a separate SQS queue that receives messages which fail processing beyond a configured threshold. When a message is received more times than the defined maxReceiveCount, SQS automatically moves it to the DLQ rather than retrying indefinitely. DLQs must be the same queue type as the source queue. DLQs do not automatically reprocess messages and you must explicitly move them back to the source queue once the underlying issue is resolved.10. Securing queues
SQS uses HTTPS by default for encryption in transit. SQS supports server-side encryption (SSE) using AWS KMS. Producer (client) should encrypt using the open source AWS Encryption SDK. IAM policies control which identities can interact with SQS queues. You can grant or restrict actions such as sqs:SendMessage, sqs:ReceiveMessage, and sqs:DeleteMessage at both the identity level and queue level using resource-based policies.11. Amazon SNS fundamentals
Amazon Simple Notification Service (SNS) is a managed publish-subscribe service. SNS uses a push model. Publishers (Event Producer) send messages to a topic. The maximum SNS message size is 256KB per message. Consumers listen (subscribe) to a topic, and receive messages passively, without needing to request them.12. Scale
You can have up to 12.5 million subscriptions per topic. SNS does not retain messages; if a subscriber is unavailable, and no DLQ or durable subscriber (eg SQS queue) is configured, the message can be lost. SNS allows applications to distribute a single event to multiple consumers simultaneously.13. Message filtering
SNS supports message filtering policies. SNS filter policies allow subscriptions to receive only the messages they care about, rather than every message published to a topic. Without a filter policy, a subscription receives all messages. Filter policies are applied at the subscription level, not the topic level.14. Filter logic and evaluation
Filters are defined as JSON and matched against message attributes Multiple conditions in a filter policy are evaluated as AND logic. Multiple values within a single condition are evaluated as OR logic. Filter operations include exact string or numeric matching, prefix matching, anything-but (exclusion) and numeric range matching.15. SNS security
SNS uses HTTPS by default for all message delivery. SNS encrypts data at rest via SSE (supported via AWS KMS). You must ensure KMS key permissions to allow SNS to use the key. IAM policies enables you to control which identities can publish to or subscribe from a topic.16. Let's practice!
Time to test your SQS and SNS knowledge.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.