Queue Storage overview

1. Queue Storage overview

In this video, you’ll learn to integrate Azure Queue Storage to build asynchronous workflows.

2. Introduction to message queues

You’ll know how to create and configure a queue, use it for delayed or batched processing, trigger applications from queue messages, and explain how a message-based system differs from an event-based one.

3. Message queues overview

Think of a message queue as a to-do list for your cloud. When your app gets a spike of work, for example, when thousands of images upload, you don’t want to block users while you process each file. Instead, you drop lightweight messages into Azure Queue Storage and let background workers process them at their own pace.

4. Queue Storage overview

In Azure, the simplest way to set up message queues is by using Queue Storage. The applications that publish messages to the queue and the applications that read such messages can connect to a specific message queue via its connection string. Messages are processed in the same order as they've been added to the queue.

5. Queue triggers

Each message from the queue is picked up by a queue trigger. Once its been processed (i.e. the trigger ran to completion and no errors were thrown), it disappears from the queue so the next message can be processed. You would point the trigger at your queue using the connection string, and the piece of code, a function or a method, will keep firing automatically as long as there are still messages in the queue.

6. Error handling

But what happens if the queue trigger doesn't manage to process the message and crashes in the process, returning an error? In this case, the message still disappears from the queue, but it doesn't just vanish. It goes into the so-called poison queue. This unblocks the main queue while a failed message can be processed later. It's similar to how a broken-down car is taken to a lay-by so it doesn't block the main highway.

7. Delaying and batching

Queue storage also supports delaying messages. For example, imagine you are setting up a follow-up email to be sent X number of hours after your original email. You can schedule a message by setting an initial invisibility delay when you add it to the queue. For batching, consumers can read multiple messages at once and process them together to improve throughput, which allows a lot of data to be processed at once, speeding up the processes.

8. Messages vs events

So how is this different from event-based systems like Event Grid? In a message-based workflow, a message is a unit of work intended for a specific consumer. It’s pull-oriented, meaning that workers retrieve messages and mark them complete when done. In an event-based system, an event is a fact that something has happened. It’s push-oriented, meaning that publishers raise events, and multiple handlers can react independently. Use queues when you need reliable background work, ordering within a partition, retries, and back-pressure. Use events when you need to notify external services that something has happened.

9. Let's practice!

Let's see how this is applied in practice.

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.