1. Queuing
Great job on those last exercises! Now that we've had an introduction to event-based processing, let's talk about another common processing method: queuing.
2. What is queuing?
A queue is something that most people have an inherent understanding of based on interacting in society.
A queue is a more technical name for a line.
Queues are extremely useful for processing information or tasks in order.
Queues (usually) have the characteristic of being first-in, first-out, also known as FIFO. This means that the data that is processed first is the first data provided to the system. This same structure is often referred to as a buffer. If you're familiar with data structures, queues are the opposite of a stack (last-in, first-out).
It's important to note that we're talking about queues in a general sense, rather than a specific implementation. The details can vary a lot based on how the queue is implemented.
3. Why queues?
You may wonder why would we use queues for data processing. Relying on physical world examples, we know that lines generally work well for organizing tasks into groups and in order. The same is true for data processing where we need to maintain the order of the data we're working with.
Using real-world examples, there can be a single process (ie, one cash register) open, or there can be multiple. The implementations of these may be a single long line serviced by multiple registers (possibly like a post office), or as multiple lines (such as a supermarket).
An important aspect of queues is they can be easily disconnected from the remainder of the processing pipeline. Consider in the post office example what happens if there are originally two workers helping customers but one needs to leave. The queue is maintained, it's likely just processed more slowly, as it's "disconnected" from one of the "workers" in the "pipeline".
As such, another attribute of queues is their easiness to scale, both vertically and horizontally.
To vertically scale a processing queue, you can obtain faster computer hardware and process more data in the same time.
To horizontally scale, you could add more process executors to work with each element.
4. Queue issues
Let's take a moment to mention some of the potential downsides of using queues.
The first issue is dealing with bad data or processing errors. Depending on the implementation, an error due to data processing can drastically slow the queue servicing, mainly due to everything being on hold until that instance is handled. Consider if a customer tries to pay with an invalid credit card - everyone must wait while this is handled.
Data size variances can make it difficult to properly service a queue (consider the supermarket fast lane if someone had 100 items). Optimizing queue sizes is its own area of expertise, but outside the scope of this course.
It can be difficult in a processing pipeline to know the actual length of the queue at any point in time. This makes it tricky to know the number of resources required to service it. Consider showing up to a movie's first preview showing. It's likely that when you join the queue, you can't even see the entrance and have no clue when you will be sitting in your seat.
This leads to another issue with scaling processes. As discussed in the previous chapter, there are inherent limits to vertical scaling of computing power. If you scale the number of queues, it can be tricky to handle potential scenarios (such as two long lines at a supermarket where one register must close - consider what happens to the people in the line that closed.)
5. Let's practice!
Queues are a staple of modern computing and data processing methods. Let's practice what you learned in the exercises ahead.