1. Intro to event-based computing
Glad you're back! We've just learned about batch processing, its advantages and disadvantages, and hinted at other potential options. In this chapter, we'll be introducing the ideas behind streaming data processing. Let's get started by learning about event-based computing.
2. Old days
When computers were originally put in production, they were a shared resource. The systems were expensive enough that multiple people had to use them to make it worth the cost.
To make this process work, the tasks were batched, in much the same way as we described our batch data processing.
A user would prepare their program, provide it to the operator (often on punch cards!), and wait for the result. Once complete, the results were returned to the user.
While this option worked reasonably well, there were problems with delays (such as when did a task need to be turned in and how long until it was complete?), missing or incorrect results, and so forth. This probably sounds similar to some issues we have with batch data processing.
3. Personal computers
The era of personal computers changed a few things, such as computers now being a single user machine (vs the time shared machines of the past).
But even though there were now fewer users, most programs still operated in a batching fashion.
Users would run programs which were basically sets of tasks executed in order as given to the machine.
Once graphical user interface (GUI for short) based systems such as the Macintosh and Microsoft Windows came on the scene, programmers realized there was a need for a different paradigm. This gave rise to event-based processing.
4. Event-based processing
Unlike batch processing, event-based processing does not run at a specific time (such as 10am).
Tasks instead run when a specific event occurs. An event in this case is a general term that means "something" happened to cause the process to occur.
This could be a user clicking a button in a GUI,
or it could be a new file that finished uploading to a directory on a server. It's important to note that an event in this case is largely a general concept that can be applied to many specific uses.
An event-based process can have components that are batched (ie, user clicked a button to start the batch) but it is not a traditional batch scenario.
The biggest takeaway about event-based systems is that they are waiting for something to occur, without knowing exactly what or when that might be.
5. Example event-based task
We'll discuss specific details about event-based architectures and streaming later on, but it's often easier to consider an example to start. Let's look at web click-stream monitoring, a form of analytics for web data.
In this case, user activity is based on the user (or users) clicking on links or components of a webpage.
Once this event occurs, the client application then determines what resources are required and requests these from a server.
The server then returns the appropriate information and usually logs the requests.
The requests (or user events) are often sent to a central location for storage and analysis. This could consist of troubleshooting, or looking for trends in user activity.
6. Let's practice!
We've just introduced event-based processing. Let's test your knowledge in the exercises ahead.