Real-Time Intelligence with Eventhouse and KQL
1. Real-Time Intelligence with Eventhouse and KQL
Most analytics works with data that has already been collected, but some situations can't wait for tomorrow's report. Fabric's Real-Time Intelligence supports this kind of continuously changing data.2. Real-Time Intelligence in Fabric
Real-Time Intelligence is like Fabric's operations control room. Its tools help users receive, store, explore, and act on continuously arriving events, logs, and telemetry, which is useful when a nightly batch would be too slow.3. Event data and streaming data
Real-Time Intelligence focuses on two data types: event data and streaming data. An event is a single status update describing something that happened, such as a customer clicking a button or a sensor measuring a temperature. Streaming data is a continuous feed of such events over time, like live temperature measurements or live stock prices. Real-time systems work with both the individual event and the ongoing flow.4. Eventstream
This is where the Eventstream we saw earlier fits in. It receives messages from many sources, may transform or filter them, and routes them to the correct destination. Like the Eventhouse we're about to meet, it's created from the Real-Time Intelligence workload.5. Eventhouse
If the data captured by Eventstream needs to be analyzed in real time, Eventhouse is a suitable destination. It's built for time-series data, such as streams and events, where each value is attached to a timestamp. It ingests events continuously, makes them queryable within seconds, and integrates with OneLake.6. KQL queries
Eventhouse uses a language called KQL, which stands for Kusto Query Language, to query time-series data. Our first query reads SalesEvents, then vertical bar, called a pipe, take ten. It starts with the table name, SalesEvents. The pipe passes the table's rows to the next step, and take returns a sample of ten records. The take operator doesn't promise which ten, so when the newest events matter, we sort the data explicitly instead.7. KQL keywords and clauses
KQL does more than return records. Writing SalesEvents, pipe, count, returns the total number of records in the table. The summarize operator computes aggregations. Here, count() counts the events within each group, and bin groups the EventTime values into one-hour intervals, giving an event count per hour.8. Using sample data
The easiest way to get familiar with an Eventstream is to select one of the many sample datasets, which adds a source to the stream.9. Publishing an Eventstream
An Eventstream doesn't become operational until it's published. In edit mode it is only a design. Once published, it starts ingesting data from the source and delivering it to destinations such as an Eventhouse.10. Getting data for Eventhouse
A newly created Eventhouse is empty. We connect a data source to it from the Get data menu. It includes various sources, such as Eventstream, as well as a Sample option that creates a sample dataset.11. Batch or real-time?
Earlier in the course, we compared batch and real-time approaches for getting data into Fabric. Now that we've covered the real-time tools, let's revisit that choice. The key question is whether the data is time-sensitive and must be acted on as quickly as possible. A monthly maintenance report can wait for a batch process, but a live application error needs an immediate response. A nightly sales file suits a batch workflow, and structured financial records may fit a warehouse. Website clicks and application errors arrive continuously, so they need the real-time tools we've just covered.12. Let's practice!
Now let's practice identifying scenarios that need a live operations control room.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.