Identifying performance issues
1. Identifying performance issues
In this video, we’ll talk about the tools available to identify and resolve performance issues in a Fabric lakehouse or warehouse.2. Fabric capacities
Fabric capacities provide the computing power. Activities, from running a dataflow to executing a query, can be supported by one capacity. This simplifies deployment and licensing, because you only need to provision one capacity instead of having to provision and license multiple services. Computing power is measures in Capacity Units, or CUs.3. Fabric capacities
When you purchase Fabric, you specify the desired computing power SKU. If required, you can increase the capacity by upgrading to a larger SKU at any time.4. Fabric SKUs
Fabric SKUs, also known as F SKUs, vary from F2, the smallest, to F2048, the largest. The higher the number, the more computing power available to the capacity.5. Resource sizing challenges
The Fabric capacity model simplifies deployment, but there's the risk that if you underestimate your resource needs, you might exhaust the shared compute resources of a capacity. This will lead to poor performance.6. Resource sizing challenges
On the other hand, there's also the risk that you might overestimate your needs and pay for a larger capacity that you're not fully using.7. Automatic resource management
Out of the two resource usage scenarios, depleting resources is the more damaging one. Fabric uses some automatic mechanisms to avoid resource depletion. With Bursting, Fabric can temporarily exceed the capacity limit to execute a workload faster.8. Automatic resource management
With smoothing, Fabric spreads the extra capacity over a period of time.9. Automatic resource management
Throttling delays or rejects operations when a capacity is experiencing high demand of compute capacity.10. Monitoring capacity usage
Throttling affects performance, so you should monitor the capacity and detect if any throttling is occurring. The Fabric Capacity Metrics app allows you to monitor the usage of compute and storage resources.11. Fabric Capacity Metrics app
This app displays compute usage over time, the number of operations executed per hour, and number of users performing operations per hour.12. Monitoring capacity usage
It also displays the rate of jobs that are delayed or rejected due to throttling.13. Monitoring capacity usage
A table displays metrics for each item on the capacity. Here you can find the resource consumption by all items in the capacity, including lakehouses, warehouses, notebooks, and dataflows.14. Monitoring capacity usage
Finally, the storage page displays the OneLake storage used by the capacity's workspaces over time.15. Monitoring hub
The Monitoring Hub is another tool useful to monitor performance of Fabric. It provides more detailed information about performance of individual activities. While the Capacity Metrics app shows aggregated resource consumption over time, the Monitoring Hub shows the execution time of individual activities including data pipelines, dataflows, lakehouses, notebooks, semantic models and Spark jobs.16. Monitoring hub
On the Fabric portal, the Monitoring hub can be accessed by clicking on the Monitor icon on the left pane. It will display a list of activities, the success or failure status of their last run and statistics such as time of run and duration.17. Let's practice!
Now, let's do a couple of exercises to practice installing and using the Fabric Capacity Metrics app.Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.