1. Explainability and interpretability
The path to building reliable and trustworthy AI systems entails embracing the fundamental principles of explainability and interpretability. Let's dive deep into these concepts!
2. Explainability and interpretability
Explainability is the human ability to access and understand explanations for the decisions, predictions, or outputs returned by an AI system. An example of an explainable AI model is an ML classifier for penguin species in which you can easily find out why a certain penguin data observation has been classified as a Gentoo penguin.
Meanwhile, interpretability focuses on understanding how an AI system works from the inside, be it a specific algorithm, a trained model, or a series of data operations and transformations.
Both concepts are very similar, but explainability focuses on explaining outputs from inputs, and interpretability aims at unveiling the internal AI process in between.
3. White-box vs black-box AI systems
Based on their degree of interpretability or transparency, AI systems -particularly ML/DL models- can be classified into white-box and black-box models.
White-box systems and models are transparent and easily interpretable. This is usually the case of classical and relatively simpler ML models like linear regression.
4. White-box vs black-box AI systems
and decision trees. The last of which easily shows
5. White-box vs black-box AI systems
how a given combination of attributes' values in a new data observation leads to a specific predicted output, as you can see in this example of penguin classification.
6. White-box vs black-box AI systems
On the other hand, black-box AI systems and models are generally more complex and sophisticated than their white-box counterparts. Although in practice sophistication often relates to more effective and powerful models, effectiveness normally comes at the cost of sacrificing interpretability.
The best example is deep learning models
7. White-box vs black-box AI systems
with lots of layers of neurons, connections, and computational processes behind inferences, making it hard to understand why a particular prediction has been made.
8. Basic Explainable AI (XAI) tools
Making black-box models more transparent is the main subject of study in the field of Explainable AI (XAI), which encompasses methods and tools like:
Model introspection: looking inside model parameters to understand how it makes decisions.
9. Basic Explainable AI (XAI) tools
Model documentation, to share information about model architecture and design choices.
10. Basic Explainable AI (XAI) tools
And model visualizations, to represent insights about model outputs and data fed into it in a human-friendlier manner. Heatmaps are a popular example of data and model visualization tools.
11. XAI tools: feature importance
Feature importance takes center stage as an Explainable AI tool.
It measures the impact or contribution each input data feature or predictor has on your model outputs, helping understand how models of varying complexity make decisions. It also helps detect and remove biases and other factors that may lead to undesired model performance.
The importance of a feature also indicates how much the model performance may deteriorate if that feature were removed from the dataset.
SHAP is a powerful visualization toolbox, not only capable of describing the importance of features for a model as a whole, like in this example of a regression model to estimate red wine quality based on chemical properties.
12. XAI tools: feature importance
In numerical predictive problems like regression, we can also investigate the positive and negative relationships between predictor features and the target output.
13. XAI tools: feature importance
We can even get explanations for a single prediction on a data observation, which is often the most illustrative way to convince your audience about your model's effectiveness.
14. XAI tools: feature importance
Making it possible to explain how the attribute values in a given observation contributed strongly or weakly, positively or negatively, to the resulting prediction value.
15. Practical implications of XAI
Let's conclude by summarizing other practical implications of incorporating XAI practices into AI solutions.
Explainable AI contributes to algorithmic transparency: understanding how algorithms process data and make decisions.
Local interpretability relates to understanding a system's behavior for a specific prediction, whereas global interpretability describes its overall behavior on a dataset or real problem.
Explainability and interpretability are crucial in addressing ethical AI concerns, which we will further explore later in this chapter.
And finally, they are crucial in promoting reliable human-AI collaboration based on trust and feedback.
16. Let's practice!
Now, let's put our newfound knowledge of explainable and interpretable AI to the test through some practical activities.