Feature based explanations: Model specific

1. Feature based explanations: Model specific

Let's look at model specific approaches, particularly, methods applicable to deep neural network models. The term model specific might sound inflexible, but in fact, can provide finer explanations for very complex models. The first method is integrated gradient or IG. IG is a feature-based post-hoc explanation method that aims to explain the relationship between the model's predictions in terms of its features. It is designed for differentiable models such as neural networks. IG primarily focuses on explaining individual predictions. Differentiability is crucial for gradient-based explanations like IG, as it allows us to compute gradients of the model's output with respect to its input features. Neural networks. A common type of differentiable model employed back propagation for training. Back propagation is an algorithm that efficiently propagates the error from the output layer back to the input layer, allowing the network to adjust its parameters in a way that minimizes the error. While back propagation focuses on gradients with respect to model parameters, interpretability techniques like IG require gradients with respect to input features. These gradients represent the sensitivity of the model's output to changes in a particular input feature. A larger gradient indicates a stronger influence of the corresponding feature on the model's output. In essence, the differentiability of the target model enables gradient-based explanations like IG to quantify the impact of individual input features on model predictions, providing valuable insights into the model's decision making process. Then why not visualize the gradients over the image? The image in the middle is the gradient mask of a DNN-based image classification model for a camera image. As you can see, the image appears too blurred to distinguish the classified camera object. Here lies the issue. As your model learns the relationship between the range of an individual pixel and the correct class, the gradient becomes increasingly small. It might even go to zero. This is called saturation. In comparison, on the right, notice how integrated gradients are much better at identifying the edges of the camera object. In particular, highlighting the pixels around the lens as being important. It captures a better representation of the camera that is more human interpretable. How do integrated gradients overcome gradient saturation? Let's walk through an example. Let's say, we had a model that correctly predicted this image as a fireboat. In IG, we first set a baseline image, usually, a completely black image and we generate a linear interpolation between the baseline and the original image. Interpolated images of small steps denoted as Alpha in the feature space. Rather than directly calculating gradients for the original image, we calculate the gradients for images along a path from the original image to a baseline image. Integrating these gradients yields the integrated gradients. But why do we need to integrate the gradients? To answer this, let's visualize the gradients from the previous step to connect theory to practice. The left plot shows how your model's confidence in the fireboat class varies across Alphas. Notice how the gradients or slope of the line largely flattens or saturates between 0.3 and 1.0 before settling at the final fireboat predicted probability of about 85%. The right plot shows the average gradient magnitudes over Alpha. Note how the values sharply approach and even briefly dip below zero after 0.2. In fact, your model learns the most even at lower values of Alpha. Intuitively, you can think of this as your model has learned that the pixels of water cannons to make the correct prediction is at 0.2 sending these pixel gradients to zero after that. However, the model is still uncertain and is focused on bridge or water jet pixels as the Alpha values approach the original input image. Here are the actual gradient maps on each Alpha interpolation, starting with a baseline of an all black image. We can see the important information like that of the water cannon is actually captured around Alpha 0.2. At Alpha 1, large gradients are only looking at the background. Now, we can integrate all the gradients. Practically, we just compute the Reimann sum approximation by using each Alpha interpolation because we don't know this function itself. Here is the result of integrated gradient showing the water cannon pixels that captured the fireboat class. However, note that the model is not looking at the boat itself. It's worth checking whether this model works well, even when a fireboat is not using a water cannon. Integrated gradient also has its drawbacks. One of the biggest difficulties is related to the baseline image. The implication of a black baseline is if black pixels are important to the prediction, they receive no attribution as a result, the choice of the baseline plays a central role in interpreting and visualizing IG's pixel feature importances. On the left, you can see how IG attributions completely miss the solid black beetle in the top image. On the right, you can see how changing the baseline to a white image corrects the interpretation of the IG attributions. How can we improve upon IG and its baseline selection problem? Through XRAI, an explainable AI method from Google Research. XRAI solve for the baseline selection problem. XRAI also added a novel region based attribution method for clear mass generation. Look at the image on the right. Instead of identifying individually important pixels, XRAI creates and highlights a region of the original image as important. XRAI overcomes the baseline selection problem by using a black and a white baseline image together to compute IG attributions. XRAI also improves upon integrated gradients IG by over-submitting the image into smaller regions and ranking them based on their IG positive attributions. This approach provides more localized and intuitive explanations for natural images. Although IG remains valuable for domains requiring fine-grained feature explainability, such as the medical image domain. XRAI excels in generating intuitive explanations for natural images.

2. Let's practice!

Create Your Free Account

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.