Feature based explanations: Model agnostic
1. Feature based explanations: Model agnostic
>> Lets look more in depth into each model. Agnostic approach. First, feature based explanations. Again, feature-based explanations provide attribution scores, masks, or some visualizations that highlight the importance of each input feature for a given task. What this looks like depends on the type of data youre using. For each data modality of images, tabular data, and text, feature attributions look different for images. Feature attributions will show you which pixels or regions within an image most contributed to the model's classification. For example, for this image of a husky, it looks like the model is focusing mostly on the Husky's nose and facial features, which is likely what we would hope for with a model seeking to classify dog breeds for tabular data. Feature attributions tell you how much each feature column contributed to either a specific prediction or the model overall. Some methods just return the magnitude of impotence as positive values, while others can give you directions of feature contribution to outputs like positive or negative contribution. For text, feature attributions tell you how much each word or token in a sentence or utterance contributing to the sentiment score. For example, in this sentence, the cake tastes delicious. Feature attributions could show you that the high sentiment score of 0.9 is largely attributable to the word delicious, which had a strongly positive influence on the text classification, whereas the other words in the phrase had significantly less influence. Let's look at some popular feature-based explanation techniques. There are global techniques like permutation feature importance and partial dependence plots. And local methods such as LIME, Shapley values, integrated gradients, and XRAI. First, permutation feature importance this is a pulse hoc global model agnostic method that measures the impact of each feature on a model's performance. It works by randomly shuffling the values of a single feature and observing the resulting change in the model's error rate. The higher the increase in error, the more important the feature is considered to be. Here is an overview of the process. For each feature, randomly shuffle its values while keeping all other features unchanged. Use the shuffle feature values to generate predictions. Calculate the error rate of the model with the shuffle feature. Compare the new error to the original error rate to determine how much the model's performance was affected by shuffling the feature. Based on this, then, a feature is considered important if randomly shuffling and increases the model error. This indicates that the model relied heavily on that feature for making accurate predictions. A feature is considered unimportant if randomly shuffling its values results in little to no error differences. This indicates the model wasn't heavily influenced by the feature in the first place. So, in summary, the importance of a feature is determined by the magnitude of the error increase caused by shuffling its values. Permutation feature technique can be intuitive and is easy to implement, but sometimes it can be misleading. Randomly shuffling a feature can create artificial patterns in the data and potentially cause misleading results. Additionally, the random nature of this method can cause the importance scores to vary significantly across different trials. Partial dependence plots, or PDPs, are a post-hoc, global, and model agnostic technique used to visualize the relationship between a model's predictions and the values of specific input features. They show how the models predictions change as we vary the values of one input feature while holding all other features constant. More specifically, the partial dependence function at a particular feature value represents the average prediction if we force all data points to assume that feature value. PDPs often visualize feature distributions in plots, which can show areas in feature space with limited data. While it is possible to visualize PDPs from multiple feature interactions, humans can have difficulty interpreting high dimensional visualizations. Therefore, PDPs are typically limited to two interacting features, one on each axis. To do this, choose the input feature you want to analyze systematically. Change the values of the selected feature while keeping all other features constant. For each feature value, compute the average predicted outcome across all data points. This represents the partial dependence of the prediction on that feature value. Plot the feature values on the x axis and the corresponding average predictions on the y axis. This visualizes the partial dependence relationship. PDPs provide a valuable tool for understanding the relationship between input features and model predictions. They can help identify important features, detect nonlinear relationships, and uncover potential biases in the model. Now, let's take a look at local methods. First, is LIME which stands for local interpretable model agnostic explanations. This has been popular for a number of years. LIME is post-hoc, local and model agnostic. LIME creates an explanation by approximating the underlying model locally with an interpretable one. We usually use a linear model or a decision tree. The process for the LIME method looks like this. Select your instance of interest. For example, an image of a tree frog in image case. Split the image into superpixels which are a subset of segments. Generate perturbations of the input image by hiding some features or image superpixels. For each perturbation, use a target model such as a deep neural network model we want to interpret in order to predict the output. Construct a new dataset where each input represents the presence or absence of a feature and the corresponding output is the predicted probability of the perturbation being a frog. Calculate weights for each perturbation based on a distance metric. This typically measures the difference between the perturbed image and the original image, such as the cosine distance. On the dataset constructed in the step above, train an intrinsically interpretable model, such as a decision tree or linear regression. Using the newly constructed dataset of perturbations, interpret the trained model to identify which areas of the image have a stronger association with the prediction of a tree frog. This can be achieved by examining the model's coefficients or decision rules to understand how it weighs different features in the perturbed images. Interpreting a simpler surrogate model instead of a complex one is an intriguing approach. However, the perturbation procedure varies depending on the data type. For images, replace superpixels with meaningless values such as gray values. For text, substitute words with a special token like an unknown word token to eliminate meaning. For tabular data, sample data from a feature distribution and calculate weights based on similarity instead of hiding information. Next, let's talk about Shapley values. Shapley values are a concept from cooperative game theory. It has been adapted to explain the contribution of individual features to a machine learning model's predictions. In game theory, Shapley values are used to distribute the payoff of a cooperative game among the players based on the magnitude of the contribution to the game. In machine learning, the players are the features and the payoff is the attribution score for the model's prediction. Shapley values are a post-hoc technique. For an understanding of this, let's look at an example. Imagine Sangita, Tomio, and Devon form a team for an e-sports competition, collectively winning $1,000. Recognizing their varying skill levels, they decided to split the prize based on their individual contributions to the game. Instead of an equal division, Sangita, a mathematician, proposes using Shapley values to quantify each players contribution. They create a gaming simulator and repeatedly play the same game with different team combinations, resulting in varying prize amounts. The Shapley value method for this scenario involves changing the combination of featured teams, marginalizing the impact of other features, and calculating the average prediction. The result tablet is rearranged as a path to add players one by one in different orders. In this case, there are six possible paths based on the previous steps. Result table the result difference is calculated for each cell along each path. In this case, all differences are positive, indicating that adding a player increases the overall price. However, in machine learning predictions, differences can be negative, reflecting the influence of features that push predictions towards smaller values. By averaging the result gaps when a player joins a team, we can compute the Shapley value, direction, and magnitude of each player's contribution. Sangita's Shapley value is 400 higher than Tomio and Devon's. A key characteristic of Shapley values is their additivity. Thanks to this property, Sangita, Tomio, and Devon's Shapley value sum up to $1,000, the amount they earn together. This allows them to easily distribute the prize based on their respective Shapley values. Although Shapley values offer a mathematically sound approach to explaining feature importance, their computational complexity poses a practical challenge. Calculating Shapley values for a large number of features becomes computationally challenging as the number of features increases exponentially. For example, computing Shapley values for 1000 features requires running a simulation or making predictions equal to two to the power of 1000 times, which is simply not feasible. So should we avoid using this method if we have many features? Fortunately, many efficient approximation algorithms of Shapley values are available, such as Sampled Shapley, Kernel SHAP, and Tree SHAP. Each has slightly different approximation strategies and applications, but are all widely used in many machine learning interpretation use cases.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.