InstaSHAP: Interpretable Additive Models Explain Shapley Values Instantly

Could you provide more insights into why the high-dimensional evaluations were limited to the CUB bird dataset?

Adding evaluations on popular high-dimensional datasets in CV (like CIFAR-10 or ImageNet) and NLP (such as SST-2 or IMDB) would better showcase InstaSHAP’s performance… a discussion on InstaSHAP’s potential limitations or expected behavior on these kinds of datasets would be beneficial.

High-dimensional experiments were limited to the CUB dataset because we feel it should be completely representative of the larger phenomenon we identify which is underpinning a vast swath of the existing SHAP literature, particularly in high-dimensional CV/NLP datasets. This is in contrast to the GAM literature which has not seen widespread usage in similar high-dimensional tasks. Some of the closest existing attempts are [3] which do not operate on the input pixel feature in the same way as SHAP, but rather work with CNN extracted features, meaning such GAMs are not relevant for this work.

In terms of comparison with other CV datasets, our methods would easily extend and we would expect behavior to follow the trend of larger and more complex datasets like ImageNet having even more difficulties with applying GAM models, further proving our hypothesis.

In NLP, there is a major barrier to applying GAM methods because of variable length inputs. The common technique of handling these is through the transformer architecture; however, there do not yet exist attempts to merge a local GAM structure with a variable input architecture like the transformer. Although we expect the findings would be extremely similar. For instance, bag of words methods (similar to GAM 1 methods) and bigram methods (similar to GAM 2 methods) have long been out of date and cannot achieve modern accuracies.

…might feel overwhelming for readers who aren’t well-versed in the math without needing to go too deep into the technical details.

simpler examples to illustrate the variational formulation

The words “variational” we use throughout, for those without a theoretical background, in mathematics is similar in nature to “optimization” in computer science terms. “variational” is used to emphasize that the optimization is taking place over a functional space rather than the typical Euclidean space ($R^d$). That is to say KernelSHAP uses an optimization based definition of the Shapley value and FastSHAP uses a variational (functional optimization) based definition of the Shapley value. These are perhaps obvious to an expert in the related works, such as yourself; however, we have added flavor text to Section 4 to enhance its readability.

InstaSHAP relies on certain assumptions about feature correlations,

The authors simply cannot understand this question. There are absolutely no assumptions about the feature correlations in the dataset. This is the particular advantage of InstaSHAP when compared to all existing previous works, especially compared with those focusing on theoretical developments. Moreover, this is the key development when compared to the previous work [1]. Can the reviewer please clarify what assumptions are made about the input feature correlations and/or how other previous works somehow make less assumptions than our work?

how InstaSHAP would perform in comparison to other interpretability methods, such as LIME, Integrated Gradients, or counterfactual explanations?

The authors do not believe that additional comparison with LIME, IG, or counterfactual explanations would provide significant value to the work. In particular, when the reviewer says they are interested in the ``performance’’ of each method, can they elaborate on what is meant by this?

The paper points out that InstaSHAP has trouble scaling with higher-order interactions in high-dimensional data, which impacts its performance on tasks with complex dependencies, as seen in the bird classification experiment.

Do you see potential ways InstaSHAP could be adapted to handle these scenarios more effectively?

Yes, InstaSHAP is a GAM method. GAMs previously have never had success in these complex tasks like CV and NLP. This limitation is the key point we highlight in this work. By highlighting this fact, we provide an ultimatum: use something other than SHAP or adapt GAMs to handle these complexities. We see such potential ways to handle these scenarios as future work for the GAM literature enabled by our theoretical insights.

Including a section with practical guidelines, detailing ideal use cases or specific considerations for InstaSHAP’s application in real-world scenarios, would provide valuable context for practitioners.

We have added an additional subsection to the end of our theory section, Section 3, which focuses on the practical interpretation of the results as is highlighted in Figure 1.

[3] “Scalable Interpretability via Polynomials” Abhimanyu Dubey et al. 2022.

More diverse datasets (e.g., time series, medical) would better demonstrate robustness.

We provide additional experiments to the end of the appendix demonstrating our insights are consistent across more diverse tabular datasets including those coming from the medical and financial domains.

A broader comparison with methods like Integrated Gradients or LIME is needed.

Can the reviewer please explain in their own words why they believe a comparison with Integrated Gradients or LIME is necessary?

The paper lacks details on computational efficiency for large datasets. Quantitative analysis of runtime and latency is needed to validate real-time claims.

First, to be clear, no runtime performance is necessary to `validate’ the real-time claims. Real-time claims are in accordance with the paradigm introduced by FastSHAP and furthered by our work InstaSHAP. In that sense, real-time is in reference to the lack of test-time sampling required by previous SHAP approaches instead of the functional amortization approach. Nevertheless, benchmarks are provided for your convenience: a Resnet50 CNN takes 6.53 hours to finetune on CUB for 300 epochs; a InstaSHAP Resnet18 GAM takes 6.58 hours to finetune on CUB for 300 epochs, both on a 2080 Ti GPU.

The method's performance in non-GAM-friendly tasks needs clearer boundaries.

How does InstaSHAP manage complex, high-order dependencies?

Have you tested it in scenarios where GAMs typically underperform?

Yes, we have tested our method in scenarios where GAMs typically underperform. GAMs have never had success in high-dimensional CV tasks. CUB is a high-dimensional CV task. It is one of our main thesis points that this is a previously unidentified problem existing in the literature. If this is not sufficient explanation, can you please explain what is meant by you in this question?

Intuitive explanations alongside proofs would improve accessibility.

We have added a third subsection to our theory section which gives a brief summary of the results to be more easily interpreted by a practitioner.

Could you improve figure captions for clarity?

We have improved the captions of Figure 1 and the figure itself slightly. Hopefully, the caption will read more clearly. More specific feedback on what is meant by this suggestion would also be useful.

Are there plans for easy implementation or integration with SHAP tools?

Yes, our method will be made open source and integrated with common libraries to allow for ease of use.

Would you consider KernelSHAP or TreeSHAP?

We cannot easily consider TreeSHAP because it is an architecture-specific SHAP approach, and our target high-dimensional applications (e.g. CUB) cannot be solved easily by tree-based machine learning approaches. From a modern lens, however, TreeSHAP could be considered a special case of [1] and [1] could be considered a special case of our work (where the key insights is that for trees, the max depth automatically restricts the maximum size of feature interactions, $k$ in our work).

We can consider KernelSHAP and add it as another baseline like permutation sampling, although generally this approach has slightly fallen out of favor because its two stage procedure is more difficult to use than permutation sampling. These results are updated in Appendix F. Generally, we see that the performance tracks very closely to the permutation sampling approach, although appearing to be slightly more stable on this synthetic data.

Given that InstaSHAP uses purified GAM models to approximate Shapley values, are there cases where this might result in different Shapley attributions compared to traditional methods?

It is maybe first important to remember that there is only one Shapley value after fixing the model to explain; however, there are many various ways to approximate it. For the synthetic datasets where we benchmark different approximations, the methods we compare are almost always getting errors less than 1.0e-2. In that context, although our method is better and/ or faster at calculating the Shapley value, there is not a huge difference in the final attributions compared to traditional methods.

On the other hand, there is a large gap between the Shapley values of a blackbox model and the Shapley values of a GAM model. We specifically identify this on the high-dimensional CUB dataset. In Figure 6, you can see the Shapley values for a CNN (calculated with permutation sampling to ensure no GAM bias is induced) vs. the Shapley values for three different GAM models (calculated with InstaSHAP) have a significant gap in their explanations. This difference we theoretically identify as a fundamental limitation of Shapley, meaning that there must be information which is being destroyed by the Shapley value. Moreover, from a practitioner’s standpoint, seeing that the CNN and GAM-1x1 have different Shapley explanations on this dataset implies that the practitioner cannot completely trust the CNN Shapley values.