WASUP: Interpretable Classification with Weight-Input Alignment and Class-Discriminative SUPports Vectors

The paper is significantly too long and can not be judged in 10 pages. Only the appendix contains results on multiple datasets and the theoretical evaluations. A significant part of the paper, pretty much the entire results section, are not understandable without looking at the appendix. Therefore, the presentation and soundness are "poor". If the paper length was 16 pages, they would be "fair" soundness and "good" presentation.

The novelty is fairly limited, as it combines B-cos networks with prototypical methods, for which the sota methods are backbone-independent. Specifically, (1) with b-cos backbone would be fairly similar.

While there is a section for it (l. 311), I am not sure what the global explanation looks like.

The writing could be polished in these lines: 118, 190, 315 (unclear), 467

Using support-features and training the model that way most likely causes uninterpretable not human-like similarities to emerge, as measured in the Hoffman et al. paper cited, or dicussed in (2), since the features are already encoding the classification. This is also evident in e.g. 4.c, in which plant soil is similar to the flower, or in the spurious correlation of the keyboard in 4d. A human study as in the Hoffman paper would be beneficial to test if a human would be able to predict the similarities.

As noted by the authors in the appendix, a more thorough investigation of the result with random seeds would be beneficial.

The evaluation is missing the default datasets for interpretable image classification, CUB and StanfordCars, and a comparison to competitors, such as the vanilla b-cos model, or other interpretable models with / without b-cos backbone such as PIP-Net, ProtoPool(1) or Q-SENN(2).

(1) Rymarczyk, Dawid, et al. "Interpretable image classification with differentiable prototypes assignment." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022. (2) Norrenbrock, Thomas, Marco Rudolph, and Bodo Rosenhahn. "Q-senn: Quantized self-explaining neural networks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 19. 2024.

→Images used to construct support vectors: It is not clear to me how you select the initial images that generate the initial support vectors. I believe that randomly selecting images in the beginning from a class could lead to a lack of variability if they are not well sampled, and this could affect the learning process. What happens if all the images selected from the dataset are similar and provide a single type of support vector, or what happens if the dataset used to sample the images is biased? I think it would be interesting to see results from different initially selected subsets of the dataset to construct these vectors. I would also suggest using a biased dataset to show what happens when we sample from a biased training set. Can we see the bias in the resulting prototype images?

You might also include some experiments on: Number of images to construct the support vectors; how the support vectors change when you change the set of images; what each support vector represents (semantically).

Also, during the training process you mention that you use k-means to extract the centroids, it would be interesting to see experiments with k-means parameters. I imagine the exact number of classes as k is not enough to describe all the possible differences within each class. What was the best k in this process?

→ Global explanations: as you mentioned, if nothing is similar to a sample, it will show the most similar support vector (even if it is not similar) with smaller contribution. But what happens if the image is far from all the support vectors? Can we say that the model does not know it or that we did not construct the support vectors correctly?

As I understood, we could analyze the output space of the model to have the proximity on an input image to others in the dataset. But how do you analyze the global behavior of the model? Is it correct or biased? It would be interesting to see some examples of global explanations (not just sample-based explanations).

→ I don't understand exactly why you don't use the original vector representation and only the positive-valued vectors. What happens without the relu? Also, why can we not have negative attributions? Doesn't that mean that this feature is not present in the class according to the model's knowledge? It would be interesting to extend the explanation about this.

→ Interpretation: I can see that the visualization focuses attention on the class object, but I cannot be sure with this visualization which features in the object are important and what differentiates one support vector from the other in the same class. To me, it is a good explanation, but I find it difficult to interpret.

→ Comparison with other methods: I would suggest that the authors also compare their methodology (even with a qualitative analysis) with methods such as ACE that also provide global explanations.

The weaknesses in short are:

• Masking limitations in novelty.

• A weak measurable evaluation with missing ablation study.

• not measuring faithfulness and reliance on a purely axiomatic definition of it.

• Masking limitations in novelty: lets be clear:

The Nadaraya-Watson head is (a simplified version of) few-shot learning. It is a softmax over negative distances between support samples and the test image.

While it is appreciated that Wang and Sabuncu, 2022, gave a name to their analysis in order to emphasize a predecessor of few-shot learning, using this term in this paper suggests a larger or different novelty than there actually is. It should be made prominently clear in the manuscript that the Nadaraya-Watson head is effectively few-shot learning (seemingly without sampling random subsets of classes).

The evidence head is a standard few shot head. Taking the positive part is a ReLU applied on a feature map. Again, that is renaming common parts to sound uncommon / novel.

Masking limitations in novelty in such a way is disliked by the reviewer. This results in a low score for presentation.

• A weak measurable evaluation with missing ablation study.

Table A.2 WASUP is not compared against pure B-cos backbones on which it builds on, but only with a black-box standard CNN.

By that one cannot distinguish whether the contributions are actually mostly from the B-cos network or whether the few-shot head plays any role in (a) predictive performance or (b) attribution map quality.

An ablation study is missing to quantify how much of the predictive performance and attribution map quality comes from the B-cos network or the few-shot head. In the worst case the B-cos network alone does all the heavy lifting.

If one checks the B-cos networks, then it appears to be well possible that the attribution map quality originates predominantly from the B-cos networks (e.g Fig 3 in https://arxiv.org/pdf/2306.10898, the pointing game shows high spatial selectivity, but also Fig. 4).

-- not measuring faithfulness and reliance on a purely axiomatic definition of it.

"We prove that explanations provided by WASUP fulfill the axioms required to be faithful."

"To thoroughly assess the interpretability of WASUP, we conduct a theo- retical evaluation of its explanations based on the axioms defined by Sundararajan et al. (2017), ..."

Faithfulness should be measurably quantified. That is a purely theoretical assessment.

Axioms are not a generally accepted definition of faithfulness. Axioms in this field are not uniquely determined. One can define different, non-compatible sets without the ability to rank or to exclude them.

For example, integrated gradients satisfy the axioms in their own paper, yet they have a ratther low measurable faithfulness under most data-driven faithfulness measures.

Minor: 1 . "The prediction of our model is a linear transform with the temperature and the bias term" That is not true if stated like that, because the weights are scaled depending on the test sample. It is not a big issue though

2. Pascal VOC: as most classes have very few positives, the standard of evaluation is average precision, not accuracy. One should report average precision, not accuracy for this data set.

1. The paper’s contribution is limited. The faithfulness of the proposed model relies heavily on existing B-cos networks, while its global interpretability comes from a Nadaraya-Watson head. The work largely made engineering efforts to combine these two established ideas without introducing new insights.
2. The original design element in this paper—using k-means clustering to extract class-specific centroids for support vectors—is intuitive but lacks novelty.
3. The paper would benefit from more thorough proofreading by the authors, as numerous notational issues affect comprehension. For instance, the notation ||w||=1 on line 190 is factually incorrect, Equation 2 for \hat{W} is in recursive form but lacks an explanation of initial values, the term c' in lines 163–167 is undefined, etc.
4. The methodology is similar to prototype learning, and a stronger case for the interpretability of the approach could be made by including both numerical and visual comparisons.

• Originality: The paper is not novel. The support vectors are, in essence, prototypes (as in a ProtoPNet). The comparison between an input image's latent representations and support vectors is also similar to the comparison between the latent representations and prototypes in a ProtoPNet. The proposed method is simply a combination of a B-cos network and a ProtoPNet.
• Quality: The authors only compared the accuracy of their WASUP models with non-interpretable models. They did not compare with other interpretable models.
• Quality: There is no quantitative and qualitative comparison of interpretability with other interpretable models as well.
• Significance: Since the main ideas behind the paper are mostly explored in prior work and there is no novelty, the paper lacks significance.