Explaining Contrastive Models using Exemplars: Explanation, Confidence, and Knowledge Limits

1. Lack of Comparison with Other XAI Methods: The paper does not compare its proposed techniques with existing XAI methods for contrastive models. A comparative analysis with other approaches would help establish the uniqueness and superiority of the proposed methods.

2. Clarity on Hyperparameters and Thresholds: The paper mentions optimal values for some hyperparameters, such as α and β, but does not provide detailed information on how these values were determined. Providing insights into the hyperparameter tuning process and sensitivity analysis would help readers implement the framework effectively. Moreover, the rationale behind selecting specific threshold values for OOD detection should be explained.

3. Reproducibility and Open Source Implementation: To enhance the reproducibility and adoption of the proposed methods, consider providing open-source code, pre-trained models, and guidelines for researchers and practitioners to apply your framework to their own tasks. This would make it easier for the community to validate and build upon your work.

• The "Related works" section should be better organized. The author should summarize works that are directly related to the article and provide concise summaries highlighting points of relevance to this paper.
• Exemplar Selection in Section 3.1: How does the method ensure that the selected exemplars are indeed representative? If the process requires calculating the embedding of all training images first, it could lead to extensive computational overhead.
• The introduction to "AttrMethod" is not clear. Since it's mentioned as "any Attr Method," a preliminary explanation should be provided to the reader for better understanding.
• The entire approach seems to be a combination of various methods (A+B+C). It would be beneficial if the authors could convincingly argue why such a combination is necessary and what unique advantages it brings.
• The explanation presented in section 4.1 is somehow not complete. Instead of providing a support explanation for the current test image, the authors emphasize the attribution map of the exemplars. While this might offer some support, it doesn't adequately explain the decision basis for the current test instance.

1. The proposed contribution from the perspective of explainability is minimal. It does not reveal new insights. Exemplars is not a new idea; why would it be a contribution to extend it to contrastive models?

2. In principle, Fig.1 or Eq2 is just an attribution of similar features between two images (nearest neighbor images). It does not seem to be relevant to the output of any downstream task. The CLIP encoding of one specific ostrich image is a unique vector, and it’s always (most probably) close to other ostrich images. But this type of analysis, finding the nearest neighbor without considering the target class in any downstream task, always gives the Ostritch image in any task. Consider one of the Ostrich images. Classes, such as trees and grass, are also in the image. Both grass and trees get high clip similarity scores (to text encoder). The method, in that case, better show top neighbors considering these other concepts and attribute to those. The current formulation is limited.

3. It is assumed that ANY attribution approach will work as expected. Unfortunately, this is not the case, as attributions are known to be disagreeing. Each one reveals different information. Thus, it’s best to avoid generalization to ANY attribution method and just state what is done in the paper (which is activation values). It is better to put more detail on how this is exactly computed.

4. More evidence is needed to prove if the attribution is working as expected. It might be that we are just highlighting salient features of the image irrespective of the other image. Try using the same Train Exemplar image with test images of other classes. This should be shown systematically, leaving no questions. (though I believe the authors if they let me know they tried and the method is sensitive)

Minor

• The visual attribution does not give much information. Most of the information regarding explainability comes from exemplars themself which is a previous work. It would be more useful if it’s more fine-grained with respect to concepts in the images (using CLIP text encoder).
• There are only a few qualitative examples of attribution. The work requires quantitative evaluations (for example, using segmentation masks) or some systematic qualitative experiments to convince us that it is working.
• The confidence score is not a very interpretable number (imagine a user using your approach, use a short description useful to the user)

• Emphasizing the salient regions doesn't necessarily lead to clearer explanations. It's not guaranteed that the subject is crucial for prediction. Specifically, an article titled "Sanity Check for Saliency Maps" by (Adebayo et al.) highlights the potential pitfalls of methods like Smoothgrad, as examined in their study. A consistency of salient regions between a pair of exemplars cannot reveal its similarity.
• The authors claim that the proposed framework is adaptive to any attribution methods. However, they only verify the proposed method on GradCAM, which is not sufficient. The authors are highly encouraged to add more experiments under the usage of other attribution methods, such as Smoothgrad and Integrated Gradient.
• In specific contrastive learning frameworks, like He et. al [1], positive examples are chosen by matching input labels. This method is somewhat at odds with measurements based on the concept of saliency maps. For instance, some dog images may display only the dog's head, while others may show the dog's body. In such cases, the saliency maps would vary significantly in their attributions, yet, these images are still chosen as positive examples for the input.
• The framework claims to be well-adaptive in explaining contrastive learning models; however, the proposed framework is only evaluated on the RN101-based CLIP model. The authors are highly encouraged to test their framework on more contrastive learning models, such as a representative one from He et. al [1].

[1] He et. al "Momentum Contrast for Unsupervised Visual Representation Learning" 2020