HyperPg - Prototypical Gaussians on the Hypersphere for Interpretable Deep Learning

Thanks for your valuable feedback! We appreciate that you find our idea innovative and like the overall motivation and structure of the paper.

We regrettably agree, that our experiment does not sufficiently highlight the advantages of our model. We are currently running more CNN based experiments to address this issue. One point which we need to better highlight when describing our experimental setup, is the use of full images on CUB-200-2011. In contrast to other prototype learning work, we do not use crop to the bounding box for our experiments. Additionally, the experiments on Stanford Cars were not fully converged until the submission deadline. We posted a new revision with updated results, which are also more in line with prior reported results. Having said all that: We are preparing additional experiments with different backbone models.

Thank you for pointing out the additional publications. They are of high interest to us and warrant further discussion. On a first, quick glance, [1] could provide an interesting test if the prototypical parts are actually aligned. Our current hope is, that the inclusion of the RRC loss could mitigate this issue during learning, and a gradient based visualization method during the explanation stage. We will look into this work more in depth.

[2] does really interesting work with learning a hybrid input head to disentangle color and shape. We hope that a better qualitative evaluation will show that our model can successfully learn prototypes for concepts such as "red head" and "white head". But directly learning disentangled prototypes for "red", "white" and "head" with our current model would be a welcome surprise. We will provide a more thorough analysis of our model to strengthen our manuscript and double check we do not claim inflated contributions.

We also see CBM as an important method to compare to. For CUB-200-2011, CBM's performance (75.1) is on a level with our models (75.8 & +RRC 71.9). However, it can only say if a concept is present, not where, reducing the insights it provides. Moreover, CBM struggles with multi-modal concepts, which HyperPgNet can learn by increasing the number of prototypes per concept. This is why CBM's performance on Stanford Cars is lower, because the provided concepts like "front lights" are rather broadly defined. CBM scores 79.9% compared to our methods 87.5% and +RRC 84.7%. We will highlight these results and emphasize the advantages of our model in the manuscript.

Please let us know if you have any additional questions or concerns.

Thanks for your valuable feedback! We appreciate that you like the manuscript and the simplicity of our approach. We regret and agree that the current manuscript does not sufficiently highlight our contributions. Regarding your comments:

1. We appreciate your request for additional datasets. The chosen Datasets are in line with prior work on prototype learning such as [2], [3], [4]. However, we are now also looking at datasets such as Stanford Dogs and Oxford Flowers. We hope to provide additional results, soon. We apologize for not highlighting our ablation studies better. We present some quantitative results with different number of prototypes in Appendix B.2, although qualitative results would also be of interest.

2. We selected Concept Bottleneck models as a baseline for a model, which also incorporates concept information similar to our method. ProtoPNet is the prototype learning base model, most prototype learning models, including ours, is built upon. We will improve our manuscript to better highlight the differences between our model and other works. For example, ProtoKNN uses a KNN classifier after ProtoPNet's prototype layer instead of a linear classification layer. Or Deformable ProtoPNet learns a spatial combination of a set of smaller prototypes.

3. The choice for ConvNeXt-tiny was inspired by other recent work such as [5], [6]. Regarding Segformer: You are right, this is a segmentation model and are currently verifying our implementation. However, we believe we correctly used a classification implementation built on MiT-B4 transformer introduced in the Segformer work. As the performance of the downstream models (ProtoPNet and HyperPgNet) is comparable to other reported results, we are confident the experiments are valid. Still, we acknowledge that this causes unnecessary confusion and distracts from our contribution. We are currently running more CNN based experiments, but hope to also provide additional results with a different, transformer based backbone.

4. This is a shared sentiment among all reviewers, which we regretfully agree with. We are working on a more extensive evaluation and hope to provide results soon. A comparison with other methods would be further strenghten our work.

5. We apologize for not finishing the experiments you mentioned in time for the first manuscript submission. We already updated a revision with converged results. As you will notice, with enough training time, the other methods also reach comparable results. However, we can already see that HyperPgNet und HyperPg prototypes require fewer training epochs to achieve this level of performance. Additionally, we hope to show the increase in interpretability by providing more qualitative results.

6. We will update the manuscript to better motivate our approach and provide more meaningful results.

We hope that we could answer your questions and we will keep you updated with new results as they come in.

[2] Chen et al 2019 - ProtoPNet https://proceedings.neurips.cc/paper/2019/hash/adf7ee2dcf142b0e11888e72b43fcb75-Abstract.html

[3] Donelly et al 2022 - Deformable ProtoPNet https://openaccess.thecvf.com/content/CVPR2022/papers/Donnelly_Deformable_ProtoPNet_An_Interpretable_Image_Classifier_Using_Deformable_Prototypes_CVPR_2022_paper.pdf

[4] Ukai et al 2023 - ProtoKNN https://openreview.net/forum?id=lh-HRYxuoRr

[5] Wang et al. 2024 - MCPNet https://eddie221.github.io/MCPNet/

[6] Nauta et al 2023 - PIPNet https://openaccess.thecvf.com/content_ICCV_2019/papers/Hwang_SegSort_Segmentation_by_Discriminative_Sorting_of_Segments_ICCV_2019_paper.pdf

Thank you for the valuable insights and feedback. We appreciate that you find our paper well written and our method interesting. One common concern amongst the reviewers is the need for more extensive quantitative and qualitative experiments, with which we wholeheartedly agree. We are taking the feedback to start additional experiments, for which we will provide the results as soon as they come in.

Thank you for pointing out the reference. We will add it to the related work section. For a somewhat short comparison, we focus on the proposed von Mises Fisher Distribution (vMF) and our HyperPg activation:

1. We added a Section to the Appendix (A.3) which shows the equivalency of the unnormalized vMF distribution, as used by [1], with the HyperPg activation with fixed $\mu=1$. Please note, that the vMF mean vector $\mu$ corresponds to the HyperPg anchor vector $\alpha$, not HyperPg's Gaussian mean skalar $\mu$. Because HyperPg can optimize and learn the Gaussian mean, as it is not fixed at 1, which leads to the complex activation patters as depicted in Fig 1.

2. From an implementation standpoint, the formulation of HyperPg as a two step approach with some anchor similarity and learned probability function enables future extensions. By using a different manifold and similarity metric, different properties could be enforced. Or a different, multi-modal probability distribution could be used instead of the Gaussian distribution.

We appreciate your request for additional datasets. The chosen Datasets are in line with prior work on prototype learning such as [2], [3], [4]. However, we are now also looking at datasets such as Stanford Dogs and Oxford Flowers. We hope to provide additional results, soon. We hope to show with more qualitative results that the quality of the labels are sufficient for HyperPgNet to learn prototypes aligned with the human defined concepts.

Regarding your questions:

1. As a different reviewer noted, the results on Stanford Cars were unfortunately not converged in time. With enough training time, the other methods also reach comparable results. However, we can already see that HyperPgNet und HyperPg prototypes require fewer training epochs to achieve this level of performance. Additionally, we hope to show the increase in interpretability by providing more qualitative results. A model with black-box model level of performance and inherent interpretability will hopefully enable downstream tasks with increased requirements for model trust in safety critical applications
2. We apologize for our oversight of not including the hyperparameters in the first manuscript. For ProtoPNet, we follow the hyperparameters proposed by the authors, with 0.8 and 0.08 for cluster and separation loss. For the newly introduced HyperPg prototypes, all loss terms are weighted equally at 1. We have added this information in the revision.
3. Thank you for highlighting this issue. Currently, our qualitative results do not sufficiently convey the capabilities of our model. We are preparing and updated result section and aim at explanations in the manner of: "This input is classified as A, because this image region is activated by the prototype for concept B. The concept B can also be found in these regions from known training images...". We agree that the currently presented results make our contribution look like a common saliency map or attention map. We hope, updated results will better demonstrate the capabilities of our method.
4. Despite running multiple hyperparameter sweeps, we were unable to achieve better results. However, we feel the results are still valid, as the downstream models still performed well. Although, following feedback from all the comments, we hope to run additional experiments with other backbone models.

We hope that we could answer your questions and we will keep you updated with new results as they come in.

Edit: Forgot to add the references. [2] Chen et al 2019 - This looks like that (ProtoPNet) https://proceedings.neurips.cc/paper/2019/hash/adf7ee2dcf142b0e11888e72b43fcb75-Abstract.html [3] Donelly et al 2022 - Deformable ProtoPNet https://openaccess.thecvf.com/content/CVPR2022/papers/Donnelly_Deformable_ProtoPNet_An_Interpretable_Image_Classifier_Using_Deformable_Prototypes_CVPR_2022_paper.pdf [4] Ukai et al 2023 - ProtoKNN - https://openreview.net/forum?id=lh-HRYxuoRr

Thanks for providing the rebuttal. However, the current form of the paper lacks enough evidence to show the benefits of the proposed method, therefore I will maintain my current rating.

Thank you for your valuable feedback. One common concern amongst the reviewers is the need for more extensive quantitative and qualitative experiments, with which we wholeheartedly agree. We are taking the feedback to start additional experiments, for which we will provide the results as soon as they come in.

In the meantime we will improve the clarity of the manuscript and address the points you mentioned. As a first change, we reordered the sections describing the losses to improve the logical structure, and will continue to improve the motivation for our approach in contrast to traditional prototype methods.

Regarding your third point: We added some more details regarding the choice of hyper-parameters (equally weighted for HyperPg prototypes), but do not feel confident we answered your question. Could you please clarify, what kind of complexity you mean?