One Wave to Explain Them All: A Unifying Perspective on Post-hoc Explainability

1. The primary weakness is the lack of significant differentiation between the proposed attribution method and existing approaches. The paper provides limited analysis or visualizations that convincingly show how WAM offers better hierarchical explanations. While some comparisons are presented (e.g., in Figures 2 and 12), I don’t see clear and enough advantages in explaining input structures. The authors should further explore or emphasize the distinctive aspects of their method.
2. The quantitative results do not consistently demonstrate improvements over existing attribution methods. While comparisons with older methods are acceptable given the novelty of the proposed approach, the proposed method falls significantly behind in several key metrics, such as in Tables 3 and 5 (Appendix). Additionally, the results in Table 1 are concerning; given the definition of the Faithfulness metric in Eq. 9, the output should always be positive. Why, then, are many results in Table 3 reported as zero?
3. The organization of the results section could be improved. For example, the discussion of perturbation-based attribution methods in Section 4.2 appears abruptly and feels disconnected from Section 4.1.

The approach used in this paper shares similarities with the WaveletX(Kolek et al, 2022) method, which also performs saliency mapping in the wavelet domain. The primary distinction lies in the use of gradients as a mask in this work, While WaveletX optimize the mask on wavelet domain. However, this difference may not be significant enough to constitute a radical contribution to the field. It would be better to explicitly compare both methods and highlight the novel aspects of WAM.

The paper does not include quantitative assessments for 3D shape analysis and relies solely on qualitative results. Incorporating quantitative metrics would strengthen the evaluation and provide a more comprehensive understanding of the method's performance in this domain.

Although the authors claim that the Wavelet Attribution Method (WAM) outperforms other approaches across different domains, the results in Table 2 suggest otherwise. Specifically, WAM does not consistently outperform Integrated Gradients, indicating that the performance advantage may not be as significant as claimed.

The experimental comparisons primarily involve methods like Integrated Gradients, GradCAM++, and SmoothGrad, which are not the most recent or best-performing approaches according to the fidelity metric. Including comparisons with more recent and state-of-the-art methods, such as LRP-αβ (Samek et al., 2016), LayerCAM (Jiang et al., 2021), Guided Backpropagation (Selvaraju et al., 2016), AttnLRP (Achibat et al., 2024), and SRD (Han et al., 2024), would strengthen the evaluation and better demonstrate WAM's superiority.

The paper does not adequately demonstrate how WAM enables an understanding of what features the model uses to make decisions. While the method highlights important wavelets, it does not clarify the specific meaning or relevance of these wavelets to the classification task. For instance, approaches like CRAFT (Concept Recursive Activation FacTorization for Explainability, Fel et al., 2023) offer more explicit explanations by identifying meaningful concepts (e.g., "elephant tusk") that recur across multiple samples. Providing a similar level of interpretability by linking important wavelets to specific semantic features would improve the explanatory power of WAM.

Despite its ambitious goals, WAM introduces several ambiguities and potential oversights. Key among them is the unclear use of 'structural components', a term the paper uses to describe feature-level insights that the method claims to provide. This concept, critical to WAM’s claims of 'what' explainability, lacks a clear definition or grounding in quantifiable relationships among components, making it difficult to ascertain whether these are indeed structural features rather than just relevant input attributes. Furthermore, while wavelet decomposition is introduced as a novel approach to attribution, the practical interpretability of multi-scale heatmaps remains underexplored in the paper. it is unclear how users can derive specific insights from these maps without a more explicit explanation. WAM’s assertion of state-of-the-art (SOTA) performance is another potential weakness, given that its comparisons rely largely on 2017 models like SmoothGrad, Grad-CAM, and Integrated Gradients, raising questions about whether the method is genuinely competitive in the context of more recent advancements in the XAI field. Additionally, the effectiveness of the faithfulness metrics used to benchmark WAM’s performance could benefit from further clarification, especially given the method’s claims of surpassing existing techniques across domains.

1. I believe there is a lack of sufficient baselines. It would be helpful to include more options such as LIME, SHAP, and concept-based explanations for image and audio data. Since there is no quantitative evaluation in 3D settings, adding 3D LIME, SHAP, sensitivity analysis, and Layer-wise Relevance Propagation (LRP) for 3D baselines would be a solid starting point.

   References:

   [1] "Why Should I Trust You?": Explaining the Predictions of Any Classifier
   [2] A Unified Approach to Interpreting Model Predictions
   [3] Towards Automatic Concept-based Explanations

2. The experiments were conducted on only one dataset; therefore, it would be essential to include results from several datasets.

3. In the audio results (Figure 1 and Figure 4), it is quite challenging to identify the areas being explained. Making the less important areas grayscale while highlighting the significant areas in red would improve interpretability.

4. It would have been better to conduct a human study for the qualitative evaluation. For example, utilizing Amazon Mechanical Turk (MTurk) to ask annotators to evaluate WAM while providing explanations for other baselines would be beneficial.