Counterfactual Explanations for 3D Point-Cloud Classifiers

1. The method is compared only with DGCNN as a backbone. This architecture is indeed basic and elementary in point cloud classification, but it is not cover a wide range of possible architectures. It is recommended to select diverse architectures (PCT for transformer, maybe PointNet for simple learnable architecture, or more up-to-date networks like Recon).
2. The quantitative and qualitative analysis is mainly focused on the comparison between AE to diffusion, whereas the more profound idea here is the analysis of the results themselves. What can I learn from the visualizations about which elements are crucial for detecting a chair for example. I can see that the handle is transformed, but is it consistent across a wide range of chairs? is it the most prominent feature? (what if it is absent?)
3. Comparison to Other 3D XAI Methods: The paper does not provide a comparison with other XAI methods for 3D data. As a result, it is unclear how this approach performs relative to existing methods. A common strategy for XAI comparison is the perturbation test, which has also been applied in 3D contexts (e.g., see https://arxiv.org/pdf/2403.07706). While this comparison approach is particularly suitable for methods that rank points by importance, potentially differing from your approach, I believe an evaluation that allows for some level of comparison would be beneficial.

1. I feel the technique innovation is limited. I do not see specific design that is tailored for 3D point cloud; it just follows the diffusion based counterfactual explanation techniques in 2D field. The authors state that "Our work therefore diverges from existing CE methods by not focusing on 2D image-based models but embracing 3D data processing challenges, with a particular emphasis on point cloud classifiers." In my view, here is just a direct reuse of previous methods for 3D point cloud setting. Moreover, specific references should be given for "existing CE methods".
2. What is the difference between 2D image and 3D point cloud, in terms of counterfactual explanation?
3. While generating counterfactual explanations in this way is innovative, diffusion models have already been widely applied in 3D point cloud generation across various conditions and representations. I didn’t see any particularly clever or tailored use of the diffusion model that sets this method apart for counterfactual generation. It would be helpful if the authors could highlight any unique contributions of this method specifically for the counterfactual generation challenge.
4. About SDS loss. I’m not fully convinced about the need for SDS loss here. In the AIGC field, SDS is often used to bring in prior knowledge from diffusion, but it’s not entirely clear why it’s needed for counterfactuals in this case. While the results show that SDS improves generation quality, I’m not sure it’s contributing directly to the counterfactual sample generation itself. A bit more justification on this point would be useful, and perhaps some ablation experiments to assess whether it’s really essential for the counterfactual task.
5. The literature review is not comprehensive. Some relevant efforts are missing, such as Interpretable3D: An Ad-Hoc Interpretable Classifier for 3D Point Clouds, AAAI.
6. The paper could use more details on the structure of the diffusion model and the training setup. Information like the specific architecture, data size, optimizer settings, etc., would improve reproducibility and help readers understand exactly how the method was implemented. Adding these details would make the paper feel more complete.
7. For point cloud tasks, I think the autonomous driving setting is more important for explanation as it is safety-aware. But it seems that the proposed technique cannot be used for explaining 3D autonomous driving models.
8. The proposed method seems limited to specific types of classifiers, like DGCNN. Extending the approach to a broader range of 3D models would provide a more comprehensive evaluation.
9. Considering the claim of “… to improve the memory footprint and the computational costs of diffusion-based CE generation,” the computational costs should be discussed. Diffusion models are computationally expensive, especially in high-dimensional data like 3D point clouds. The paper does not address strategies for managing computational load, which might limit practical applications. 10.The method is evaluated primarily on the ShapeNet dataset, which may not represent all 3D data scenarios. Evaluation on diverse datasets, especially those with more complex and noisy real-world data, would strengthen the validation.
10. The generated counterfactual explanations might be challenging for non-expert users to interpret without additional visualization tools or interpretive guidance.
11. A comparison with other explainable AI methods, even those adapted from 2D to 3D data, could offer insight into the specific advantages or trade-offs of the proposed approach.
12. Inconsistent Reference Formatting: For example, in L572 and L574, “ … Advances in Neural Information Processing Systems, ...” should be “ … In Advances in Neural Information Processing Systems (NeurIPS), …”; in L578, “ … International Conference on Machine Learning ...” should be “... In International Conference on Machine Learning (ICML), …”. 14.Typos: For example, in L265, “f(x” should be “f(x)”; punctuation should be added after equations in the Appendix.
13. Limitations and societal impact should also be discussed to provide a comprehensive understanding of the approach's practical constraints and broader implications. Addressing these aspects helps identify areas for improvement and promotes Explainable AI (XAI) development by anticipating potential ethical and societal challenges.

• Although the paper claims to generate human-interpretable counterfactuals, it lacks a thorough quantification of the "semantic relevance" of these explanations, limiting the interpretability and effectiveness of the proposed approach.
• Experimental validation is limited to a single dataset and task (binary classification using ShapeNet). This narrow scope restricts the generalizability and fails to demonstrate the method's robustness across a broader set of data and classification tasks.
• The evaluation is restricted to a single point cloud classifier (DGCNN), without extending the analysis to other architectures. This narrow focus raises concerns about the method’s applicability across diverse model types.
• Although an ablation study on loss functions is provided, the paper lacks a detailed examination of each loss component's impact on the generated counterfactuals, which would strengthen claims regarding the method’s effectiveness.

Overall, the experimental results fall short of validating the authors' claims. While the idea is interesting, the findings remain unconvincing due to limited and narrowly focused evaluations.

The final loss is a complex combination of 4 losses. The ablation study does not seem to give a clear answer to see which losses are needed and what their individual contribution is. If I interpret the column correctly each loss is combined individually with L_cf, what makes sense because that is the counterfactual loss. However, combinations such as L_cf + L_prox + L_div but without L_st is unknown. But it is not only the loss combination, also the mixed performance does not give a high degree of confidence that all losses are needed or well balanced.

Given that this is, as the authors claim, the first generation of it's kind, it is hard to compare it to existing state of the art.

The qualitative examples could be a matter of debate. The concrete semantical changes in the airplane category could also stem from noise. Semantic changes of chairs seem also minor.