On Explaining Equivariant Graph Networks via Improved Relevance Propagation

We thank Reviewer 6J4F for comments on the paper. We have provided pointwise responses below.

According to the formula in this paper, the contribution of the atom may change with the reference frame in which the molecule is observed.

We believe there is a misunderstanding. The node explanations by EquiGX are invariant to rotations. In other words, the node importance scores from EquiGX remain unchanged when the input molecule is rotated by a random rotation matrix. Therefore, EquiGX preserves the equivariance of the network. We have also conducted experiments to verify and confirm this invariance.

Can EquiGX be extended to other equivariant models?

EquiGX can be readily extended to other spherical equivariant models, such as EquiformerV2, MACE, and PACE, which also rely on tensor product operations. However, due to the significant computational cost, we leave such extensions for future work. We would also like to point out that models like SphereNet and ComENet are different, as they operate solely on invariant features without incorporating equivariant ones, and thus do not rely on tensor product operations. We believe these models are beyond the scope of this paper.

In line 205, $d_i,j$ should be $d_{i,j}.$ In Eq 2, $C^{(\ell_3,m_3)}_{(\ell_1,m_1),(\ell_2,m_1)}$ should be $(\ell_2,m_2)$.

Thanks for pointing out the typos. We will update the manuscripts.

We are very glad Reviewer ffwb had a positive initial impression and appreciate your constructive comments. We provide pointwise responses below.

Figure 1 is confusing.

We apologize for the confusion. In Figure 1, the ground truth is shown in the upper-left corner. Nodes forming the cube motif are highlighted in red. Across all examples, node positions remain consistent, so better alignment with the ground truth reflects a more accurate explanation. In the figure caption, “the other areas” refers to nodes that do not belong to the cube motif. In other words, these nodes form a pyramid shape. We also highlight the nodes in blue in the ground truth.

Clarification on importance calculation.

As described in Section 3.2, for a single TP-based message passing layer, we decompose the relevance score of each message into three components, including the hidden features, the directional part, and the distance part. The relevance score attributed to the hidden features is further backpropagated toward the input. The relevance scores of the directional and distance components do not propagate beyond this layer. Within each message passing layer, the scores assigned to edge direction and edge distance capture their respective contributions to the final prediction. To compute overall contributions of edge directions and edge distances, we sum these directional and distance relevance scores across all layers. To obtain the final importance score for each input node, we combine the relevance score of the node itself with the relevance scores of its connected edges. In other words, the node-level explanation is calculated as the sum of a node’s own relevance plus half of the relevance of its neighboring edges.

Figures 3 and 4, while visually compelling, are somewhat unclear. It is unclear which part of the visualization corresponds to β-sheets.

Beta sheets typically appear as flat, arrow-shaped ribbons pointing in a specific direction, often aligned side-by-side to form sheet-like structures. In Figure 3, the regions with arrows represent the beta-sheets. For reference, the red arrows in this Wikipedia image illustrate their typical appearance. We will revise the manuscript to clarify this in the figure captions and text.

Additionally, we would like to point out that the entirely red visualization produced by the LRI-Bern method does not necessarily imply higher effectiveness. An ideal explanation should selectively highlight the beta-sheets in red, while keeping less relevant regions such as the coiled structure near the top and the thin, string-like lines in blue.

We are very glad Reviewer KEDe had a positive initial impression and appreciate your constructive comments. We provide pointwise responses below.

The method is limited to TP-based models.

We admit that our method is focusing on spherical equivariant GNNs (TP-based models) and leaving generalization to other architectures as future work. However, we would like to highlight that TP-based equivariant networks represent a broad and widely adopted family of models in the AI for science domain. Developing explainability techniques for these models is important, as it can enhance our understanding of existing architectures and potentially lead to their improvement.

The limitations are not discussed.

Thanks for the suggestion. One limitation of our method is its current focus on tensor product operations, specifically within spherical equivariant GNNs. While this represents a subset of equivariant GNNs, it is a significant and impactful area. Consequently, our method does not yet extend to all types of equivariant GNNs. However, this focus allows for a deep and thorough exploration of tensor product operations, and we plan to generalize our approach to other architectures in future work.

The Shapes and Spiral Noise datasets are very similar, and their visualizations look a bit repetitive. It might be better to show the visualization of one and move the other to the appendix.

Thanks for the suggestion. We would update the visualization during camera-ready stage.

The experiments focus on graph-level prediction tasks. Why not include node-level or edge-level predictions?

Evaluating explainability in equivariant networks is inherently challenging, largely due to the difficulty of obtaining meaningful and verifiable ground truth for explanations. In this paper, we address this by constructing synthetic datasets and carefully selecting real-world datasets where such ground truths are available. However, edge-level tasks like force prediction present additional complexity. While molecular dynamics can provide an approximate reference, the underlying rationale behind force prediction remains an open research question. Thus, we leave the exploration of node-level and edge-level tasks to future work.

Why isn’t ActsTrack included in the qualitative evaluation?

ActsTrack has an average of over 100 nodes per graph, and unlike proteins, it lacks a well-established visualization standard. Additionally, the ground-truth explanations are not immediately obvious, making clear and interpretable visualizations more challenging. As a result, including ActsTrack in the qualitative evaluation could lead to confusing visuals. We plan to include it in the paper later with more refined visualization methods.

The source code is not provided.

We plan to release our code during camera-ready stage.

We are very glad Reviewer V1gi had a positive initial impression and appreciate your constructive comments. We provide pointwise responses below.

This paper is an extension of the LRP framework that has been widely applied for vision tasks. Although tensor-product networks are different from normal MLPs or CNNs, the technical contribution of this work could be limited.

We would like to humbly clarify our contribution. We admit that EquiGX is an extension of LRP rules for spherical equivariant graph neural networks. We would like to point out that there are currently no established LRP rules for spherical equivariant Graph Neural Networks. Besides, developing propagation rules for new architectures is a challenging task. For instance, references [1] and [2] extend LRP rules to transformers, and references [3] and [4] extend them to traditional 2D GNNs. Additionally, different approaches to using Taylor decomposition can yield various LRP rules, such as $z^+$ rule and $w^2$ rule. Furthermore, spherical equivariant GNNs, despite having aggregation operations similar to those of traditional GNNs, fundamentally differ due to their reliance on tensor product operations. We are the first to explicitly consider tensor product operations to develop new LRP rules for spherical equivariant GNNs.

Limited Backbone Models: Only one model, TFN, is tested.

Given the widespread use of powerful spherical equivariant GNNs, understanding their key components, i.e. the Tensor Product (TP), is a fundamental step toward explainability in equivariant models. In this paper, we focus on TFN, the most classical and representative spherical equivariant GNN. Our method is general and can be extended to other spherical equivariant GNNs. However, due to the high computational cost, we leave such extensions to future work.

Why existing (2D) GNN methods are not suited for explaining 3D-equivariant GNNs?

Thanks for the question. Most existing 2D GNN explanation methods are developed for graph data that capture only topological relationships. However, 3D equivariant GNNs go beyond topology by incorporating rich geometric information, such as interatomic distances, angles, and torsion angles, which are essential for many tasks in scientific domains. These models are specifically designed to respect spatial symmetries, enabling consistent predictions under 3D transformations. 2D explanation methods typically do not consider atomic positions in space or spatial relationships such as angles and distances. As a result, they often fail to capture critical geometric cues. For instance, a small change in a node's 3D position can lead to significant shifts in angles or torsion angles, which may affect the model’s prediction. Since 2D methods are not sensitive to these changes, they tend to provide limited or misleading explanations when applied to 3D-equivariant GNNs.

Equivariant GNNs are conducted on widely-used molecular datasets such as QM9 and MD17. Is there a specific reason why the experiments on these datasets are not conducted?

While Equivariant GNNs are commonly evaluated on datasets like QM9 and MD17, these datasets lack ground-truth explanations. The underlying rationale for quantum chemical properties and energy predictions is still an open research question, with no definitive consensus. As a result, these datasets are not well-suited for evaluating explainability methods. Therefore, in this paper, we construct synthetic datasets and carefully select real-world datasets where meaningful and verifiable explanations can be obtained.

What does it mean for LRI treats the model as a black box and overlooks the equivariance of the model?

Both LRI-Gaussian and LRI-Bern, like other perturbation-based XAI methods, treat the model as a black box. This means they rely solely on input-output behavior without requiring access to the model’s internal parameters or gradients. In other words, they evaluate how changes in the input influence the output, without considering the internal structure of the model.

Besides, LRI-Gaussian overlooks the equivariance of the model, because the learned Gaussian noise is not equivariant. Specifically, when the input point cloud is rotated, the learned noise does not necessarily rotate accordingly. As a result, the explanations can become sensitive to the choice of the input’s reference frame, which is undesirable for explaining equivariant models.

In both Eq 2 and 5, should it be ($\ell_2,m_2$) in $C^{(\ell_3,m_3)}_{(\ell_1,m_1),(\ell_2,m_1)}$?

We apologize for the typo. Yes, it should be $(\ell_2,m_2)$.

Reference

[1] Transformer interpretability beyond attention visualization. CVPR 2021.

[2] XAI for transformers: Better explanations through conservative propagation. ICML 2022.

[3] Higher-order explanations of graph neural networks via relevant walks. TPAMI 2021.

[4] Relevant walk search for explaining graph neural networks. ICML 2023.