DualEqui: A Dual-Space Hierarchical Equivariant Network for Large Biomolecules

RQ1 & RQ2: About the difference between our SH neighborhood and spherical tensor-based baselines.

Thank you for raising this important question. We believe there may be some misunderstandings, and we clarify the differences below.

(1) There are two distinct classes of methods that utilize spherical harmonics:

Type 1: Methods based on spherical tensor products using Clebsch–Gordan coefficients and Wigner-D matrices. These methods, such as TFN, SE(3)-Transformer, and SEGNN, perform full tensor contractions, which lead to high computational cost up to $O(L^6)$ [4], where L is the maximum tensor degree. This is because they compute tensor products across all combinations of degrees up to L. A detailed explanation can be found in Section 5.3 of [2]. While e3nn is a Python library and not a specific method, the above baselines rely on it for implementation.

Type 2: Even though the calculation of spherical tensor products is too expensive, the high-order spherical tensors are proved to contain more information [2]. To address the complexity issue, some recent baselines still use spherical tensors but try to replace the “tensor product process” with “vector inner product” (Inner product itself is $O(L)$). In this way, we incorporate the spherical tensor information while avoiding the complexity.
This kind of methods includes HEGNN [5], GotenNet, and our DualEquiNet. Similar discussions can also be found in Sections 1 and 2 of HEGNN [5] and Sections 1 and 2 of GotenNet [6].
Because these two categories differ fundamentally in their computation and architecture, it is not feasible to use a Type 1 method as an ablation for our Type 2-based approach.

(2) On our SH-based neighborhood definition:
The SH-based neighborhood is a core contribution of our method and is not present in any existing baseline. It enables long-range, symmetry-aware interactions by linking nodes with similar local geometric environments, even if they are spatially distant—complementing standard Euclidean neighborhoods.
Regarding the reviewer’s concern: SH similarity is invariant to global rotations due to our equivariant design (see Appendix C.1 and B.2). Therefore, two benzene rings with similar local structures will still be linked regardless of their global orientation. Additionally, in both Type 1 and Type 2 methods, spherical tensors are computed based on directional vectors (i.e., relative positions), and are independent of absolute distances.

We will revise the paper to make these distinctions and design choices more explicit.

Q3: The benchmarks are not very widely used. More interesting dataset from MolecularNet should be considered. In particular, the prediction of SARA and torsion angles seems to be not proper for this case. Note that the model input has all the geometric information, and SARA and torsion angles are just special properties that can be analytically calculated.

RQ3: Since there is no reference attached, we guess you mean the “MoleculeNet” instead of “MolecularNet” in [1]. The datasets in MoleculeNet, like QM9, are about small molecules and have been widely used in papers of the baselines. However, our DualEquiNet aims to deal with the long-range dependencies in the large biomolecules. Tasks in MoleculeNet are not well-suited for evaluating this capability. Therefore, instead of using those datasets of small molecules, we develop new datasets of large biomolecules like protein and RNA. We identify this as a contribution instead of a weakness of our paper.

Indeed, the SASA and torsion angles can be calculated if we have ground-truth 3D structures. However, in real-world scenarios, we don’t have access to the ground-truth 3D structures. Especially for RNA, the 3D structures are highly noisy. So here our point is how accurately we can infer the SASA and torsion angle if we only have noisy 3D structures?
To this end, we ensure that ground-truth structures are used only to generate labels, while all models, including ours and the baselines, are provided only with noisy 3D structures derived from sequence-based prediction methods. This better simulates real-world conditions and tests a model’s capabilities.

We appreciate the reviewer’s comment and will include any proposed changes in camera-ready version. If we have addressed your questions, we kindly request you to consider updating your score. Thank you very much for your time and thoughtful review!

Reference:
[1] MoleculeNet: A Benchmark for Molecular Machine Learning.
[2] A Hitchhiker’s Guide to Geometric GNNs for 3D Atomic Systems.
[3] On the Expressive Power of Geometric Graph Neural Networks.
[4] From Peptides to Nanostructures: A Euclidean Transformer for Fast and Stable Machine Learned Force Fields.
[5] Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?
[6] GotenNet: Rethinking Efficient 3D Equivariant Graph Neural Networks.

We thank the reviewer for the valuable feedback, and we address the concern below.

W1: Significance of the evaluation datasets in Tables 3 and 4, and new results on RNAGlib.

RW1:
Significance of Tables 3 and 4 : Tables 3 and 4 focus on torsion angle prediction and solvent-accessible surface area (SASA) estimation. These tasks were chosen specifically because they capture fine-grained 3D geometric features that are highly relevant in drug discovery. Torsion angles play a central role in determining local conformations of biomolecules, directly influencing active site geometry and ligand binding modes. Accurate torsion angle prediction is critical for modeling flexible regions such as RNA loops and protein side chains, which are often key determinants of druggability. Experimental techniques such as NMR, X-ray crystallography, and cryo-EM can provide this information but are costly and time-consuming, motivating accurate learning-based approaches.
SASA is a well-established property used in structure-based drug design to identify hydrophobic and hydrophilic regions, estimate ligand accessibility, and correlate with binding affinities. Models capable of reliably predicting SASA provide valuable insight for guiding ligand optimization and binding site characterization.

These two tasks are widely used in molecular modeling literature [1][2][3] to benchmark a model’s ability to learn detailed structural properties in drug discovery applications.

We thank the reviewer for this valuable suggestion. In response, we have conducted additional experiments on the RNAGo and RNAProt datasets from RNAGLib and will include the results in the camera-ready version. DualEquiNet achieves state-of-the-art performance on these as well, further validating its effectiveness (see Table below). We also note that the RNA property prediction benchmarks reported in the main paper (e.g., Ribonanza, COVID and Tc-Ribo) are themselves widely used and established in works [4][5][6], ensuring that our evaluation is based on community-accepted standards and datasets. Due to time constraints, experiments on the ProteinWorkshop datasets, which are much larger, are not feasible now. We plan to include these results in the camera-ready version. We will update all references accordingly.

W2: Significance of architectural components through ablation.
RW2:
(1) Ablating components in DualEquiNet indeed changes the performance. The absolute RMSE differences in Table 5 translate to consistent percentage improvements and are data-dependent. For example, removing CSIP increases error by ~8% on Tc-Ribo and ~7% on Ribonanza, while disabling cross-space message passing further increases error by ~9% on Tc-Ribo. The combined removal of all dual-space components yields the largest degradation, with ~11% higher error on Tc-Ribo and ~7% on Ribonanza. Overall, these results indicate that DualEquiNet’s architectural contributions—particularly contextual pooling and cross-space learning—are synergistic and yield up to ~10% improvements on different tasks.

(2) Explanation of results in Table 5.
DualEquiNet w/o CSIP/Cross/Dual in Ablation Table 5 is not equivalent to standard Euclidean GNN baselines. Even in this ablation, we preserve the biological organization of large molecules (e.g., atom-to-nucleotide for RNA, atom-to-amino acid for proteins) using hierarchical mean pooling. We also mentioned this in Line 346 in our submission. Specifically, “w/o CSIP” replaces cross-space interaction pooling with standard mean pooling, but the pooling is still applied over biologically defined hierarchical units rather than only individual atoms. This explicit incorporation of biologically meaningful structure is one of our key contributions, ensuring that even our Euclidean-only variant captures context beyond atom-level interactions used by standard atom-level Euclidean GNNs, which explains its stronger performance, which explains why it already outperforms those baselines. We will make it more explicit in the camera-ready version.
Originally, we did not apply hierarchical pooling to the baselines to respect their original designs, and some are not directly compatible with such pooling. However, we have now run selected Euclidean baselines augmented with hierarchical mean pooling. Their performance improves over their original versions (reported in Table 1) and matches our Euclidean-only variant but still lags significantly behind the full DualEquiNet with cross-space interaction pooling (CSIP) and dual-space message passing (e.g., 4.6%, 8.7%, and 9.0% gains over best EGNN Pooling on COVID, Ribonanza, and Tc-Ribo datasets respectively).

W3: The authors highlight the importance of E(3) equivariance for 3D biomolecules, but they don't discuss the relationship between E(3) and SE(3) equivariance for modeling molecular chirality. As such, it's currently unclear if the authors' proposed method is truly E(3)-equivariant and therefore can't detect chirality or whether it is actually SE(3)-equivariant and can detect chirality. A discussion of this point would be helpful.

RW3: SE(3)-equivariant models are equivariant to translations and rotations but not to reflections. This allows them to distinguish chirality because mirror images are treated as distinct configurations. Most fully SE(3)-equivariant models—such as Tensor Field Networks (TFN), SE(3)-Transformer, and SEGNN—achieve this by using Clebsch–Gordan tensor products over spherical harmonics, which provide full orientation-sensitive tensor features but introduce significant computational overhead (up to $O(L^6)$ for maximum tensor degree $L$). To avoid this cost, our DualEquiNet, like recent HEGNN and GotenNet models, uses spherical harmonic–based inner products instead of full tensor products. This design enables rich geometric reasoning at scale while remaining computationally efficient, but it also makes the model reflection-equivariant (E(3) rather than SE(3)). Extending DualEquiNet to an explicitly SE(3)-equivariant variant to handle chirality more directly is an exciting direction for future work.

Q1: Thoughts on potential of simpler transformer-Based 3D Biomolecular Models.
RQ1: We thank the reviewer for raising this timely question. We agree that recent developments, such as AlphaFold3, have shown that large-scale Transformer-based architectures with rotation data augmentation can implicitly learn geometric symmetries and achieve excellent results. However, as highlighted in [7] learning equivariance can provide greater data efficiency and generalization in many applications such as molecular property prediction and dynamics simulation. Similarly, [8] shows that explicit equivariance offers substantially improved data efficiency and better compute efficiency following power-law scaling.
DualEquiNet is designed precisely for such settings: it combines explicit geometric inductive biases with hierarchical biomolecular organization to achieve strong generalization even when labeled data is limited, while also supporting dual-space reasoning required for cross-scale biomolecular interactions. We view Transformer-based approaches trained with augmentation and DualEquiNet as complementary: Transformers excel for large-scale foundation models, while architectures like DualEquiNet remain crucial for data-limited and physics-sensitive problems.

Please let us know if there are any remaining concerns. If we have addressed your questions, we kindly request you to consider updating your score. Thank you very much for your time and thoughtful review!

Reference:
[1] Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoders.
[2] Deeprna-twist: Language model guided rna torsion angle prediction with attention-inception network.
[3] Rna-torsionbert: leveraging language models for rna 3d torsion angles prediction.
[4] Deep learning models for predicting rna degradation via dual crowdsourcing.
[5] Ribonanza: deep learning of rna structure through dual crowdsourcing.
[6] Tuning the performance of synthetic riboswitches using machine learning.
[7] Equivariance is dead, long live equivariance? (a blog post on Molecular Modelling).
[8] Does equivariance matter at scale?

Dear Reviewer, as the review update deadline is approaching, please let us know if you have any further questions. We’d be happy to clarify. Otherwise, if your concerns have been addressed, we’d greatly appreciate it if you could consider raising the rating.

W1: The submission states that the model works well, but it doesn't go into much detail about how hard it is to compute in the main text, instead putting it in the appendix. There needs to be more upfront discussion about the resources it needs compared to baselines, especially for the "large biomolecules" it targets.

RW1:
Following reviewer's recommendation, we will move this discussion from the Appendix to the main text in the camera-ready version. To emphasize, our DualEquiNet has substantially lower compute compared to tensor-product-based models such as TFN or SE(3)-Transformer (complexity $O(L^6)$) because DualEquiNet relies only on inner products over spherical harmonic features ($O(L)$). In practice, on the COVID dataset with identical hyperparameter settings, DualEquiNet achieves 3–5× faster training on large biomolecules compared to these tensor-product-based models, while maintaining strong predictive performance (see also table below for time per epoch and GPU memory requirements).

Additionally, for all baselines and our method, we constrained the number of parameters to at most 4 million and ensured that all models fit within the memory of a single NVIDIA A6000 GPU. Hyperparameters for each model were optimized using Optuna under the same computational budget, ensuring fair comparisons.

W2: Comparison with other long-range baselines.

RW2: We appreciate the reviewer’s suggestion to include additional baseline comparisons. Our original submission already compared against a recent long-range model (FastEGNN). FastEGNN passes global information by introducing a small set of virtual nodes that connect to all real nodes and each other, enabling global message passing on graphs.

Following your recommendation, we further considered three additional baselines (UnIF, Neural atoms and EWALD). Unfortunately, to the best of our knowledge, the implementation of UniF is not publicly available; therefore, we conducted experiments with Neural atoms and EWALD. Specifically, we employed SchNet and DimeNet enhanced with Ewald-based message passing and Neural Atom interactions, respectively as reported in their original works. The results, summarized below, show that our proposed DualEquiNet consistently outperforms these strong baselines across all evaluated datasets (RMSE reported below).

W3: The proposed architecture adds several new modules (dual-neighborhood construction, bidirectional cross-space attention, CSIP projections) and new hyperparameters without clear instructions on how to set them. Without design heuristics or simpler versions, it is hard for practitioners to adapt, tune, or extend the method to new datasets. This could make it harder to reproduce and use it more widely.

RW3: Thank you for pointing this out. We argue that we did not introduce many new hyperparameters. The only new hyperparameter we added is the threshold used to construct SH-based neighborhoods ($d_{SH}$). In other modules, such as CSIP, we make the coefficients learnable by design—specifically to reduce manual tuning and avoid introducing additional hyperparameters, as the reviewer noted.
For $d_{SH}$, we acknowledge it is a unique component of our method and have discussed possible adaptive thresholding mechanisms in our response to RQ2 in this thread below. Additionally, if one prefers not to search over $d_{SH}$, a practical heuristic can be applied: We can compute the distribution of SH similarities (i.e., cosine similarity between SH vectors) across all node pairs in the dataset, and then select a threshold such that the number of SH-based edges is approximately equal to the number of Euclidean edges. This balances the neighborhood density between Euclidean and SH spaces and provides a data-driven way to set $d_{SH}$ without tuning.
Additionally, DualEquiNet has two depth-related parameters—$L_{atom}$ and $L_{res}$—which specify the number of layers in the atom- and residue-level modules. These are analogous to layer counts in standard GNNs, which are typically shallow. We searched over a small space {1,2,3,4} making this tuning lightweight and practical, so we don’t count this as a new hyperparameter. Overall, we have deliberately designed the architecture to minimize new hyperparameters and make it accessible for adaptation to new datasets. We will clarify this in the revised version.

Q1: How would DualEquiNet work with more complex biomolecular systems, like protein-RNA interactions?

RQ1: Following the reviewer’s suggestion, we evaluated DualEquiNet on the RNAProt dataset from RNAGlib [1], which focuses on predicting protein–RNA interactions. The results (shown below) demonstrate that DualEquiNet achieves the highest accuracy among existing baselines, thereby validating its effectiveness for protein–RNA interaction tasks.

Q2: The SH-space neighborhood is implemented by a threshold and the cosine similarity between points. Could you provide some insights on how to implement an adaptive threshold mechanism to prevent tuning hyperparameters?

RQ2: We appreciate the reviewer’s insightful suggestion. To reduce reliance on a fixed threshold d_{SH}, we can consider the following alternatives in future work. (1) Percentile-based thresholding: Each node selects neighbors whose SH cosine similarity falls within the top-p percentile of all pairwise similarities to that node: $cos(r_i, r_j) ≥ Percentile_p({ cos(r_i, r_k) }_k)$. While $p$ remains a hyperparameter, it is scale-invariant and often easier to generalize across datasets than an absolute similarity threshold. It also adapts the number of neighbors per node based on local similarity distribution.

(2) Learnable neighborhood gating:
We can further eliminate manual thresholding by introducing a learnable, soft connectivity mechanism. Specifically, for each node pair (i, j), we define: $η_{ij} = σ(w · cos(r_i, r_j) + b)$, where $\sigma(\cdot)$ is the sigmoid function, and $w, b$ are learnable parameters. $\alpha_{ij}$ can be used directly as edge weights or to probabilistically construct neighborhoods. This approach is fully adaptive and differentiable, enabling end-to-end learning of neighborhood structure in SH space.

These strategies offer practical and principled alternatives to manual threshold tuning. We plan to explore and benchmark them in future work.

Q3: Could you provide empirical results or discussion with models specifically designed for long-range tasks, such as [1,2,3], as they also share a similar design idea?
RQ3: We have now provided the comparison to the other long-range methods that you recommended. Please see RW2 earlier in this thread for the results demonstrating our DualEquiNet outperforms other long-range methods.

Please let us know if there are any remaining concerns. Thanks a lot!

Reference:
[1] A Comprehensive Benchmark for RNA 3D Structure-Function Modeling.

Thank you for your positive feedback and for recognizing our responses! We truly appreciate your time and effort throughout the review process and we will incorporate the rebuttal content in the camera-ready version of our paper.

We thank the reviewer for their valuable feedback, and we address the concern below.

Q1: Ablation explanations and difference of our euclidean only message passing version with other euclidean baselines (such as EGNN) :

We appreciate the reviewer’s observation and clarify a key source of confusion: our fully Euclidean model (DualEquiNet w/o CSIP/Cross/Dual in Ablation Table 5) is not equivalent to standard Euclidean GNN baselines. Even in this ablation, we preserve the biological organization of large molecules (e.g., atom-to-nucleotide for RNA, atom-to-amino acid for proteins) using hierarchical mean pooling. Specifically, “w/o CSIP” replaces cross-space interaction pooling with standard mean pooling, but the pooling is still applied over biologically defined hierarchical units rather than only individual atoms. This explicit incorporation of biologically meaningful structure is one of our key contributions, ensuring that even our Euclidean-only variant captures context beyond atom-level interactions used by standard atom-level Euclidean GNNs, which explains its stronger performance, and hence it already outperforms those baselines. We will make it more explicit in camera-ready version.

Originally, we did not apply hierarchical pooling to the baselines to respect their original designs, and some are not directly compatible with such pooling. However, we have now run selected Euclidean baselines augmented with hierarchical mean pooling. Their performance improves over their original versions (reported in Table 1) and matches our Euclidean-only variant but still lags significantly behind the full DualEquiNet with cross-space interaction pooling (CSIP) and dual-space message passing (e.g., 4.6%, 8.7%, and 9.0% gains over best EGNN Pooling on COVID, Ribonanza, and Tc-Ribo datasets respectively).

Integration of euclidean and spherical harmonic features leads to substantive improvement to existing work: Beyond the explanation provided above, integrating the euclidean and spherical harmonic features indeed provide consistent percentage improvements across datasets. For example, removing CSIP increases error by ~8% on Tc-Ribo and ~7% on Ribonanza, while disabling cross-space message passing further increases error by ~9% on Tc-Ribo. The combined removal of all dual-space components yields the largest degradation, with ~11% higher error on Tc-Ribo and ~7% on Ribonanza. Overall, these results indicate that DualEquiNet’s architectural contributions—particularly contextual pooling and cross-space learning—are synergistic and yield up to ~10% improvements on different tasks.

Q2. Architecture and design choices of our “fully euclidean” model/parameter count and training hyperparameters compared to other baselines:

We believe this question likely arises from the confusion addressed in the point above but we provide further details for completeness. DualEquiNet w/o CSIP/Cross/Dual uses the same design as the full-version DualEquiNet, except the ablated part, i.e., they use the same multi-head attention and the same network depth/width. For DualEquiNet and all other baselines, we constrained the number of parameters to at most 4 million and ensured that all models fit within the memory of a single NVIDIA A6000 GPU. Within this resource budget, we used Optuna to perform hyperparameter optimization and select the best configuration for each model. This ensures that our comparisons are fair and conducted under consistent computational and hyperparameter constraints for all models and baselines.

Q3. Ablation study with 10 random seeds:

As suggested by the reviewer, we have run ablations with 10 random seeds. The results are consistent with those reported in Table 5 of the paper, with similar performance and comparable standard deviations. Due to rebuttal time constraints, we were able to run it on four tasks (see table below). We found a typo in the reported standard deviation for DualEquiNet on the Ribonanza dataset in Table 5: it should be “0.558 ± 0.012” instead of “0.558 ± 0.123”. The correct value is also reflected in Table 1 of the paper and only the ablation Table 5 has this typo. With this correction, the normalized improvement on Ribonanza becomes 0.038/0.012, which is approximately 3.17x standard deviations using 10 seeds, or 0.034/0.012, which is about 2.83x standard deviations using 5 seeds. We will correct this typo in the camera-ready version.

Please let us know if there are any remaining concerns. If we have addressed your questions, we kindly request you to consider updating your score. Thank you very much for your time and thoughtful review!

Thank you for your positive feedback and for recognizing our responses! We truly appreciate your time and effort throughout the review process and we will incorporate the discussion in the camera-ready version of our paper.