Angle Graph Transformer: Capturing Higher-Order Structures for Accurate Molecular Geometry Learning

Thank you for recognizing our work. We will address your questions below.

W1: Thank you for raising questions about the accuracy of our paper's contributions. We believe our core innovation lies in proposing a fundamentally different paradigm for direct modeling of higher-order structures. While models like QuinNet and ViSNet indeed introduced four or even five-atom interactions to enhance model expressiveness and accuracy, these methods essentially still focus on local representations of atomic nodes and chemical bonds, implicitly capturing higher-order features through complex combinatorial operations between atom-level tokens. In contrast, our approach directly transforms higher-order graph structures into independent token representations, enabling the model to directly learn and represent structural patterns in molecules, which is crucial for model interpretability and effective utilization of expert prior knowledge. From an information propagation perspective, in traditional methods, higher-order structural information (such as four-body and five-body interactions) must propagate gradually along the graph topology, creating significant information bottlenecks. As TGT research points out, even information exchange between adjacent embeddings is restricted. Our method, through direct structural token representation, not only avoids this bottleneck but also enables efficient access and utilization of these key higher-order information by all graph nodes, providing a more effective framework for learning molecular structural information.

W2: Thank you for your deep consideration of chirality issues. Consider a simple but representative example: in three-dimensional space, four atoms A, B, C, and D, where A, B, C form a planar triangle, and D is positioned on one side of this plane. Let: r₁, r₂, r₃ represent the distances between AB, BC, and AC respectively, θ represent the angle ∠ABC, and φ represent the dihedral angle ABCD. It can be proven that when D rotates around the BC axis to position D' (i.e., φ becomes 2π-φ):

1. All original interatomic distances remain unchanged: d(A,B) = r₁, d(B,C) = r₂, d(A,C) = r₃, d(A,D) = d(A,D'), d(B,D) = d(B,D') d(C,D) = d(C,D')
2. For any point P(x,y,z) in space, after rotation angle α around the z-axis, the new coordinates P'(x',y',z') satisfy: x' = x·cos(α) - y·sin(α) y' = x·sin(α) + y·cos(α) z' = z

For the dihedral angle transformation (φ → 2π-φ), it can be proven through appropriate coordinate transformations that both configurations have identical distance matrices but represent different molecular conformations. This situation is common in real molecules, especially those containing chiral centers or flexible structures. For instance, in protein backbones, even with fixed bond lengths and angles, multiple possible conformations exist (such as cis and trans conformations) that share identical distance matrices but different dihedral angles. Therefore, relying solely on interatomic distances indeed cannot fully determine molecular 3D structure, particularly in cases involving chirality and conformational isomerism. We understand your point that most molecules are not completely centrally symmetric geometric configurations, but when local chiral structures are at molecular terminals, the differences in distance matrices become extremely subtle if other asymmetric structures are far from the local chiral structure, as these structures inherently have larger distances, and rotational changes due to chirality are not prominent. Thus, predicting chirality solely through distance matrices is extremely sensitive to noise. This precisely underscores the importance of directly modeling bond angles and dihedral angles.

We conducted systematic evaluations on the PCQM4M training set, which contains 3,803,453 molecules, of which 1,772,922 (46.61%) have chiral centers. The evaluation method compares model-predicted 3D conformations with high-precision DFT-calculated conformations, using angular deviations near chiral centers as criteria, with a deviation threshold of π/6.

Experimental results show that AGT outperforms the baseline model TGT in three key metrics: Bond angle MAE: AGT achieves 0.209 rad compared to TGT's 0.246 rad, a 15.0% reduction; Torsion angle MAE: AGT achieves 0.334 rad versus TGT's 0.597 rad, a significant 44.1% reduction; Chirality prediction accuracy: AGT reaches 74.7%, far exceeding TGT's 32.5%, a 130% improvement. These quantitative results strongly confirm AGT's superiority in chiral structure modeling, particularly in complex torsion angle prediction and overall chirality determination tasks.

W3: Thank you for your guidance regarding model lightweight design. We understand your concern about the trade-off between model scale and performance. We have supplemented comparisons of smaller-scale AGT models with various specifications of TGT and Unimol+ models, with results as follows:

From the table analysis: First, at similar layer counts (6 layers), while AGT-68M has more parameters than Unimol+-27.7M, it achieves significant performance improvements (MAE reduced from 71.4 meV to 69.4 meV). At medium scale (12 layers), AGT-127M (69.1 meV) outperforms both Unimol+-52.4M (69.6 meV) and TGT-116M (70.9 meV). Particularly at larger scales, AGT-241M achieves an MAE of 66.2 meV, showing significant improvement over TGT-203M's 67.1 meV.

This performance advantage, which becomes more apparent as model scale increases, validates our theoretical hypothesis: accurately modeling higher-order molecular geometric structures (such as bond angles and dihedral angles) requires sufficient parameter capacity to capture complex spatial relationships. While larger model scales indeed bring additional computational overhead, considering the importance of prediction accuracy in molecular property prediction tasks, we believe this is a reasonable trade-off. Meanwhile, we provide model versions of different scales (68M to 241M), allowing users to choose appropriate model configurations based on specific application scenarios and computational resource constraints.

Thank you for your comments. Since the concerns raised in Weaknesses and Questions sections are largely overlapping, we have consolidated our responses under Questions.

Q1: We appreciate your attention to the quantitative analysis of chirality prediction. We conducted a systematic evaluation on the PCQM4M training set, which contains 3,803,453 molecules, including 1,772,922 molecules with chiral centers (46.61%). The evaluation methodology compares model-predicted 3D conformers with high-precision DFT-calculated conformers, using angular deviations around chiral centers as the assessment criterion, with a deviation threshold of π/6.

Experimental results demonstrate AGT's superiority over the baseline TGT model across three key metrics: Bond angle MAE: AGT achieves 0.209 rad, a 15.0% reduction compared to TGT's 0.246 rad; Torsion angle MAE: AGT reaches 0.334 rad, significantly lower than TGT's 0.597 rad by 44.1%; Chirality prediction accuracy: AGT attains 74.7%, substantially outperforming TGT's 32.5% with a 130% improvement. These quantitative results strongly validate AGT's excellence in modeling chiral structures, particularly in complex torsion angle prediction and overall chirality determination tasks.

Q2: Thank you for raising concerns about computational complexity and inference time. Below are the performance and efficiency comparisons between AGT and other baselines on OC20 valid sets and PCQM4Mv2.

On the OC20 dataset, we conducted comprehensive comparisons with both pre-trained and non-pre-trained methods. AGT's training time here specifically refers to fine-tuning duration. Results show that AGT's inference time is comparable to non-pre-trained methods, with fine-tuning time (240 minutes) similar to mainstream models like DimeNet++ (230 minutes), GemNet-T (200 minutes), and SphereNet (290 minutes). While ComENet shows faster training speed (20 minutes), AGT achieves significantly better performance in key metrics such as energy MAE (399.5 vs 588.8 meV) and FwT (8.99% vs 3.56%), demonstrating the value of pre-training strategies. Moreover, compared to Uni-Mol+, AGT achieves superior performance while substantially reducing pre-training time (34 days vs 112 days using A100 GPU), showing a better balance between efficiency and performance.

Based on experimental results, we comprehensively analyze AGT's method from both efficiency and effectiveness perspectives. Regarding computational complexity, where N represents the number of atoms, AGT requires O(N³) complexity for standard atom and pair embedding interactions, plus additional interactions between bond angles and torsion angles. In typical molecules, the number of bond angles ranges from 1.5N to 2N, and torsion angles from N to 2N, resulting in an additional computational complexity of O(N²), yielding an overall complexity of O(N³) + O(N²). On the large-scale PCQM4Mv2 dataset, AGT demonstrates an excellent balance between performance and computational efficiency. Specifically, an AGT model with only 68M parameters achieves comparable performance to Uni-Mol+ (77M parameters) with MAE of 69.4 meV versus 69.3 meV, while significantly reducing training time (approximately 14 days versus 40 days using A100 GPU). Compared to TGT, despite incorporating additional angular information and direct angle modeling mechanisms, AGT maintains similar training efficiency (approximately 34 days versus 32 days using A100 GPU) while achieving superior performance.

Q3: We appreciate the reviewer's question. Analyzing the experimental results on the QM9 dataset, we observe uneven performance across different tasks. Specifically, AGT achieves significant success in certain property prediction tasks, such as energy-related metrics (εH: 8.8, εL: 9.1, Δε: 16.4, all best in class) and some physical properties (Cᵥ: 0.020, tied with TGT for best). The inability to achieve comprehensive leadership in other metrics may stem from several factors:

1. Mismatch between pre-training objectives and specific tasks: AGT's pre-training optimization primarily focuses on overall molecular structure representation, potentially not fully capturing detailed features required for certain physicochemical properties, such as μ requiring better characterization of atomic electronegativity differences.
2. Task specificity: Some property prediction tasks may require more specialized model architectures or loss function designs, which general pre-trained models might struggle to accommodate. Particularly, the pre-training supervision signal from PCQM4Mv2's HOMO-LUMO gap may bias the pre-trained model toward similar metrics.
3. Optimization trade-offs: To maintain model generality, the pre-training process may compromise performance on certain specific tasks.

Q4: The performance of different-sized AGT models is shown in the table in Q2, where we systematically analyzed the trade-off between model scale and performance. Results show that 6-layer AGT (68M parameters) achieves an MAE of 69.4 meV, comparable to 18-layer Unimol+ (77M parameters) at 69.3 meV. As model layers increase, 24-layer AGT (241M parameters) reduces MAE to 66.2 meV, significantly outperforming 24-layer TGT (203M parameters, 67.1 meV MAE). Notably, although AGT's theoretical complexity of O(N³) + O(N²) is slightly higher than baseline models' O(N³), 12-layer AGT (127M parameters) maintains competitive performance (69.1 meV MAE) while reducing training time from 38 to 20 GPU days, with corresponding inference time reduction. These results indicate that AGT architecture is competitive even at smaller scales and can better leverage its structural modeling advantages as parameter count increases.

Q5: Our experiments with AGT span datasets with significantly different molecular scales: PCQM4Mv2 (average 15 atoms), downstream tasks MolHIV and MolPCBA (average 26 atoms), and larger-scale OC20 systems (approximately 80 atoms). We have not observed significant performance degradation with increasing molecular size. From a theoretical perspective, molecular scale processing is primarily limited by two factors:

1. Computational resource constraints: Mainly determined by GPU memory capacity, which dictates the maximum molecular system size processable with 3D conformer data
2. Algorithm complexity: For large-scale molecular systems (like proteins), local attention mechanisms or sliding window strategies can be introduced to optimize angle interaction computation ranges, effectively reducing computational complexity

These findings suggest that through appropriate algorithmic optimization and computational strategies, the model has the potential to handle larger-scale molecular systems without significantly impacting its predictive performance, enabling applications across a broader range of molecular systems.

As we approach the final hours of the rebuttal period (4 hours remaining), we wanted to ensure that we have adequately addressed all your concerns. We have carefully revised our manuscript according to your valuable suggestions and responded to all the questions raised thus far. If there are any remaining questions or aspects that require further clarification, we would be most grateful to address them before the deadline.

Your insights have been instrumental in improving our work, and we remain committed to ensuring all your concerns are fully addressed. Thank you for your time and dedication to the review process.

W1: We appreciate the reviewer's concern about computational efficiency. Indeed, encoding higher-order structures as tokens increases model complexity, presenting challenges for AGT's practical deployment. However, we believe the computational overhead introduced by incorporating bond angles and torsion angles is well justified. First, the increased model complexity expands information capacity and enhances multi-dimensional feature representation, theoretically improving the model's discriminative power. This has been thoroughly validated in local chirality recognition, where we formally prove that incorporating angular information enables accurate chirality discrimination. Second, the introduction of higher-order structures simplifies the feature space and enhances model robustness, effectively preventing the accumulation of basic structural errors. Furthermore, from an interpretability perspective, since molecular property changes primarily originate from local group modifications, focusing on higher-order structures rather than individual atoms and bonds better aligns with chemical intuition. Based on the reviewer's suggestion, we have proposed and validated computational complexity optimization strategies that significantly reduce computational requirements while maintaining model performance, as detailed in Answer for Questions.

W2: We appreciate the reviewer's comments regarding the dependency on 3D conformer data. AGT was specifically designed to address the scarcity of high-precision 3D conformer data. Specifically, during conformer prediction, the model employs self-supervised learning methods, utilizing limited high-precision 3D conformer data for denoising training, enabling it to generate DFT-level 3D conformers in downstream tasks where high-precision conformer data is lacking. Notably, after pre-training, AGT only requires low-computational-cost 3D conformers generated by open-source tools like RDKit as input. Consequently, AGT can obtain high-quality initial representations on datasets with high-quality 3D data (such as QM9), while maintaining stable performance on datasets with noisy or incomplete data (such as MolHIV, MolPCBA, and PCQM4Mv2 valid set).

Thank you for acknowledging our work and raising questions about computational efficiency and scalability. In AGT, the additional computational resource consumption primarily focuses on two aspects. First is the attention interactions between all bond angles and torsion angles, where AGT employs full attention interaction for optimal performance, calculating mutual attention weights between all bond and torsion angles. An intuitive optimization approach is to use fixed-size sliding windows for local attention computation based on the sequence order of bond and torsion angles, rather than global attention. Each angle only computes attention with k nearby angles, avoiding the substantial resource consumption of global attention matrices. Second is the interaction between different levels of graph structures. AGT uses Geometric edge-angle interaction, communicating paired embeddings between triangle edges corresponding to bond angles and tetrahedral edges corresponding to torsion angles. Under computational resource constraints, we can directly use two bond angle graph structures corresponding to torsion angles instead of six tetrahedral edge pair embeddings to communicate with torsion angle embeddings, simplifying computation. While this may increase model fitting difficulty, it significantly reduces computational resource consumption. The model's computational complexity reduces from O(N³) + O(N²) to O(N³) + O(N).

We conducted comparative experiments on PCQM4Mv2 using these two optimization techniques for reducing computational and space complexity (named AGT Simplify Angle with window size k=5). We also experimented with smaller-scale AGT models to examine AGT's performance under lower computational complexity.

Experimental results show that at comparable parameter scales (AGT 6-layer vs. Unimol+ 18-layer), AGT demonstrates performance levels comparable to Unimol+. As model scale expands, AGT's performance advantages over Unimol+ gradually emerge, while also achieving significant improvements over TGT under similar parameter conditions. This phenomenon indicates that models directly encoding molecular higher-order structure representations require sufficient parameter capacity to fully realize their modeling advantages. Notably, the computationally optimized AGT maintains its performance advantages even with parameter scales comparable to TGT, strongly demonstrating AGT's superiority in molecular property prediction even in resource-constrained environments.

Thank you for your valuable feedback and constructive suggestions. We have carefully revised our manuscript according to your comments, and the updated version has been uploaded. For your convenience, all modifications are highlighted in blue in the revised PDF. We would greatly appreciate your review of these changes. Should you have any additional questions or concerns, please do not hesitate to raise them. We remain committed to addressing any further feedback to improve the quality of our work.

Thank you for your comments and suggestions. Regarding the concerns about lacking baseline methods and complete dataset demonstrations mentioned in Weaknesses 1, 2, and 4, we have supplemented our work with comprehensive performance comparisons on the OC20 IS2RE validation set and all metrics on QM9.

After adding baselines, AGT achieves state-of-the-art performance across various distributions on the OC20 IS2RE dataset. It significantly outperforms EquiFormer+NN and EquiFormer, which were trained on the complete OC20 dataset (ALL+MD), and surpasses EquiformerV2+NN(direct), the best non-pre-trained method trained directly on OC20 IS2RE.

Analyzing the QM9 dataset results, we observe varying performance across different tasks. Specifically, AGT achieves significant success in certain property prediction tasks, such as energy-related metrics (εH: 8.8, εL: 9.1, Δε: 16.4, all best in class) and some physical properties (Cᵥ: 0.020, tied with TGT for best). AGT also achieves near-optimal performance in optical and quantum features like α and ZPVE. For U₀, U, H, G, and R², AGT outperforms existing pre-trained models (e.g., Transformer-M).

Regarding the innovation concerns raised in Weakness 3, we believe our core innovation lies in proposing a fundamentally different paradigm for modeling higher-order structures. While models like DimeNet, GemNet, and SphereNet utilize angle information for implicit modeling of higher-order interactions, these methods essentially rely on local representations based on atomic nodes and chemical bonds, indirectly capturing higher-order features through complex combinatorial operations between atom-level tokens. In contrast, our approach directly transforms higher-order graph structures into independent token representations, enabling the model to learn and represent molecular structural patterns directly, which is crucial for model interpretability and integration with domain knowledge. From an information propagation perspective, in traditional methods, higher-order structural information (like bond angles and torsion angles) must propagate gradually along the graph topology, creating significant information bottlenecks. As TGT research points out, even information exchange between adjacent embeddings is restricted. Our method, through direct structural token representation, not only avoids this bottleneck but also enables efficient access and utilization of these key higher-order information by all graph nodes, providing a more effective framework for learning molecular structural information.

Regarding Weakness 5, we will address this in Q2 of Answer for Question below.

For the writing and typesetting issues raised in Weakness 6, we have made the necessary corrections in the paper and greatly appreciate your feedback.

Q1: Thank you for raising concerns about data leakage. We compared PCQM4Mv2 molecules in SMILES format with downstream evaluation datasets MolHIV, MolPCBA, and LITPCBA. No identical molecules were found between PCQM4Mv2 and LITPCBA. PCQM4Mv2 shares 112 molecules with MolHIV, 149 with MolPCBA, and 10 with QM9. Considering PCQM4Mv2 contains 3.3 million molecules, while MolHIV, MolPCBA, and QM9 have 40K, 430K, and 130K molecules respectively, the highest overlap rate is less than 0.25%. Given that all dataset tasks are vertical, we believe data leakage is negligible. Using Morgan fingerprints, a common method in molecular chemistry for measuring structural similarity (scale 0-1, where 0 indicates complete dissimilarity and 1 indicates identity), the average similarities between PCQM4Mv2 and downstream datasets MolHIV, MolPCBA, and QM9 are 0.0508, 0.0523, and 0.0586 respectively. Statistically, this indicates extremely low similarity between pre-training and downstream evaluation datasets, validating the effectiveness of AGT's pre-training evaluation on downstream tasks.

Q2: Thank you for your attention to computational complexity and inference time. The detailed training and inference times and computational complexity comparisons on PCQM4Mv2 and OC20 are shown in the table above.

Based on experimental results, we comprehensively analyze AGT's method from both efficiency and effectiveness perspectives. Regarding computational complexity, where N represents the number of atoms, AGT requires O(N³) complexity for standard atom and pair embedding interactions, plus additional interactions between bond angles and torsion angles. In typical molecules, the number of bond angles ranges from 1.5N to 2N, and torsion angles from N to 2N, resulting in an additional computational complexity of O(N²), yielding an overall complexity of O(N³) + O(N²). On the large-scale PCQM4Mv2 dataset, AGT demonstrates an excellent balance between performance and computational efficiency. Specifically, an AGT model with only 68M parameters achieves comparable performance to Uni-Mol+ (77M parameters) with MAE of 69.4 meV versus 69.3 meV, while significantly reducing training time (approximately 14 days versus 40 days using A100 GPU). Compared to TGT, despite incorporating additional angular information and direct angle modeling mechanisms, AGT maintains similar training efficiency (approximately 34 days versus 32 days using A100 GPU) while achieving superior performance.

On the OC20 dataset, we conducted comprehensive comparisons with both pre-trained and non-pre-trained methods. AGT's training time here specifically refers to fine-tuning duration. Results show that AGT's inference time is comparable to non-pre-trained methods, with fine-tuning time (240 minutes) similar to mainstream models like DimeNet++ (230 minutes), GemNet-T (200 minutes), and SphereNet (290 minutes). While ComENet shows faster training speed (20 minutes), AGT achieves significantly better performance in key metrics such as energy MAE (399.5 vs 588.8 meV) and FwT (8.99% vs 3.56%), demonstrating the value of pre-training strategies. Moreover, compared to Uni-Mol+, AGT achieves superior performance while substantially reducing pre-training time (34 days vs 112 days using A100 GPU), showing a better balance between efficiency and performance.

Q3: Please refer to Answer for Weakness and Q2.

Q4: Thank you for your attention to model scalability. AGT's computational requirements when handling molecules of different sizes are primarily influenced by two factors: First, the scaling of angular information. As molecule size increases, the number of bond angles and torsion angles increases significantly, directly affecting attention computation complexity. We propose a local structure-based optimization strategy: using fixed-size sliding windows for local attention computation instead of global attention. Specifically, each angle only computes attention with k nearby angles, reducing computational complexity from O(n²) to O(kn), where n is the total number of angles and k is the fixed window size. This optimization is particularly important for large molecules. Second, the computational overhead of hierarchical graph structure interactions. When handling large molecules, we can simplify the Geometric edge-angle interaction mechanism. Specifically, for torsion angle modeling, we can directly use communication between its corresponding two bond angle graph structures instead of the complete six-edge pair embedding of a tetrahedron. While this might slightly affect model performance, it significantly reduces computational complexity, making the model more suitable for handling large molecular systems. These optimization strategies enable AGT to effectively handle larger molecular structures while maintaining predictive accuracy.

Q5: Thank you for your suggestions. Regarding reducing pre-training requirements, we have indeed considered two viable optimization approaches: Given AGT's direct modeling of higher-order structural features, we believe we can reduce dependence on high-precision conformer pre-training through structure-aware self-supervised learning strategies. Specifically, we can design self-supervised tasks using molecular local structural information, such as predicting the atomic and chemical bond composition of local bond angles and torsion angle structures (2D information), or performing 2D reconstruction of molecular scaffolds. This approach can help the model learn effective structural representations with fewer high-precision ensemble annotation data. Additionally, given AGT's hierarchical structure representation's good generalization capabilities, we can explore combining few-shot learning techniques by designing targeted task adaptation layers to accelerate specific task learning processes. This method could significantly reduce dependence on large-scale pre-training while maintaining model performance on downstream tasks. These alternative approaches show promise in maintaining model performance while reducing computational burden, and we will conduct relevant experimental validation and optimization in the future.

Thank you for your comments and suggestions on our paper. We have incorporated your feedback and conducted additional experiments and analyses, which we will address point by point.

W1: In the new version, we have modified the citation format and provided the full name of DFT in the Introduction section, along with a basic introduction to density functional theory (DFT) in the appendix. We appreciate your suggestion.

W2: Thank you for raising the question about 3D conformer initialization. AGT introduces low-precision 3D conformers to accelerate the conformer prediction phase and reduce training difficulty. In fact, TGT also has a version based on RDKit-generated low-precision conformers, and they mentioned using this version in their main experiments to accelerate fitting. Similarly, AGT does not depend on low-precision 3D conformers; we simply didn't emphasize this as our main contribution or feature. Below are comparative experiments of AGT using only 2D structures for conformer prediction, showing similar performance between the two approaches, with the only difference being that AGT using only 2D structures requires longer training time and more iterations. Moreover, for current tasks, generating molecular 3D conformers using open-source tools like RDKit has extremely low computational cost, so we believe it's reasonable to use low-precision 3D conformers as input to improve model fitting speed.

W3: Thank you for your attention to the ablation studies. We conducted additional experiments on Directed Cycle Angle Loss and Hierarchical Virtual Node separately to understand the contributions of different components, detailed in Answer for Questions. We demonstrated the performance of different Mode Distributions on MolPCBA and MolHIV downstream tasks, with pre-training Mode Distribution maintained at 4:1.

From the table, we find that MolHIV performs best at 8:1 Mode Distribution, while MolPCBA shows optimal performance at 4:1. However, different Mode Distributions have relatively minor impacts on downstream tasks, with no significant distinctions between them. This might be because the overall weights of distance and angle losses in downstream tasks are relatively small (AGT default setting is 0.1).

W4: Thank you for your concern about potential challenges with the DCA loss function. The DCA loss function addresses the circular nature of angles that prevents optimization along the shortest path like scalar distances. It doesn't add additional computational complexity, with final computational complexity correlating directly with the number of angles to be calculated. For large molecular datasets, a potential optimization strategy is to ignore angles that remain fixed in certain groups and focus only on angle changes near active sites or key structural properties. HIV-1 protease serves as a typical example, where the active site is located at the interface between two monomers. When studying drug molecule binding with the protease: most of the protein backbone structure (about 90%) remains relatively stable, making dihedral angle calculations unnecessary for these regions; focus can be placed on conformational changes near active sites, particularly the dihedral angles between Ile50-Gly51 and Gly51-Gly52, which directly affect the substrate binding pocket's opening and closing. This strategy significantly reduces the number of angles to calculate while maintaining accurate description of key conformational changes.

Regarding noise resistance, the DCA loss function actually reduces noise impact by selecting optimal optimization paths. For instance, when handling angles near 0° or 360°, traditional loss functions might oscillate between these values, while DCA provides consistent optimization direction (clockwise or counterclockwise). Thus, it may reduce rather than increase noise sensitivity. However, your point about noise sensitivity is valid, particularly when angles approach 180°, where DCA, like traditional angle loss functions, may oscillate in optimization direction. We are considering implementing a determined optimization direction for angles near 180° to enhance DCA loss function's noise resistance.

Weight adjustment is a common issue in all multi-loss function joint training, relating to both data characteristics and dataset tasks. Our experimental experience suggests that the principle for adjusting weights between angle and distance loss functions is to maintain both losses at the same order of magnitude over extended training periods, allowing AGT's overall optimization process to better balance both optimization directions and step sizes. We plan to improve this by making weight adjustment adaptive or learnable.

Thank you for your questions regarding experimental result analysis. Your questions are very important, and due to space limitations in the main text, we couldn't fully explain why certain configurations or settings produced better results. We will now explain the ablation study results.

Based on the ablation experimental results, we can draw several conclusions:

1. From the perspective of individual module contributions, all three modules improve performance, proving their effectiveness. The AGT Attention Module alone reduces MAE from 73.6 to 71.3 meV, an improvement of 2.3 meV, showing the best effect. This demonstrates that directly modeling higher-order structures and their interactions with lower-order structures effectively enhances AGT's representation capability. Using Directed Cycle Loss alone, which approximates adding bond angle and torsion angle constraints as auxiliary loss functions to the model's input embedding layer, shows the smallest gain among the three modules. This suggests that while adding angle features can benefit the model, implicit modeling of higher-order structures yields less benefit than explicit modeling, indirectly proving AGT's potential to surpass previous models that implicitly model four-body or five-body structures.
2. Regarding synergistic effects, the complete combination of three modules (1:1 configuration) achieves 70.3 meV, outperforming any single module. This indicates these modules are complementary rather than redundant. Functionally, the AGT Attention Module uses attention mechanisms to effectively extract higher-order structural features like bond and torsion angles, Directed Cycle Loss strengthens geometric constraints and reduces model optimization difficulty by providing reasonable gradient directions, and Virtual Hierarchical Node enhances hierarchical information interaction through dual virtual nodes, adjusting weights of different hierarchical features in a learnable way.
3. In Mode Distribution ratio optimization, among configurations from 1:1 to 8:1, 4:1 achieves the best results (69.1 meV). This trend shows moderate increases in distance mode weight (2:1 to 4:1) benefit performance, while excessive increases (8:1) lead to performance degradation. In PCQM4Mv2 pre-training, we observed that during the initial 50 epochs, the ratio between distance loss and angle loss values ranges approximately from 1:3 to 1:5, gradually approaching 1:2 during training. This indicates the importance of balancing distance and angle losses in early training stages, requiring appropriate balance within similar value ranges to help the model balance both optimization directions. If angle loss becomes too large, the model focuses on building distance proportions to ensure angle accuracy, making distance optimization insufficient to offset learned distance proportion changes, thus making optimization difficult. This also suggests that weights of both losses should change with training loss values, providing direction for future improvements.

As we approach the final hours of the rebuttal period (4 hours remaining), we wanted to ensure that we have adequately addressed all your concerns. We have carefully revised our manuscript according to your valuable suggestions and responded to all the questions raised thus far. If there are any remaining questions or aspects that require further clarification, we would be most grateful to address them before the deadline.

Your insights have been instrumental in improving our work, and we remain committed to ensuring all your concerns are fully addressed. Thank you for your time and dedication to the review process.