Reaction Graph: Towards Reaction-Level Modeling for Chemical Reactions with 3D Structures

Thank you for acknowledging that our method is novel, crucial, and effective.
We also appreciate your valuable suggestions for improving this work.

Ⅰ. Impact of Reac and 3D Info in RG

1. Impact on Reaction Types

Tab 1: Impact of Reac and 3D info on 12 reaction types.

3D is more critical for Unknown reactions (type 0), as their mechanisms are diverse and reaction patterns are difficult to capture. For C–C bond formation (type 3) and Oxidation (type 8), the contribution of Reac info is greater.

2. Impact on Specific Datasets

Tab 2: Impact of Reac and 3D info on different datasets.

Both 3D and Reac info contribute to condition prediction. For reaction classification, which is directly related to the reaction change, Reac info is more effective.

Ⅱ. Error Analysis

1. Condition

The overall error pattern is in Appendix Fig 18. A: US20110105766A1 and B: US20040204386A1 are representative failure cases.

• Case A. Correct: PdCl₂ (0.05%) Pred: Pd(PPh₃)₄ (1.82%)

  Conditions with low frequencies (0.05%) may be misclassified, but the Pred still exhibit similarity to the ground truth.

• Case B. Correct: CuBr₂ (0.01%<) Pred: None (86.81%)

  For extremely rare conditions (0.01%<), model struggles and classifies them as None. Data augmentation or pretraining are promising solutions for future exploration.

2. Yield

Apart from low label quality, the non-smooth relation between structure and yield is the bottleneck of yield prediction.

We find the top-5 closest reactions of C using RXNFP. They have similar structures, yet the yield variance is large (0.5–0.94). This distinction hinders model from correctly predicting D and F.

C: CCC(C)(C)N.CCN(C)C.Cl.[Cl-]>>CCC(C)(C)NCCN(C)C

Tab 3: Correct and Pred yields of 5 similar reactions.

3. Reaction Classification

The overall error pattern is in Appendix Fig 21. H and I are representative failure cases.

• Case H. Correct: FGI; Pred: Unrecognized

    C1=C[C@H]2[C@H]3C=C[C@H](C3)[C@H]2C1>>C1C=CCC=1
  

  For rare types of reaction changes, the model may misclassify them as Unknown.

• Case I. Correct: Oxidations; Pred: FGI

    Cl.N#CC1CCCC=1N.NO>>N#CC1CCCC=1NO
  

  In this case, the annotated and predicted types are both acceptable, but the Pred seems more reasonable.

4. Other Tasks

Detailed analysis is in Sec H1 and H2.

Ⅲ. Hyperparameter Selection and Training Strategy

Hyperparameter Selection

Results show that the current setting is optimal, while the performance may scale with the hidden dim.

Tab 4: Impact of hyperparameters.

Training Strategy

According to Tab 5 and Appendix Fig. 19, our proposed 2-stage training strategy effectively addresses the negative transfer problem in multi-task learning like condition prediction.

Tab 5: Impact of two-stage training strategy.

Ⅳ. Data Quality

Accuracy of 3D Coords

We control the 3D accuracy by adding gaussian noise to the bond lengths. Results in Tab 5 show that when noise level is within a certain margin (5%), the performance decrease is minor (0.03).

Tab 6: Impact of 3D accuracy on 1/8 USPTO-Condition.

Label Quality

Label density is an important indicator of label quality. We investigate its impact on Pistachio-Condition, where the model suffers from sparse condition annotations. Results in Tab 6 show that label quality is more critical to model performance than 3D accuracy, and is the primary bottleneck.

Tab 7: Impact of label density on Pistachio-Condition.

Ⅴ. Computational Time Analysis

As shown in Appendix F2, for most of the reactions, the construct and inference time of RG are <50ms, meeting the real-time requirement. The inference time of large dataset (USPTO test set) are <3min, demonstrating the efficiency of RG. We are optimistic that, with the advancement of the conformer prediction, the speed of RG construction will further improve.

Thank you for acknowledging that our method "outperforms SOTA methods" and that the "results clearly support the claim".
We also appreciate your insightful suggestions for improving this paper.

Ⅰ. Separate vs. Unified Node

RG separates atoms in reactant and product into two nodes, which can preserve the uniqueness of molecular structures and facilitate structural modeling. CGR unifies atoms into a single node, causing their structural features to become entangled during message passing. This may hinder structural understanding.

Results in Tab 1 show that RG outperforms CGR in condition and yield prediction, which rely on structural modeling.

Tab 1: Comparison of RG (separate nodes) and CGR (unified nodes).

Ⅱ. Non-reactant Species

For condition prediction, we exclude non-reactant species from graph and take them as prediction target, avoiding data leakage. For yield prediction, we keep them in the graph as they have a significant impact on the yield.

Ⅲ. Why not Cartesian Coords or Torsional Angles

Our work aims to predict reaction properties (e.g. conditions), which are invariant to the specific conformations of molecules. RG uses bond lengths and bond angles, which vary slightly across different conformations and suit the task. Cartesian Coords and Torsion Angles vary significantly between conformations, introducing redundant information and hindering model learning.

Results in Tab 2 support this point.

Tab 2: Comparison of 3D structure representations.

Ⅳ. Tradeoffs in 3D Structure Calculation

The reason for using equilibrium structures is the same as in Ⅲ.

ETKDG+MMFF provides controllable errors (avg 5%< with DFT) and high efficiency (20ms< for 100 atoms). DFT are unaffordable (>10min for 6 atoms and polynomial growth), as USPTO contains >680k samples and reactions with >300 atoms

Results in Tab 3 show that controllable errors (5%) in 3D coords have a minor impact (0.003).

Tab 3: Impact of 3D error on 1/8 USPTO-Condition.

Ⅴ. Challenges of Datasets

• The key bottlenecks of USPTO-Yield lie in the low label quality and missing condition annotations[1]. Although large in scale, the distribution remains sparse due to the large variety of reaction types, making it hard to learn the non-smooth relation between reaction and yield.
• B–H/S–M are come from HTE, which provide dense, high-quality labels but limited data(<10K samples, <50 molecules). Test sets have additives held out from train set, which places high demands on model's generalization on small data scale.

More stats and discussions can be found in Appendix Sec G.

Ⅵ. Writing, Content Arrangement, and Figures

Your suggestions are very helpful for improving the manuscript's quality.

Introduction Revision

• AI4Chem: In the field of chemistry [2], AI enables precise spectral analysis [...] and quantum chemical simulation [...], improves inverse design of molecular structure [...] and retrosynthesis planning [...].

• GNN: Among various representation methods, molecular graphs [3,4] have proven inherently advantageous for various chemical tasks [...].

Figure Revision

https://huggingface.co/reactiongraph/Revision.

Section Arrangement

Sec 3.2 focuses on exploring suitable 3D features, whereas Sec 3.3 focuses on specific task.

But you made well-reasoned point that these two Secs are not logically parallel. Therefore we rearrange them as followed:

• 3.1 The roles of Reaction and 3D Information
  • 3.1.1 The Effect of Reaction Information
  • 3.1.2 The Effect of 3D Information
• 3.2 Reaction-related Tasks
  • 3.2.1 Reaction Condition Prediction
  • 3.2.2 Reaction Yield Prediction
  • 3.2.3 Reaction Classification

Ⅶ. Data Requirement

Reac info is effective across different data scales. 3D info requires larger amounts of data.

Models trained from scratch on B–H and S–M shows advantage of reac info (Sec 3.4), and we also conduct experiments on 1/8 of USPTO dataset (see Tab 4).

Tab 4: Results on 1/8 data scale.

We use different scaffolds on Pistachio to further explore the data requirements of 3D info. Tab 5 shows that limited data scale hinders the model from learning 3D priors. Results in [5] also leads to a similar conclusion.

Tab 5: Results of different data scale.

[1] Prediction of chemical reaction yields using deep learning

[2-4] correspond to the papers in your comment.

[5] Uni-Mol: A Universal 3D Molecular Representation Learning Framework

Thank you for acknowledging that this paper proposes a "new graph representation" and is "effective in various reaction tasks." We also appreciate your suggestions.

Ⅰ. Clarification on Reaction Separation and Information Loss in RXN Hypergraph

Seperate Reactions: In the RXN Hypergraph, reactants and products are represented as separate graphs, rather than integrating the entire reaction into a single connected graph.

Information Loss: This representation lacks atom mapping between reactants and products, leading to the loss of key information on bond breaking, bond formation, and atomic reorganization.

Experimental Verification: We try to convert the standard non-connected RXN Hypergraph (w/o Reac) into a connected graph (w/ Reac). The improved performance confirms our claim (see Tab 1).

Tab 1: Impact of reaction connectivity.

Ⅱ. Results of MG and RG in Typical Framework

We use the typical used RGCN implementation from DGL offical repo. As in Tab 2, RG outperforms MG on all the tasks.

Tab 2: Results of RG and MG on RGCN.

We also test MG on its own SOTA framework UniMol. As in Tab 3, our method shows advantage in reaction tasks.

Tab 3: Results of RG and MG on their own SOTA framework.

Ⅲ. Details about Ablation

Method Overview: The main contribution of this work is proposing Reaction Graph (RG), a new representation for chemical reactions, characterized by two key features:

• Reaction info are used to model reaction change, which is incorporated through reaction egde;
• 3D structures are incorporated through bond lengths and angular edges.

Ablation Settings: All ablations are conducted on the same framework, with same hyperparameter setting. We remove reaction edges for reaction info ablation, and set all edge lengths to 0 for 3D ablation.

Ⅳ. Design and Effect of Each Module in the Framework

Our main contribution is proposing a novel reaction graph representation. Still, we have carefully considered the framework and conducted extensive experiments. For more details, please refer to Appendix Sec.H.7.

1. Edge Embedding

We employ typical embedding layer for edge type, and RBF for edge length. RBF lifts edge lengths into high-dim vector by smooth mappings, which help in capturing variations in continuous data. It focus more on local structural pattern, which is suitable for tasks involving local continuous spatial relationship (e.g. molecular structures).

As in Tab 4, RBF outperforms linear projection and discretization.

Tab 4: Comparison of different edge embedding methods.

2. Vertex-Edge Integration

We map each edge type to a unique learnable message function $f_e(v,e,l_e)$, which computes messages for node feature update. Through this approach, the network can handle bond, reaction and angular edge in a targeted manner, as their semantics are different.

Compared with other vertex-edge integration methods, our method achieves better accuracy (see Tab 5).

Tab 5: Comparison of different vertex-edge integration methods.

3. Aggregation

To capture RG's global representation, we use an LSTM with attention aggregation. It helps the network focus on the sites related to the reaction change. Ablation results validates the effectiveness of each component (See Tab 6).

Tab 6: Ablation results of aggregation module.

Ⅴ. Calculation of Edge Length $l_{ij}$

First, we input each molecule into RDKit and calculate 3D atomic coords using ETKDG + MMFF. The 3D coords of atoms $i$ and $j$ are $p_i$ and $p_j$, respectively. The length of edge $e_{ij}$ is calculted as:

$$l_{ij} = \begin{cases} 0, & \text{$e_{i,j}$ is reaction edge} \\\\ ||p_i - p_j||_2, & \\text{otherwise} \end{cases}$$

Ⅵ. How 3D Info is Included

1. We calculte the edge length $l_{ij}$ (refer to Ⅴ).
2. We input $l_{ij}$ to RBF kernel network to get the edge length embedding $\boldsymbol{l}_{ij}$.
3. We concat $\boldsymbol{l}ij$ with edge type embedding $\boldsymbol{e}_{ij}$.
4. Edge length and type features are combined with vertex features through vertex-edge integration, thereby incorporating 3D info into RG.

Ⅶ. Typo Correction

The typo in line 3330 has been corrected, and the text has been re-proofread. Thank you for your reminder.

Thank you for acknowledging that our method "shows superiority on various tasks." We also appreciate your suggestions.

Ⅰ. Comparison with Recent Studies

The performance of our method outperforms recent methods ReaKE [2] and UniRXN [3] (see Tab 1).

Tab 1: Comparison with recent baselines. [*] are reported in [2-3]

CHESHIRE [1] may NOT be directly related to our paper. It focuses on filling missing edges in a Genome-scale Metabolic Graph, while we focus on predicting chemical reaction properties.

Ⅱ. Innovation in Using 3D

Previous methods typically use bond lengths and bond angles.
However, their differing physical meanings and numerical distributions may cause feature mismatches.
To address this, we innovatively introduce angular edge, which uses edge length to implicitly represent angle, achieving a unified representation.

Ⅲ. Why not Equivariant NNs?

Equivariant Networks (EqvNs) are suitable for tasks that depend on specific conformations, such as energy prediction.
Invariant Networks (InvNs) are suitable for tasks that rely solely on the type of molecule rather than specific conformations, such as predicting reaction conditions and yields.
This paper focuses on predicting reaction properties, which are invariant to specific conformations, requiring an InvN.

Results in Tab 2 validate the advantage of InvNs in reaction modeling.
Hence, our RG uses the invariant features.

Tab 2: Comparision of EqvN and InvN.

Ⅳ. Calculation of 3D Coords

The entire RG is NOT directly used as input for calculating 3D coords. Coords of each molecule are calculated individually. Specifically, we input each molecule in reaction into RDKit, and then use the ETKDG + MMFF to calculate 3D atomic coords.

Ⅴ. Further Ablation Analysis

We conduct additional ablations on model components and training strategies.
Our backbone, equipped with CRM-H and trained with a two-stage strategy, achieves a top-1 accuracy of 0.3, surpassing the baseline’s 0.2.
Incorporating RG further boosts the accuracy to 0.33. Details are in Appendix Sec. H.4.3.

Tab 3: Ablation results.

Ⅵ. Using 3D Info in Other Frameworks

The results in Tab 4 show that introducing 3D info into other frameworks leads to accuracy improvements.

Tab 4: Using 3D Info on other frameworks.

Ⅶ. The Necessity of 3D Info

Several methods have explored incorporating 3D info into molecular modeling.
Uni-Mol [5] uses large-scale 3D positions, GraphMVP [6] uses bond lengths, and GEM [7] uses bond angles.
Inspired by these works, we explore 3D in reaction modeling, which improves accuracy (see Tab 5).

Although generating 3D conformations is currently costly, many studies [8–9] are actively addressing this issue.
We are optimistic about the potential of incorporating 3D into the RG framework.

Table 5: The impact of 3D info.

Ⅷ. Discussion of Transition State (TS) Structures

1. TS methods are inspiring to us. However, as discussed in Ⅲ, TS structures are related to specific 3D conformations, whereas reaction properties like conditions are independent of specific conformations. Therefore, their focus in 3D modeling differs.

2. We explore TS generation backbones for predicting reaction properties.
   Our RG achieves better performance (see Tab 6).

3. Methods for calculating TS, such as DFT, are very time-consuming (~10min for 10 atoms[10] and polynomial growth) and unaffordable in dataset like USPTO, which includes 680k reactions with up to 347 atoms. Hence, we didn't use TS structures as inputs for the network.

   Tab 6: Comparison with TS generation backbones.

   Method	T1	T5	T15
   OARD [4]	0.13	0.20	0.31
   EquiReact [11]	0.12	0.24	0.30
   Ours	0.18	0.31	0.37

[1]-[4] correspond to those in the comments.

[5] Uni-Mol: A Universal 3D Molecular Representation Learning Framework

[6] Pre-training Molecular Graph Representation with 3D Geometry

[7] Geometry-enhanced molecular representation learning for property prediction

[8] Predicting molecular conformation via dynamic graph score matching

[9] Torsional diffusion for molecular conformer generation

[10] Fast and Automatic Estimation of Transition State Structures Using Tight Binding Quantum Chemical Calculations

[11] 3DReact: Geometric deep learning for chemical reactions