Equivariant Graph Network Approximations of High-Degree Polynomials for Force Field Prediction

Dear reviewer 8CoZ, thank you for your valuable comments and positive review! Below are our responses to your concerns and questions.

W1. Thanks for your comments. We will further consider this experiment in future work.

W2. Thanks for your suggestion. During the discussion period, we conducted experiments on AcAc and added them to Appendix D1 in the revised paper.

W3. Thank you for your advice. Since we performed more experiments this week, we modified Section 4.3 to "More Empirical Analyses", and now "Algorithm Efficiency" is under it. The text and table of "Algorithm Efficiency" are moved to Appendix D4 in the revised paper.

We appreciate it very much for your time and efforts to evaluate our paper! Hope our responses have properly addressed your concerns.

Best regards,

Authors

Dear reviewer 9963, thank you for your valuable comments and positive review! Below are our responses to your concerns.

W1. Thanks for your opinion. We have modified the corresponding part in the Section 2.3 denoted in red.

W2. We have added the figure of tensor contraction above Equation 11. Thanks to your advice, here we made a 3-body interaction example to save space.

Please see the above modifications in the revised paper. We appreciate it very much for your time and efforts to evaluate our paper! Hope our responses have properly addressed your concerns.

Best regards,

Authors

Dear reviewer LgUR, thank you for your valuable comments! Below are our responses to your concerns and questions.

W1. During the discussion period, we performed MD simulations using our PACE model on three molecules, Aspirin, Ethanol, and 3BPA. To compare the MD trajectories generated by PACE with those generated by our baseline MACE, we compute Wright's Factor (WF) [1] as a metric and present the comparison in the table below. Here, WF quantitatively measures the difference between the RDFs of the ground truth and the generated MD trajectories. For each method, we take the average of 5 MD simulations for 1000 timesteps with 5 different initial structures.

In addition, we also visualize the radial distribution functions (RDFs) of the ground truth MD trajectories and PACE-generated MD trajectories, which can be found in Figure 5 in the revised paper. These results of the MD simulations further affirm that our PACE model not only achieves state-of-the-art (SOTA) performance in force field prediction but also holds practical value in realistic simulations.

We have added MD experiments to Appendix D2 in the revised paper. We agree that low energy and force errors do not guarantee better MD simulations. However, there’s still a lack of standards to evaluate the complex MD simulations. Currently, energy and force errors are still the most common metrics to assess the ability of machine learning models to predict force fields [2]. We will consider developing machine learning models more specifically for realistic MD simulations in future works.

W2. We wish to clarify that our proposed theoretical statement aims to provide a tool to analyze the equivariant networks from the perspective of approximating equivariant polynomial functions. There are three points we wish to emphasize.

1. We wish to clarify that MLP is not an equivariant architecture. That’s to say, a simple MLP doesn’t satisfy the permutation, rotation and translation invariance or equivariance. Meanwhile, current 3D GNN architectures have shown these intrinsic symmetries are important when designing powerful practical model architectures. Therefore, the theoretical tool to analyze these invariant and equivariant architectures is important in our opinion.

2. Second, we wish to clarify that designing equivariant layer with good ability to approximate the equviariant polynomial function is not an easy task. Although the reviewer mentions that “no matter whether the polynomial many-body interaction module can approximate polynomial equivariant functions or not, the entire neural network is a universal approximator for any equivariant functions”, we hold a different opinion. Without the theoretical guidance for constructing the polynomial many-body interaction module, designing an equivariant layer with guaranteed and high capability to approximate the equivariant polynomial functions is challenging.

3. Finally, we wish to clarify that our proposed polynomial many-body interaction module in PACE can approximate any equivariant polynomial function with the highest degree $D=v=3$ with a single layer. Although the PACE layer needs more computational cost about 123% in training time and 127% in memory shown in Table $9$, we believe it is reasonable cost with carefully designed module under the guidance of obtaining the approximation capacity. Meanwhile, the final experimental performance is better.

Q1. We performed an experiment on Ethanol to show how degree $v$ affects the error and cost. In this experiment, hyperparameters are the same. Results in the table below show that higher degree archives lower prediction errors while consuming longer training time and more memory. Due to the tradeoff between error and cost, $v=3$ is used in our PACE across all datasets. We have added this experiment to Appendix D3 in the revised paper.

[1]. Grimley, David I., Adrian C. Wright, and Roger N. Sinclair. "Neutron scattering from vitreous silica IV. Time-of-flight diffraction." Journal of Non-Crystalline Solids 119.1 (1990): 49-64.

[2]. Hollingsworth, Scott A., and Ron O. Dror. "Molecular dynamics simulation for all." Neuron 99.6 (2018): 1129-1143.

We appreciate it very much for your time and efforts to evaluate our paper! Hope our responses have properly addressed your concerns. Best regards,

Authors

Dear Reviewer LgUR,

The discussion deadline is approaching in a few hours. We have submitted our detailed responses to your comments and would greatly appreciate it if you could review our response and revised paper at your earliest convenience. Your feedback is invaluable to us, and we are eager to address any further comments or concerns you may have before the discussion period ends. Thank you very much for your time and consideration!

Best regards,

Authors

Dear reviewer Nhdf, thank you for your valuable comments and positive review! Below are our responses to your concerns and questions.

W1. Thank you for suggesting the direct ablation experiment. During the discussion period, we conducted this ablation experiment on three molecules from rMD17, and the comparison is shown below. In this experiment, we removed additional self-interaction operations and used the same hyperparameters. The results confirm the effectiveness of our polynomial many-body interaction module. We have added this ablation experiment to Appendix D3 in the revised paper.

Q1. Thanks for pointing it out. In the revised paper, we have modified this sentence to the following expression. “While TFN has the ability to approximate all equivariant polynomial functions with an infinite number of layers, the approximation capacity of a single TFN layer is limited, raising the need to develop modules improving the approximation capacity.”

Q2. Thanks for this question. We wish to clarify that these architectures can achieve universal approximation to all equivariant polynomial functions up to the highest degree $D$, denoted as $\mathcal{P}^{D}$. As shown in Table $1$, a single PACE layer can achieve $D=v=3$ in the experiment setup involving two layers, whereas a single NequIP can only reach $D=1$, and Allegro, with its three layers, attains $D=3$.

We appreciate it very much for your time and efforts to evaluate our paper! Hope our responses have properly addressed your concerns.

Best regards,

Authors

I am satisfied with the response to W1 and Q1&2. As for W1, I recommend the authors to place the results of MACE in the table, and list results of more molecules in rMD17 to verify the effectiveness of the proposed self-interaction module.

Accordingly, I will keep my rating. Thank you!