The dark side of the forces: assessing non-conservative force models for atomistic machine learning

We thank the reviewer for the suggestion to investigate a solvated peptide, and have done so: we studied alanine dipeptide solvated in water, and additionally a benzene molecule adsorbed on a graphene surface, beyond the homogeneous cases of graphene, amorphous carbon and aluminium that we had already discussed in Appendix F. In all cases, the observation that non-conservative models like Equiformer and ORB display lack of energy conservation in NVE MD holds. The effect is comparable to or larger than that seen for homogeneous systems, and much larger than the drift observed for the more accurate single-purpose PET-NC model we trained for bulk water (https://imgur.com/a/CV3YmI3).

As we argue with Reviewer NcLF, our choice of a comparatively simple system, for which very accurate models can be trained, is aimed at giving non-conservative predictions the most favorable treatment possible, and these additional tests confirm that the artifacts become even more severe when using general-purpose, universal models that are less accurate and entail more pronounced violation of energy conservation.

We also thank the Reviewer for pointing out some typographical errors, which we have fixed.

We thank the Reviewer for their comments and questions. To provide more context, the use of multiple time-stepping to reduce the cost of a simulation while matching the accuracy of a “slow” potential is well-established in the molecular dynamics community. It was originally introduced to reduce the cost of long-range electrostatics (which is more computationally demanding, and harder to parallelize, than short-range components in a physically-motivated model) [Tuckerman 1992], and it has been used extensively together with different levels of electronic-structure theory [Marsalek 2016] and with machine-learning potentials [Rossi 2020]. It has also been recently applied to a very similar problem - namely correcting for symmetry breaking in unconstrained machine-learning models [Langer 2024]. In all these cases, and in that of non-conservative forces, MTS acts as a mitigation strategy, with smaller initial errors allowing for a larger stride M.

Normalizing the cost of non-conservative MD to 1, and letting $F$ denote the slowdown for conservative force predictions, $2 \leq F \leq 3$, MTS reduces the theoretical cost of well-behaved molecular dynamics from $F$ to $1+(F-1)/M$. The evaluation of the conservative force is reduced by a factor $1/M$ and there is one common evaluation every $M$ steps. Note that in our current practical implementation, we do not yet re-use data for the common evaluation, and therefore incur a cost of $1+F/M$. We find that for the PET model employed in this work, $F\approx2$.

In all our MTS experiments, the empirical timings match the theoretical ratios rather closely:

For $M=8$ and $F\approx2$, the simulation is only $\approx20$% slower than using exclusively non-conservative forces. Note that, even in the most unfavorable case (where only $M=2$ is stable), the theoretical computational overhead due to the use of conservative forces could be halved with an efficient implementation of MTS.

Even though these general arguments suggest that MTS should be broadly applicable, we do agree with the Reviewer that further evidence that “hybrid” models are useful in general would be very valuable. To this end, we also consider the case of the diverse OC20 dataset (containing molecular fragments adsorbed on different catalytic surfaces) [Chanussot 2020].

First, we performed a “conservative fine-tuning” exercise - reducing the training time by a factor of more than 3 while achieving the same final accuracy as the conservative model. This shows that the idea of pre-training with direct forces and then turning on backpropagation to fine-tune, therefore obtaining a hybrid model inexpensively, is far from specific to the water system. (Incidentally, this training method that we propose has already been picked up by the community to train state-of-the-art models, for example in [Fu 2025].)

Subsequently, we use this model to perform MD on a representative structure (an oxygen atom adsorbed on an alloy surface). Even though the direct-force model shows a much worse temperature drift than observed in the experiments on water (consistent with the lower accuracy of the fit for this more broadly applicable potential) MTS still succeeds in stabilizing NVE trajectories up to M=8 (https://imgur.com/a/PY96n58). This provides a concrete example that MTS also works for less accurate, general-purpose direct-force models.

To summarize, our work not only highlights the potential problems related to using non-conservative forces, but it also provides simple and effective solutions in the form of conservative fine-tuning (at training time) and MTS (at inference time), which allow practitioners to retain nearly all the advantages of using direct forces while avoiding their physical pitfalls.

[Tuckerman 1992] https://doi.org/10.1063/1.463137

[Marsalek 2016] https://doi.org/10.1063/1.4941093

[Rossi 2020] https://doi.org/10.1021/acs.jctc.0c00362

[Langer 2024] https://doi.org/10.1088/2632-2153/ad86a0

[Chanussot 2020] https://doi.org/10.1021/acscatal.0c04525

[Fu 2025] https://arxiv.org/abs/2502.12147

We thank the reviewer for pointing out additional relevant literature, which we have added to the manuscript. We also added the recent related work by Eissler et al. (arXiv:2503.01431), in which a non-conservative and non-equivariant transformer model is trained on a large dataset and used to study the degree of energy non-conservation in MD simulations.

We chose water as a benchmark because it is an important system, ubiquitous in chemistry and biophysics, and because it is challenging due to the combination of high-frequency vibrations of the covalent O-H bond and the long-term dynamics of the hydrogen-bond network. Its relative compositional simplicity and homogeneity also means we can train a very accurate direct-force model, which allows us to show that non-conservative forces can produce unacceptable sampling artifacts even in this most favorable scenario.

However, we agree that additional testing on more diverse molecular and materials systems is desirable. We had already included three examples (graphene, amorphous carbon, aluminum) in the appendices. Following the recommendation of this Reviewer (as well as Reviewer J2Ug), we performed additional tests on two heterogeneous systems: a benzene molecule adsorbed on a graphene surface, and alanine dipeptide solvated in water. We find that the non-conservative universal ORB and Equiformer models display severe lack of energy conservation in these cases, comparable to or worse than in the other systems we considered (https://imgur.com/a/CV3YmI3).

Regarding the use of additional thermostats, we deliberately did not test Nosé-Hoover thermostats, which are non-ergodic in their original form and which, when using the “chain” extension [Martyna 1992] which improves (but does not fully fix [Patra 2014]) ergodicity by producing chaotic thermostat dynamics, amount essentially to a complicated strategy to generate poor-quality random numbers. We cannot think of a single case (besides perhaps the theoretical study of classical chaotic dynamics) in which N-H thermostats cannot be replaced by a stochastic thermostat. We agree with the Reviewer that it would be ideal to explore an “adaptive” thermostat to better substantiate our claim that thermostatting is not a valid fix for non-conservative dynamics. To that effect, we ran some tests using the “GLE-RESPA” [Morrone 2011] thermostats that use a bespoke generalized Langevin equation, tuned to stabilize noisy dynamics without disrupting the sampling of diffusive dynamics. Being local and adaptive, one could expect this approach to be preferable to a global (SVR) or plain Langevin thermostat for stabilizing direct force predictions. Our tests show that instead this GLE thermostat (which is less aggressive for slow degrees of freedom and hence does not slow down diffusion) is less effective at controlling the temperature drift induced by direct-prediction forces. The table below shows the atom-type resolved temperature for a simulation of bulk water at a target temperature of 300K, similar to table 2 in the manuscript. Results closer to 300K are better.

This is consistent with our observation that non-conservative effects are more pronounced for long-range correlations and slow dynamics and reinforces our claim that thermostats can mitigate, but not solve, the artifacts induced by non-conservative forces. Finally, we would argue that the MTS strategy we propose provides a much better and more broadly applicable fix to the shortcomings of non-conservative forces compared to thermostats.

[Martyna 1992] https://doi.org/10.1063/1.463940

[Patra 2014] https://doi.org/10.1103/PhysRevE.90.043304

[Morrone 2011] https://doi.org/10.1063/1.3518369