A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics

Thank you very much for your detailed review! We are delighted that you found that our “experiments demonstrates the effectiveness of the model” and that our “writing is clear”. Please see our answers to your comments below.

Questions:

Q1.

Please refine the notation system in the paper. You may remove some unnecessary notations and descriptions. See my Weakness points 3, 4, and 7.

3. What’s the difference between Eq. (3) and Eq. (4)? For example, in Eq. (3), you state $X_{\gamma(t)} \in \mathcal T_{\gamma(t)}\mathcal T \mathcal Q$. In Eq. (4), however, the set is $\mathcal T_{\gamma(t)}\mathcal Q$. Why do smooth trajectory lie in $\mathcal Q$ rather than $\mathcal T \mathcal Q$, where is $\dot q$?

4. What is $V(q;\theta_V)$ in Line 162? Since there are massive notations, the author is advised to complete the proofreading.

5. Notations in Eq. (8) are unclear to the reader. What are $l, h, x$, and PT? Can the author align them with the previous notations?

Thank you for pointing out to unclear notations. We answer the individual points below.

3. We understand the confusion of using the same letters for both equations and updated them to denote the general configuration manifold as $\mathcal{M}$ in Section 2.2, and refer to its instance in the Lagrangian case as $\mathcal T\mathcal Q$, i.e., $\mathcal{M}=\mathcal{TQ}$. We clarified this point in Section 2.2. Also, we removed a previous equation to save space, so the indexing shifted by one number up.

   Eq. 4 (now 3) refers to the standard problem in MOR, where we aim at reducing a first-order dynamical system that lives on a configuration manifold $\mathcal Q$ (now $\mathcal M$), so that it generates trajectories $\mathbf\gamma(t) \in \mathcal Q$ (now $\mathbf\gamma(t) \in \mathcal M$). Its state evolution is described by a vector field $\mathbf{X}$ that lies on the tangent space, i.e., $\dot{\mathbf{\gamma}}(t) = \mathbf{X} \in \mathcal T_{\mathbf{\gamma}(t)} \mathcal Q$ (or now: $\in \mathcal T_{\mathbf{\gamma}(t)} \mathcal M$)).

   Eq. 3 (now 2) describes the Lagrangian system, which is characterized by second-order dynamics, rewritten in a first-order form. In this case, a trajectory is part of the tangent bundle (being a smooth manifold representing the state-space) $\mathbf{\gamma}(t) = \left[ \mathbf{q}^{\intercal}, \dot{\mathbf{q}}^{\intercal}\right]^{\intercal}\in \mathcal T\mathcal Q$ . Its time derivative (i.e. the vector field) $\dot{\mathbf{\gamma}}(t) = \left[ \dot{\mathbf{q}}^{\intercal}, \ddot{\mathbf{q}}^{\intercal}\right]^{\intercal}\in \mathcal T_{\mathbf{\gamma}(t)}\mathcal T\mathcal Q$ is part of the tangent space of the tangent bundle at the point $\mathbf{\gamma (t)}$.

   Eq. 4 (now 3) is equivalent to Eq. 3 (now 2) by setting the configuration space of the system in equation 4, $\mathcal Q$ (now equation 3 and $\mathcal M$,), to be the tangent bundle from equation 3 (now 2), i.e. $\mathcal T\mathcal Q$.

4. $V(q;\theta_V)$ is the potential energy of a Lagrangian function $\mathcal L$ (see line 103) parametrized as a network $V(q;\theta_V)$.

5. In this section, we denote the loss function generally by $\ell$ and the optimization variable by $\mathbf{x}$. $h$ is a function of the Riemannian gradient, which depends on the selected optimization method. Finally $\mathrm{PT}$ denotes a parallel transport operation, which is described in Appendix A.1. We thank the reviewer for pointing this out as missing and completed Section 3.2 accordingly.

Q2.

The background and design motivation should be emphasized for readers unfamiliar with Lagrangian systems. See my Weakness points 1, 2.

1. In the Introduction, for readers unfamiliar with the high-dimensional Lagrangian system, the author should provide some references and emphasize their importance. The author is also suggested to provide some quantitative descriptions, e.g., the range of the dimensionality.

2. Is preserving the geometry important between FOM and ROM? Can the author summarize what the benefits are of preserving the structure in physical systems and control?

1. Thank you for pointing this out! We emphasized the importance of high-dimensional Lagrangian systems for fluid flows, continuum mechanics, and robotics in Section 1. Moreover, we now explicitly stated that gray-box approaches, such as LNNs and HNNs are mostly limited to systems with 5 or less dimensions.
2. Preserving the geometry between FOM and ROM is crucial for the ROM to preserve the physical properties of the FOM. For instance, structure-preserving ROMs preserve the energy-conservation and stability properties of the corresponding FOM. This may be leveraged, e.g., for energy-based control of FOMs via the ROMs.

1. For my question, "Is preserving the geometry important between FOM and ROM? Can the author summarize what the benefits are of preserving the structure in physical systems and control?"

I am still confused about your answer. Can you provide some examples or references?

2. From Section 2.3 to 2.4, you said "In contrast to MOR, LNN consider low-dimensional systems with unknown dynamics that we aim to learn." From my understanding, MOR also involves dimensionality reduction. E.g., you include AE in your Equation (5). So, it's still unclear about the difference between MOR and LNN.

Many thanks for your insightful feedback! Please see below for our answers.

Weaknesses:

Major:

1. It is unclear where the novelty of the idea lies and what advantage it brings with respect to the previous works. The use of auto-encoder to reduce the dimensionality of the system is not a new idea. In fact, several papers in the same field like Hamiltonian Neural Network, Lagrangian Neural Network also used auto-encoder to learn dynamics of systems from images. If I understand it correctly, the novelty of the idea lies in a neural network learning both the mass matrix and potential for the Lagrange term. However, it is unclear what benefits does this new formulation of Lagrange term has in comparison with, for example, DeLan and LNN.

   Please refer to our response to all reviewers for a discussion: Point (1) clarifies our contributions, see in particular 1d.

   We would like to additionally emphasize that our approach features two main differences compared to LNN and DeLaN:

   • As in our approach, DeLaN [1] uses two networks to parametrize the mass-inertia matrix and the potential energy. However, DeLaN relies on a Cholesky decomposition network $\mathbf{L}(\mathbf{q};\theta)$ to parametrize the mass-inertia matrix, so that $\mathbf{M}=\mathbf{L}\mathbf{L}^T$. Instead, we directly parametrize the mass-inertia matrix with a SPD network $\mathbf{M}(\mathbf{q};\theta)$, which accounts for the intrinsic geometry of mass-inertia matrices. This leads to improved performances compared to DeLaN, as shown in Section 5.1.
   • DeLaN and LNN are limited to predict the dynamics of low-dimensional physical spaces (i.e., 2-5 dimensions) and do not scale to high dimensions. We propose an approach that allows us to learn the dynamics of high-dimensional physical systems by learning a surrogate structure-preserving low-dimensional model of the high-dimensional dynamics. Our approach differs from previous works that used LNNs and HNNs in combination with AEs to learn dynamics from images in that it does not consider high-dimensional observations (images) of low-dimensional physical systems but systems with high-dimensional state spaces, a.k.a high-dimensional dynamics.

2. Reducing the dynamics to lower order dimension does not mean lower computational demand. It is possible, for example, that the lower dimensional dynamics has stronger stiffness than the original dimension. The paper does not present any results comparing the wall-clock time of solving ODE in the original dimension vs solving learned ODE in the lower dimension using the same hardware (e.g., CPU and GPU) to be convincing that their method really achieve computational reduction.

   We thank the reviewer for pointing this out! As discussed in to our response to all reviewers (point 1), our method tackles scenarios where the high-dimensional dynamic parameters are unknown. Therefore, the ODE cannot be solved in the original dimension and we cannot provide solution times for both the FOM and the ROM.

3. Lack of comparisons with other architectures. It seems like the authors only comparing the experiment with LNN. There are a lot of other works in this area that the authors can compare their work with. For example, Hamiltonian Neural Networks (https://arxiv.org/abs/1906.01563), dissipative HNN (https://arxiv.org/abs/2201.10085), dissipative SymOden (https://arxiv.org/abs/2002.08860), Constant of motion network (https://arxiv.org/abs/2208.10387), etc.

   Thank you for these suggestions! Please refer to our response to all reviewers, points 2 and 3 for a discussion. In particular, we provide an additional comparison with the approach presented in [2], which is the closest to our work, in Section 5.2. Moreover, we would like to highlight that HNNs [3] and extensions [4,5] are, as LNNs, limited to learning low-dimensional dynamics. Preliminary experiments with HNNs with the high-dimensional systems of Section 5.2 led to similar results as LNNs.

Minor comments:

1. Please avoid using the unscientific notation like $1e^{-5}$. Instead, please use notations like $1\times 10^{-5}$.

   We updated the notations according to the comment of the reviewer.

Questions:

1. The idea of using Lagrangian principle is usually made to conserve the energy. However, some of the case (e.g., the cloth) does not seem to conserve the energy. How can the authors justify the use of Lagrangian principle in such cases?

   As you pointed out, unforced Lagrangian systems (i.e., with generalized forces $\mathbf \tau = \mathbf 0$) preserve the energy. This corresponds, e.g., to the double pendulum scenario in Section 5.1. In the cloth-experiment, we model an object with mass in contact with the sphere, and obtain the contact forces from the simulator as input to the model in the form of generalized forces $\mathbf{\tau}\neq 0$. Therefore, in this actuated scenario, the model is not expected to conserve the energy throughout the whole trajectory. However, we expect the energy to be conserved during the first part of the trajectory which corresponds to the free fall. As shown in Fig. 16, the energy level indeed seems constant during the first ~0.1 second. Notice that the generalized forces are also handled by the structure-preserving reduction, see Section 4.1.

2. The error in Table 1 seems to be very high (it reaches $1\times 10^8$). Why can this happen? Normally, a well-setup neural network train should be normalized so the error starts around 1. The error reaches $10^8$ seems to indicate that the error is not normalized. Does this mean the training is not normalized?

   Thank you for pointing this out. During training, data was appropriately scaled, as mentioned by the reviewer. Table 1 provided unnormalized results on the testing data and reported the mean of the squared distances between predictions and ground truth values over the whole testing dataset of N samples, i.e. $\frac{1}{n}\sum_{i=1}^N||\tilde{\ddot{\mathbf q}}^{\text{p,i}} - \ddot{\mathbf q}^i||^2$. For large absolute acceleration values, high-dimensional (16 dimensions) vectors $\ddot{\mathbf q}^i$, and an inaccurate model, we indeed observed high absolute values, which were hard to interpret.

   We agree that this format was not easily readable and hard to interpret. Therefore, we updated Table 1 to report relative errors in acceleration $||\tilde{\ddot{\mathbf q}}^{\text{p,i}} - \ddot{\mathbf q}^i|| / ||\ddot{\mathbf q}^i ||$, velocity $||\tilde{\dot{\mathbf q}}^{\text{p,i}} - \dot{\mathbf q}^i|| / ||\dot{\mathbf q}^i||$, and position $||\tilde{{\mathbf q}}^{\text{p,i}} - {\mathbf q}^i|| / ||{\mathbf q}^i||$. We also extended Table 1 to incorporate the additional baseline. Notice that the relative error for the full-order LNN is still significantly >1, i.e. we infer no significant training progress.

3. (Related to weakness #2) Computational complexity in solving the ODE of a system does not only depend on how many dimension it is, but also how stiff the ODE is. Is there any effort in reducing the stiffness of the equation to ensure lower computational demand?

   Thank you for this interesting suggestion! We would like to emphasize that our primary goal is not to accelerate the ODE computation, but to learn unknown dynamic parameters. Although we could consider incorporating additional bias to accelerate the computation of the low-dimensional ODE, we see this as being out of the scope of the contributions of this paper and will consider it for future research.

4. How is the roll-out of the NN done? What kind of ODE solver is used? I found in the appendix that the simulation is done using RK4. Is the same method used to roll-out the NN for the ODE loss?

   The roll-out (both in training and testing) uses a simple Euler forward integration of variable number of steps. This is a common integration method used for training of other structure-preserving gray-box methods, as described by [7]. We use the same integration method in our multi-step integration loss in Section 4.3. We added the information concerning the roll-out in Appendix F.

5. Why the number of rolled out time steps $H=8$ quite small in computing the ODE loss? What is preventing it from using higher number of time steps?

   Theoretically, any number $H$ could be chosen. However, the training complexity increases with the number of steps, due to the increased complexity of backpropagating through a multi-step loss with a longer horizon. We empirically found that $H=8$ is a good trade-off and leads to good results in our experiments.

6. How was the time step $\Delta t = 10^{-3}$ chosen? How would choosing larger time step affect the roll-out prediction error?*

   We used commonly-used values for simulating physical systems via differentiable ODE solvers, as mentioned in [6] and increased the sampling rate when working with very high-dimensional deformable structures, e.g., $\Delta t = 10^{-4}$ for the cloth experiment in Section 5.2.3, as is generally recommended, e.g. by the simulation software [7].

References

[1] Michael Lutter and Jan Peters. Combining physics and deep learning to learn continuous-timedynamics models. Intl. Journal of Robotics Research, 42(3):83–107, 2023.

[2] Harsh Sharma and Boris Kramer. Preserving Lagrangian structure in data-driven reduced-order modeling of large-scale dynamical systems. Physica D: Nonlinear Phenomena, 462:134128, 2024.

[3] Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Neural Information Processing Systems (NeurIPS), volume 32, 2019.

[4] Zhong, Y. D., Dey, B. & Chakraborty, A. Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning. In ICLR Workshop on Integration of Deep Neural Models and Differential Equations (DeepDiffEq), 2020.

[5] Andrew Sosanya and Sam Greydanus. Dissipative Hamiltonian Neural Networks: Learning Dissipative and Conservative Dynamics Separately. arXiv, 2022.

[6] Thai Duong, Nikolay Atanasov: Hamiltonian-based Neural ODE Networks on the SE(3) Manifold For Dynamics Learning and Control. Robotics: Science and Systems (RSS), 2020.

[7] Todorov, Emanuel and Erez, Tom and Tassa, Yuval: MuJoCo: A physics engine for model-based control. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012.

Thank you very much for your valuable feedback. Please find our response to your comment and questions below.

Contribution and originality:

• I question the novelty of this paper. The proposed approach closely resembles existing methods described in [1, 2]. Additionally, the authors have not cited reference [1], which appears to be highly relevant to their work.

• The contribution of this paper is overstated, and the true contribution appears to be quite incremental. Based on my understanding, the theoretical foundation relies heavily on the published work in [2], with many of the equations directly derived from [2]. There seem to be no novel contributions to the methods or framework, as the use of an autoencoder for reduced-order modeling was already proposed in [2]. This paper primarily appears to be an implementation of the original idea from [2].

• For the learning framework, this paper replaces the Galerkin method (subspace method) with an autoencoder structure for learning Lagrangian dynamics. However, the autoencoder representation is fundamentally similar to the Galerkin (subspace) method in its underlying principles. The primary distinction from reference [1] lies in the representation of the mass-inertia matrix as a symmetric positive definite (SPD) manifold within the configuration space, whereas the loss functions and implementation details remain largely consistent with those in [1]. The authors should clearly elucidate the key differences from prior work and provide a justification for why reference [1] was not cited.

Please refer to our response to all reviewers for the discussion and clarification of our contributions:

• Point 1 clarifies our contributions, see in particular 1a, 1b, and 1c for the similarities and differences with MOR methods, with [2], and with [1], respectively.
• Point 2 analyses the similarities and differences of our approach in comparison to [1].

In particular, we would like to emphasize again that:

1. Our work differs from the traditional MOR setting and from [2] in that our model does not have access to the full-order model dynamic parameters, while MOR methods including [2] are intrusive and assume to have access to the full-order dynamics, i.e, the governing equations and the dynamic parameters.
2. Our work differs from L-OpInf [1] in that it can deal with systems that are not linearly reducible and it does not assume a fixed constant reduced-order mass-inertia matrix. Therefore, it is more general and allows for more expressive models as shown by our additional comparison in Section 5.2.

We would also like to thank you for pointing us to [1] and to highlight that we now corrected this unfortunate omission by adding both a theoretical (Section 1) and experimental (Section 5.2) comparisons with L-OpInf in the revised version of the paper.

Comparison:

There is no comparison with other existing algorithms [1, 3]. The authors claim scability of its algorithm, but the current work only tests up to 600 DoFs using deep learning. In contrast, the existing work in [1] conducted experiments with 259, 744 DoFs, even using Matlab. This raises questions about the claimed scability and performance improvements.

Please refer to our response to all reviewers (points 2 and 3) for the description and analysis of the additional experimental comparisons with L-OpInf.

We would like to highlight that L-OpInf [1] already fails in the 16-dimensional example of Section 5.2 due to the fact that the high-dimensional state space of the coupled pendulum is not linearly reducible. Moreover, we were not able to generate stable results with L-OpInf for the rope and cloth datasets: The CVX solver returns solution status as ‘Infeasible’, and fails to provide a reasonable solution. Despite its reduced computational complexity and its impressive performance on linearly-reducible systems reported in [1], L-OpInf is not able to learn the dynamics of high-dimensional systems which are not linearly reducible.

Please let the authors know if their rebuttal has addressed any of your concerns. Thank you.

Response to the novelty

Thank you for your detailed response. I have raised my score to 5. However, I still have some concerns, which are outlined below.

It can deal with systems that are not linearly reducible and it does not assume a fixed constant reduced-order mass-inertia matrix.

1. The difference between [1] and your work seems primarily related to the mass-inertia matrix. From my perspective, the main change is replacing linear projection with an auto-encoder structure, which, theoretically, is not a significant departure. While the proposed method is presented as more general, this particular idea is not entirely novel. The real contribution appears to be in changing the constant mass-inertia matrix to a structure defined on an SDP manifold. I have compared it in detail with [2], and the formulation differences are minimal. I'm not sure that this contribution is sufficient for publication at a top-tier conference like ICLR.

Moreover, the optimization on an SDP manifold relates to Hilbert's 5th problem, making it particularly challenging to solve and optimize for high-dimensional matrices [3]. This raises concerns about the scalability of the proposed approach.

2. The authors mention that the mass-inertia matrix is linearly irreducible. Could this issue be resolved by using a functional representation via the Galerkin method? If so, the claimed contribution of the proposed approach may be diminished.

[1]. Sharma, Harsh, and Boris Kramer. "Preserving Lagrangian structure in data-driven reduced-order modeling of large-scale dynamical systems." Physica D: Nonlinear Phenomena 462 (2024): 134128.

[2] Buchfink, Patrick, et al. "Model reduction on manifolds: A differential geometric framework." Physica D: Nonlinear Phenomena 468 (2024): 134299.

[3] https://terrytao.wordpress.com/wp-content/uploads/2014/11/gsm-153.pdf

We are grateful for your detailed and helpful review. We appreciate that you recognized our “great effort into explaining their architectures and learning algorithms”. We address your feedback below.

Weaknesses

1. It seems like the overall idea of learning ROM using constrained AE that preserves the Lagrangian structure is already presented in [Buchfink et al.] cited in the paper. However, this is not clearly pointed out in the paper, and the authors' claims in the introduction and their explanation of the methodology in Section 3&4 somehow mislead the readers to think that the idea is also the main contribution of this paper. This reviewer thinks this paper's contribution on top of the prior work is proposing a new SPD network architecture for parameterizing the mass-inertia matrix and using Riemannian optimization in training to consider the parameters' geometry.

   This identification of the contribution should be explained clearly, and the performance of RO-LNN should be compared to the prior works in the experiments.

   Please refer to our response to all reviewers for a discussion:

   • Point (1) clarifies our contributions, see in particular 1a-1b and 1e.
   • Point (3) discusses the comparison to prior work.

2. There are no experimental results that show the benefit of preserving the Lagrangian structure in the latent space. Adding a baseline that learns the model without LNN would help to understand this benefit.

   Thank you for your comment! The main benefits of Lagrangian structure-preserving is that they enable the preservation of the physical properties of the FOM in the ROM. For instance, structure-preserving ROMs preserve the energy-conservation and stability properties of the corresponding FOM. These properties further enable the application of downstream methods such as energy-based controllers in the ROM. These benefits have been thoroughly discussed theoretically and demonstrated experimentally in the MOR literature [1, 8, 9]. Notably, they lead to more accurate and numerically-stable results than MOR methods that do not preserve structure. The linear model from [1], which relates closely to our approach, also reported stability increases compared to non-structure-preserving methods. We added L-OpInf [1] as a baseline in Section 5.2 and show that it outperformed by our model.

   Moreover, it is worth highlighting that considering a Lagrangian (or Hamiltonian) structure is highly beneficial when learning low-dimensional dynamics compared to considering black-box methods [2, 10].

   All in all, our model builds on existing works that have already demonstrated the benefits of physically-consistent models, i.e., (1) of considering a Lagrangian structure when learning dynamics, and (2) of preserving the Lagrangian structure in MOR.

7. For the cloth experiments, it seems a much shorter timestep size is used for full-horizon prediction (25 steps for 0.025s vs 2500 steps for 0.25s). Then, fig. 6 is not a fair comparison, and this should be pointed out clearly.

   This comment arose from a typo, which we now corrected. Many thanks for the alertness!

   All experiments with our cloth model were carried out with $\Delta t = 1\times 10^{-4}$ s. The full-horizon experiment predicts cloth configurations for 2500 steps (i.e., 0.25 seconds) from an initial condition, without any updates. In the 25-steps cloth experiment, the model received ground-truth updates every 25 timesteps. This indeed corresponds to predictions over only 0.0025 seconds away from an initial condition. We corrected this in the paper.

Questions:

1. Additionally, how could we connect the satisfaction of the Euler-Lagrange equations of the latent dynamics to that of the original dynamics?

   Following Buchfink et al. [8], and our section 4.1., the pullback of the Lagrangian functional $\mathcal L$ via the lifted embedding $\varphi$ guarantees (under the specific projection properties of the embedding) the preservation of the Lagrangian structure on the embedded submanifold (in this case: the tangent bundle $\mathcal T \check{\mathcal Q}$). Hereby obtaining a reduced Lagrangian $\check{\mathcal L}$, the reduced Euler-Lagrange equations can still be obtained in the standard manner from the principle of least action (see Eq. 10). The reduced solution to the equations of motion is then related to the full-order solution via the (lifted) embedding, which has been obtained via the constrained AE.

References

[1] Harsh Sharma, Hongliang Mu, Patrick Buchfink, Rudy Geelen, Silke Glas, and Boris Kramer. Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds. ComputerMethods in Applied Mechanics and Engineering, 417:116402, 2023.

[2] Michael Lutter and Jan Peters. Combining physics and deep learning to learn continuous-timedynamics models. Intl. Journal of Robotics Research, 42(3):83–107, 2023.

[3] Aasa Feragen and Andrea Fuster. Modeling, Analysis, and Visualization of Anisotropy. In Geometries and Interpolations for Symmetric Positive Definite Matrices 85–113 (2017).

[4] Z. Lin. Riemannian Geometry of Symmetric Positive Definite Matrices via Cholesky Decomposition. SIAM Journal on Matrix Analysis and Applications 40, 1353–1370 (2019).

[5] Federico Lopez, Beatrice Pozzetti, Steve Trettel, Michael Strube, and Anna Wienhard.Vector-valued distance and gyrocalculus on the space of symmetric positive definite matri-ces. In Neural Information Processing Systems (NeurIPS), volume 34, pp. 18350–18366,2021.

[6] Xuan Son Nguyen. The gyro-structure of some matrix manifolds. In Neural Informa-tion Processing Systems (NeurIPS), volume 35, pp. 26618–26630. Curran Associates, Inc.,2022.

[7] Xuan Son Nguyen, Shuo Yang, and Aymeric Histace. Matrix manifold neural networks++. In The Twelfth International Conference on Learning Representations, 2024.

[8] Patrick Buchfink, Silke Glas, Bernard Haasdonk, and Benjamin Unger. Model reduction on manifolds: A differential geometric framework. Physica D: Nonlinear Phenomena 468 (2024).

[9] Kevin Carlberg, Ray Tuminaro, and Paul Boggs. Preserving Lagrangian structure in nonlinear model reduction with application to structural dynamics. SIAM Journal on Scientific Computing, 37(2):B153–B184, 2015

[10] Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Neural Information Processing Systems (NeurIPS), volume 32, 2019.

Summary of main changes in the paper

• We refined the introduction to better highlight the contributions of our paper, as well as its novelty with respect to previous works.
• We explicitly cited [4] and explained its differences with respect to our approach in the introduction.
• We reformulated parts of Section 2.2, as well as Section 4.1 to highlight our contributions and emphasize the differences between our approach and traditional MOR methods, in particular [1].
• We added an additional comparison with L-OpInf [4] in Section 5.2, highlighting the limitations of this approach to identify high-dimensional dynamics which are not linearly reducible.
• Further minor additions corresponding to an individual reviewer’s feedback are addressed in the individual responses.

References

[1] Patrick Buchfink, Silke Glas, Bernard Haasdonk, and Benjamin Unger. Model reduction on manifolds: A differential geometric framework. Physica D: Nonlinear Phenomena 468 (2024).

[2] Harsh Sharma, Hongliang Mu, Patrick Buchfink, Rudy Geelen, Silke Glas, and Boris Kramer. Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds. ComputerMethods in Applied Mechanics and Engineering, 417:116402, 2023.

[3] Joshua Barnett and Charbel Farhat. Quadratic approximation manifold for mitigating the Kolmogorov barrier in nonlinear projection-based model order reduction. Journal of ComputationalPhysics, 464, 2022.

[4] Harsh Sharma and Boris Kramer. Preserving Lagrangian structure in data-driven reduced-order modeling of large-scale dynamical systems. Physica D: Nonlinear Phenomena, 462:134128, 2024.

[5] Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. In Neural Information Processing Systems (NeurIPS), volume 32, 2019.

[6] Yaofeng Desmond Zhong and Naomi Leonard. Unsupervised learning of Lagrangian dynamics from images for prediction and control. In Neural Information Processing Systems (NeurIPS), vol-ume 33, pp. 10741–10752, 2020.

[7] Lee, K. & Carlberg, K. T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys. 404, 108973 (2020).

Discussion and comparison with Sharma & Kramer [4]

In contrast to the aforementioned MOR methods, Sharma & Kramer proposed a non-intrusive approach called Lagrangian operator inference (L-OpInf), which identifies reduced-order dynamic parameters in a linear subspace of the high-dimensional state space. L-OpInf differs from our approach as follows:

1. L-OpInf projects the observed high-dimensional states in a linear subspace obtained via the singular value decomposition of the data matrix. However, it is well known that such linear subspaces with good approximation properties cannot be guaranteed, see [1,7]. Notice that this limitation is also acknowledged in the conclusion of [4] and that the impressively high-dimensional example considered in [4] was known to be linearly reducible. In contrast, we pursue a more general approach and learn a non-linear embedding of the FOM obtained via a constrained auto-encoder.

2. L-OpInf assumes a specific form of reduced-order dynamics where the reduced mass-inertia is fixed as equal to identity, i.e., $\check{\mathbf{M}}=\mathbf{I}$, thus restricting heavily the solution space. As discussed in Section 3.3.3 of [4], this is required to overcome numerical issues arising when identifying the dynamic parameters when $\check{\mathbf{M}}\neq\mathbf{I}$. We argue that considering a constant reduced mass-inertia metric significantly limits the expressivity of the learned model, and that it limits its applicability to a certain range of systems. In contrast, our approach considers a reduced-order mass-inertia matrix as a general function of the reduced-order position, i.e., $\check{\mathbf{M}}(\check{\mathbf{q}})$.

3. L-OpInf assumes a specific form of quadratic potential energy $V(\mathbf{q}) = \frac{1}{2}\mathbf{q}\mathbf{K}\mathbf{q}$. Our approach, as DeLaN and LNN, assumes a general form of potential energy specified via a general function $V(\mathbf{q})$ and is therefore more expressive.

4. L-OpInf identifies the linear subspace and the reduced-order dynamic parameters sequentially. Instead, our approach jointly identifies a non-linear reduced representation and the associated dynamic parameters. As shown in Section 5.2.1 and Fig. 4, jointly-trained models outperform the sequentially-trained ones.

All in all, our approach is more general, flexible, and expressive than L-OpInf and allows us to learn the high-dimensional dynamics of systems that require non-linear reductions to feature good approximation properties.

Additional experimental comparison to prior work

1. Comparisons with L-OpInf [4]: We implemented L-OpInf and considered it as an additional baseline in Section 5.2. The results on the 16-DoF coupled pendulum are reported in Table 1. We observe that L-OpInf shows significantly higher prediction errors than our propose RO-LNN. Specifically, it results in a position prediction error of 2 magnitude higher than the RO-LNN trained on acceleration, and 4 orders of magnitude higher than the RO-LNN trained with multi-step integration. Notice that we only provide results with L-OpInf for the 16-DoF coupled pendulum as, similarly to the full-order LNN, we were not able to generate stable results with the rope and cloth datasets. This illustrates the limitations of considering a linear subspace and a constant inertia matrix in the latent space, which hinder the expressivity of L-OpInf compared to our approach.

2. Comparisons with other MOR approaches: In our scenario, we assume that we do not have access to the high-dimensional dynamic parameters. Therefore, we cannot not compare our approach with other MOR methods such as [1,2,3], as they are invasive and assume that the high-dimensional FOM dynamics are known.

3. Comparisons with HNN [5] and extensions: We would like to highlight that HNNs [5] and extensions thereof are, as LNNs, limited to learning low-dimensional dynamics. We believe that additional comparisons with Hamiltonian systems go beyond the scope of this work and prefer to focus on Lagrangian models such as [4]. It is worth highlighting that preliminary experiments with HNNs on the high-dimensional systems of Section 5.2 led to similar results as LNNs.

Additional video

We added an additional video that illustrates the results of our experiments as supplementary material.