EGODE: An Event-attended Graph ODE Framework for Modeling Rigid Dynamics

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

Q1. The Rigid-Fall and Physion examples are trivial and solved problems for modern rigid-body simulators. However, the paper did not include these simulators as baselines and limited its comparison with learning-based simulators only. I struggle to see a strong motivation that can support this (lack of) comparison. The paper also contains several confusing statements regarding physics simulation in its introduction, which I feel are bending the storytelling in the paper’s favor.

A1. Although modern numerical simulations can effectively solve rigid body dynamics problems, learn-based methods also possess unique research value. It is based on long-standing considerations followed:

1. Numerical simulations are not easily integrated with learn-based neural network methods. Specifically, when simulating complex dynamic systems involving both rigid bodies and fluids, it is challenging to delegate fluid simulation and rigid body simulation separately to neural networks and numerical simulators. This makes it difficult to leverage the advantages of neural networks in fluid simulation. On the other hand, end-to-end neural networks can adapt to such complex scenarios by incorporating the simulation of rigid bodies, fluids, and their interactions together (which we are currently exploring). Although this paper does not address fluid simulation, the exploration of learn-based simulations of rigid bodies constitutes a significant precursor to the complex system with rigid and non-rigid. I believe this is one of the most impressive directions for practice and development of our EGODE.

2. Besides practical implications, our EGODE also has methodological development. We adopt an event module with coupled ODE architecture to model the instantaneous updating of object states. This novel approach can be used not only in rigid body collision simulations but in any dynamic processes involving impulse or angular impulse, and even more broadly in systems including instantaneous state changes (eg. crushing and deformation).

3. Based on the aforementioned reasons, we have sufficient motivation to study and improve learn-based methods for simulating rigid body dynamics and leverage their strengths in appropriate scenarios, so we only utilized learn-based methods as our baselines for performance comparison.

4. Moreover, the simulation speed of EGODE is not inferior to numerical simulations. You mentioned that numeral simulators can compute fairly complicated rigid bodies at a speed of several millions of time steps per second. Similarly, our EGODE can also achieve efficient inference by leveraging GPUs and deep learning toolkits such as torch_geometric. In our paper, each scenario visualization contains thousands of nodes and spans 0.5~1 minutes, yet it only requires less than 1 second to simulate and render on a single laptop.

Q2. It looks like the positions and velocities of all mesh nodes are included in the state variable and evolved in the ODE (Eqns. 1-2). The number of these variables grows as mesh resolution increases, but they are essentially governed by their underlying rigid-body DoFs (6 per rigid body) only. Is it necessary to assign and evolve DoFs at each mesh node?

A2. The positions and velocities of mesh nodes are essential for modeling the interaction and inner-action of objects (eg. contacting, sliding, and pressing), initiating collision events, and performing dynamics calculations. Although the number of these variables increases with higher grid point density, it represents a trade-off between computational cost and performance. Furthermore, the degrees of freedom of surface and interior mesh nodes enable more refined modeling of dynamic phenomena such as compression, friction, and adhesion. Therefore, unless there is a complete absence of interaction between objects, recording the states of these nodes is beneficial.

Q3. Eqns. 3-4 roughly captures the linear motion of the “center of mass” at each rigid body. Using them as the object-level state does not seem to capture the angular motions and angular velocities of each object. Missing angular information on the object level is a bit counter-intuitive from a physics perspective.

A3. In Eqns. 3-4, linear motion is explicitly recorded at the object nodes of rigid bodies because $x$ and $v$ directly influences the state updating of simulation ($x_i^{t+1}$ is largely determined by $x_i^{t}$and $v_i^{t}$). Although angular momentum and angular velocity are also important, they can be implicitly embedded with other valuable information in hidden state $h^t$, rather than explicitly represented. The model learns to express and utilize them in an optimal way.

Q4. The collision events visualized in the figures seem between parametric surfaces (cubes and spheres) only. Such collision events could be easily resolved with closed-form solutions. These examples do not seem to show the benefits of having a mesh representation in collision detection. A more complicated scene with multiple organic surfaces would be a better example to necessitate these meshes.

A4. More visualization with more complex objects can be found in Figure.1 and Figure. 7. Specifically, Figure 1 showcases complex objects such as tower-shaped and torus-shaped objects. Our EGODE is designed compatible with rigid bodies of arbitrary shapes, not limited to parametric surface objects. Moreover, our baseline Physion dataset includes objects with various geometries and trajectories of various angles of collision and contact, to validate the generalization capability of our proposed EGODE. Additionally, the results of a complex scenario like Drape, which models interactions between deformable objects and rigid bodies, are reported in Table 3. The visualization of complex scenarios like Drape will be added in the revised version.

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

Q1. Is there a reason to leave out FiGNet [1] as a baseline when its mentioned in the introduction and its limitations are discussed in Related Work? Can you add the baseline to the experiments?

A1. We appreciate your suggestion to include FiGNet [1] as a baseline in our experimental evaluation. Unfortunately, due to the unavailability of the original code publicly, we didn't implement FiGNet in our scenario as a baseline for comparison in the first time. However, we understand the importance of this baseline and have decided to replicate FiGNet using the information provided in the original paper. Given the computational resources required to run FiGNet and the need to ensure a fair comparison, we plan to provide the results in an updated version of our submission or as supplementary material. We expect to complete this within the next few days.

Q2. Could you elaborate on the factors contributing to EGODE's higher relative performance on RigidFall compared to Physion? Are there specific characteristics of these benchmarks that favor EGODE's architecture?

A2. The superior performance of EGODE on the RigidFall benchmark can indeed be attributed to the simpler nature of the physical interactions present in this dataset. RigidFall consists of simulations involving just three cubes under varying gravitational conditions, all with uniform internal properties such as particle numbers, shapes, and friction coefficients, as shown in Figure.3. In contrast, Physion is a more complex dataset, encompassing eight distinct scenarios with multiple objects that vary in size, shape, and friction. For instance, Figure.1 showcases complex objects such as tower-shaped and torus-shaped objects. Additionally, Physion includes not only rigid-rigid interactions but also rigid-deformable interactions. The simplicity of the RigidFall dataset aligns well with the strengths of our method, which excels at modeling straightforward physical interactions. In addition, the performance of EGODE across both datasets demonstrates its robustness and adaptability in handling a range of physical phenomena.

Q3. For the generalization experiment shown in Figure 5 could you add some baselines like SEGNO for comparison? Its hard to see how good the proposed method generalized without any baseline references

A3. We agree that including additional baselines would enhance the interpretability of our generalization experiment presented in Figure 5. To address this, we will incorporate SEGNO and other relevant baselines for comparison. This will provide a clearer context for evaluating the generalization capabilities of our proposed method. We aim to finalize these additions and submit the updated figures within the next few days.

We are truly grateful for the time you have taken to review our paper, your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.

Q1. It would be helpful to be a bit more concrete with existing model shortcomings in the introduction. It is not completely clear what it means for existing approaches to “fail to take…into consideration” instantaneous changes. I think I know what you’re saying (e.g. GNNs model large instantaneous changes via iterative message passing, which can lead to issues — is that right?) but it would be good to be a bit more explicit in outlining these issues.

A1. Thanks for your comment. Your understanding is correct. GNN methods utilize iterative message passing to model the spatial relationships based on current distances, which are hard to capture large instantaneous changes. We will include it in the revised version.

Q2. Benchmarks could be described better. E.g. I assume the success metric shown for Physion is the accuracy on whether the objects of interest collided, right? Yet as far as I can tell, this is never mentioned.

A2. Thanks for your comment. The success metric denotes the accuracy of predicting whether the objects of interest collided. We will include it in the revised version.

Q3. In judging significance, if I’m understanding things correctly, you’re providing standard deviation estimates in the tables. It would be helpful to report SEM or give confidence intervals. The reported +- are often not particularly small relative to the improvements, to the extent that a number of them appear like they may not be significant. It would be helpful to clarify this.

A3. Thanks for your comment. We have conducted one-sample paired t tests to justify that all of the improvements with the best baseline are statistically significant with p-value < 0.01. Our problem focuses on the classification tasks and trajectories regression tasks, which is not a parameter inference task and thus cannot have SEM or CI for performance comparison.

Q4. More broadly, it makes sense to focus on rigid bodies, but much related work is for deformable bodies as well, and arguably, a major reason for using these complex neural models is to accommodate different sorts of media. It would be helpful to at least give a sense for how this method could be extended, or what challenges there are in extending it.

A4. Thanks for your comment. Deformable body dynamics involve complex interactions and behaviors not present in rigid body dynamics, such as stretching, bending, and compressing, which require different modeling approaches. Although we do not have specific low-level modules designated for deformable objects, our model can still handle complex scenarios involving interactions between deformable objects and rigid bodies due to its generalization ability. For instance, Table 3 shows our model's ability to model deformable objects' interaction with rigid bodies in the Drape scenario, where deformable objects are represented as cloths. This result showcases the potential application of our approach in modeling deformable object dynamics. In future work, we will extend our methods to deformable bodies on more complex and challenging datasets and may design modules specifically for processing deformable objects.

Q5. Could you provide confidence intervals on experiments?

A5. Thanks for your comment. Please refer to A3.

Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.

We are truly grateful for the time you have taken to review our paper, and your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your major concern and provide additional clarification.

Q1. The learnable event function, which operates on pairwise mesh points, requires exhaustive evaluation which might hinder the scalability of the approach in terms of the number of rigid bodies that can be simulated and in terms of the mesh resolution of the simulated rigid bodies.

A1. We agree that the pairwise evaluation of mesh points for event detection could potentially limit the scalability as the number of rigid bodies and mesh resolution increase. However, our event function does not directly process all pairwise mesh nodes, but instead performs rule-based filtering first (which was not elaborated in the main text due to space limitations). Specifically, Equation (7) first uses the distance condition $d(x_i, x_j)<d_{threshold}$ to filter out the node pairs that require event function computation, and then uses GNN to propagate information. This significantly reduces the time complexity of the function. Moreover, as the nodes become denser, we can also reduce $d_{threshold}$ to alleviate the computational burden of the event function. Since denser nodes make the collision analysis more detailed, reducing $d_{threshold}$ does not significantly affect the performance. We are conducting experiments to examine the performance and efficiency pratically, in terms of the number of rigid bodies and mesh node number, the results will be provided during the interaction periods in several days.

Q2. L77. The wording might be lacking a negation. Are other approaches able to "handle intrinsic continuity and discontinuity in rigid models"?

A2. Thanks for pointing this out, We will fix it in the official release. In our work, we utilize graph ODE and event function to handle intrinsic continuity and discontinuity in rigid models. We believe that other potential methods are also worth discussing, such as using 3D space or physical inductive biases of rigid bodies. We will further explore these approaches in future work to find more solutions to this problem beyond our proposed EGODE.

Q3. How fine-grained can the mesh-based representation be without deteriorating the performance of the approach?

A3. We have already discussed in A1 how to reduce the complexity of the event function. By reasonably selecting $d_{threshold}$, the event function can be controlled within $O(N)$, where $N$ is the number of nodes. We are conducting an experiment about the most fine-grained the graph can achieve without deteriorating the performance. Due to the heavy computation burden, we will provide the results during the interaction periods in several days.

Q4. Are there any scenarios where using standard mean square error (MSE) loss leads to poor performance? Would incorporating physics priors in the loss function further improve the performance or would it potentially reduce the amount of data required for learning?

A4. Although we currently achieve good results using the MSE loss, incorporating physics priors into the MSE is a novel idea. We have tried supervising the center of mass position $X_c^t$ using MSE as well. Due to the heavy computation burden, we will provide the ablation study during the interaction periods in several days.

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.

Q1. How does EGODE compare with traditional physics engines in terms of computational efficiency and accuracy?

A1. Thanks for your comment. The simulation speed of EGODE is not inferior to numerical simulations. Our EGODE can achieve efficient inference by leveraging GPUs and deep learning toolkits such as torch_geometric. In our paper, each scenario visualization contains thousands of nodes and spans 0.5~1 minutes, yet it only requires less than 1 second to simulate and render on a single laptop. Due to the heavy computation burden of data adaptation to traditional physics engines, we will provide the experimental speed comparation during the interaction periods in several days.

Q2. Can the authors elaborate on how EGODE might be extended to handle scenarios with rigid body hinges and deformable objects?

A2. Hinges and deformable objects are both meaningful and more challenging than rigid body dynamics. For hinges, each block can be treated as an independent rigid body, and special edges can be used in our EGODE to model the interactions between different parts at the hinge points, maintaining geometric and mechanical consistency. The fixed parts of the hinges can also be considered a special case of rigid body dynamics under constraints, where we evaluate our EGODE on a similar setting called Contain in Table 3 where container objects restrict other objects movements. The model can be trained by collecting sufficient data on hinge dynamics scenarios to improve its performance. While we did not specifically evaluate our model on single deformable objects, our framework is designed to handle more complex scenarios involving interactions between deformable objects and rigid bodies. As an example, we have evaluated our method on a scenario called "Drape" and the results are reported in Table.3, where deformable objects (cloth) interact with rigid bodies. This evaluation demonstrates the potential of our approach to handle deformable object dynamics and its generalization ability.

Q3. What are the theoretical underpinnings of the coupled graph ODE framework, and are there any proofs of concept?

A3. In Appendix B, we have provided the theoretical underpinnings to show that it is possible to propagate gradients across the event times to the input arguments of the system and therefore our method can be optimized by SGD. Besides, our work focuses on empirical results since both neural ODEs and physical scenarios involve too many parameters. In future work, we consider simplifying the model and developing more interpretable models with theoretical underpinnings. We will include this in the revised version. Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.