We thank the reviewer for their valuable feedback.

W1: Possibility of Using The Method in Real Applications

Firstly, synthetic data is commonly used and facilitate the research of garment animations, such as TailorsNet [1], Cloth3D [2], MotionGuided [3], Clo3D [4].

Secondly, as shown in the dataset [5], it is feasible to obtain temporally consistent cloth data represented by vertices from real life, which can be directly applied by our method to learn the constitutive laws in real world, though it is not yet publicly available.

Thirdly, the garment geometry can be extracted either through learning based methods [6, 7] or from raw scans [8], making it possible to animate realistic garments. As shown in Figure 2 in additional PDF from global rebuttal, we animate the garments from real scans [8] for demonstration, transferring the virtual material from Blender to real-life garments from CAPE [8].

In conclusion, our work can be directly applied to real-life applications related to garment animations.

[1] Chaitanya Patel, et al. TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. CVPR2020

[2] Hugo Bertiche, et al. CLOTH3D: Clothed 3D Humans. ECCV2020

[3] Meng Zhang, et al. Motion guided deep dynamic 3D garments. TOG2022

[4] Xingxing Zou, et al. Cloth4d: A dataset for clothed human reconstruction. CVPR2023

[5] David Blanco Mulero, et al. Benchmarking the Sim-to-Real Gap in Cloth Manipulation. IEEE Robotics Autom2024

[6] Heming Zhu, et al. Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single. CVPR2022.

[7] Lingteng Qiu, et al. REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos. CVPR2023

[8] Qianli Ma, et al. Learning to Dress 3D People in Generative Clothing. CVPR2020

Q1: Qualitative Results of Dress

We acknowledge that predicting long-term dynamics in an auto-regressive manner inevitably leads to error accumulation. The baseline models either struggle with predicting long-term animations or fail to replicate similar constitutive behaviors observed in the ground truth.

In contrast, MGN-S+EUNet (ours) delivers material behaviors that more closely match the ground truth, such as a more rigid dress in Figure 4 of the main manuscript, and achieves lower prediction errors as shown in Table 2 of the main manuscript, suggesting the effectiveness of our learned EUNet. Please refer to the video in supplementary for better comparisons and the additional PDF file for more comparisons.

Q2: Explanations of Errors in Table 1 of The Main Manuscript

The unit is the energy $kg\cdot m^2/s^2$. The errors in Table 1 of the main manuscript are the square errors as indicated in Equation 8, where we sum up 1.4K energy units of the cloth with 484 vertices. The per-energy-unit errors are significantly smaller in terms of the absolute value.

Regarding the differences between the materials, leather and denim are either stiffer or heavier. These characteristics lead to larger energy changes between frames due to significant variations in velocity or increased mass, making it more challenging for the model to learn. We would include the discussion in the limitation section in our revised version.

In conclusion, as shown in Table 2 of the main manuscript, MGN-S+EUNet (ours) still achieves superior performance, suggesting the effectiveness of our EUNet. And as shown in Figure 1 of the main manuscript and the video in supplementary, MGN-S+EUNet delivers reasonable material behaviors for garments made of silk and leather, which are softer or stiffer respectively.

We thank the reviewer for the positive feedback and recognizing the value of our work. We believe the reviewer has sufficient understandings of our framework and pipeline.

W1: Details Regarding The Design of EUNet

We introduce the formulations and training procedures of our EUNet in Section 3.2, and include the implementation details in Section 4.1 at L239-249.

Furthermore, we report the time efficiency of our EUNet in the global rebuttal.

Please let us know if the reviewer still has any confusion about our EUNet. We will clarify it further in our revised version to ensure it is easier for readers to follow.

W2.1: Anisotropic Materials

A naive extension is to add the directional information as extra inputs for our EUNet. As a simplified example, we assume a rectangular wooden board that is thin enough for its thickness to be negligible. The board is stiffer along the x-axis than the y-axis. We then discretize the thin board by edges along both the x-axis and y-axis. To model such material, we can extend our EUNet by taking the edges' directions, which are either x-axis or y-axis in this example, as extra inputs by $\Phi(\cdots, \mathbf{d})$, where $\mathbf{d}$ indicates the directional information.

W2.2: More Complex Topology Than A Piece of Cloth

For cloth or garments with complex topologies used as training data, our EUNet can directly learn their constitutive behaviors thanks to the topology-independent design. To achieve accurate results, it is essential to ensure that the training sequences involve minimal collisions, so that the dynamics and deformations are primarily caused by internal forces closely related to potential energy.

The reason we chose to use a simple piece of cloth as training data is that this setup is easier to achieve with high quality in real-life scenarios. For example, [1] provides the temporally consistent cloth data represented by vertices from real life, which can be directly applied by our EUNet but not publicly available yet.

[1] David Blanco Mulero, et al. Benchmarking the Sim-to-Real Gap in Cloth Manipulation. IEEE Robotics Autom2024

W2.3: Handling of Self-collisions

Since collisions are independent of the internal forces caused by deformation, the key to accurately modeling potential energy is to minimize the effects of collisions. However, to extend our method to more complex scenarios involving collisions, one could design a new module that takes the distances between edges or faces of the cloth as inputs to model the dissipation energy caused by collisions.

In this work, we minimize the effects of collisions during the data generation process. We will include such scenarios in future work.

We thank the reviewer for their valuable feedback.

W1: Design of Dissipative Energy

As commonly adopted in physics simulation, such as the Rayleigh dissipation, the dissipation can be approximated by a function of objects' velocities, which can be calculated from $X^t-X^{t-1}$ in our work.

In addition, we agree that the absolute value of the dissipative energy is not meaningful. However, during the energy optimization process of solving dynamics, only the derivative of the dissipation energy $\lim_{\Delta x \to 0}\frac{\Phi_d(x+\Delta x, \cdot)-\Phi_d(x, \cdot)}{\Delta x}$ will take effect, and eliminate the potential impacts from the absolute values.

Lastly, if there exists a constant $C>0$ in the output of our $\Phi_d$, it indicates that the system from Blender has a constant damping effect. When the velocity is 0, the output of $\Phi_d$ is $C=3\times 10^{-6}$, which is small enough to be negligible.

W2: Dealing with Collision And Friction

We include the frictions between cloth and air, which is common in daily life, in both the generated data and modeling process. As shown in Table 1 and Figure 3 in the main text, with the modeling of dissipation by $\Phi_d$, our EUNet achieves lower errors and delivers more reasonable energy mappings as the cloth deforms, suggesting the effectiveness of $\Phi_d$.

In addition, since collision forces are independent of the constitutive behaviors, we do not model collisions as part of our EUNet. We avoid the self-collisions in training data by setting an upper bound on the initial cloth's velocities and the range of their directions.

Quantitative results in Table 1 of the main manuscript and the performance on garment animations demonstrate the effectiveness of our EUNet.

W3: More Evaluations to Verify Learned Constitutive Model

Since the final goal is learning to animate the garments where EUNet is key module of this framework, we conduct comprehensive experiments on garment animations in Table 2 and Figure 4 in the main text.

In addition, we further implement HOOD [1] as an extra baseline. Please refer to the PDF in global rebuttal for more details about the performances and the differences between baselines. As shown in Table 2 of both main manuscript and the global PDF, either MGN-S or MGN-H (used in HOOD) being the neural simulator architecture, simulators constrained by our EUNet always outperforms baselines (i.e., HOOD and others in the Table 2 of the global PDF), indicating the effectiveness of our EUNet.

[1] Artur Grigorev, et al. HOOD: hierarchical graphs for generalized modelling of clothing dynamics. CVPR2023

W4: Comparisons with HOOD and Analytical Clothing Model

Firstly, as discussed in W3, we further compare with HOOD, where simulators constrained by our EUNet outperforms it.

As for the claim 'better than the analytical constitutive model', it is worth noting that we didn't claim it in abstract and introduction. We only mentioned this in our experimental section as an empirical analysis at L298-299. We will clarify this in the revision.

Moreover, to further verify the advance of our EUNet over analytical models is not caused by the limited accuracy of neural simulator, we pair our EUNet and the analytical models with two different neural simulator architectures, namely MGN-S and MGN-H. As a result, simulators constrained by our EUNet always achieve lower errors, which means adopting our disentangled learning scheme with EUNet is better than using analytical models to train neural simulators.

Thirdly, we further report the comparisons that with or without the mini-network in Table 2 from the global PDF. The mini-network enables the unsupervised simulators, which are constrained by analytical physics model, to obtain higher accuracy.

Finally, we admit that implementing numerical simulations or PDE solvers within a short period is challenging. However, within the scope of learning to animate garments and based on the experiments, our method can better enhance learning-based simulators to achieve lower errors and faithful animations.

We thank the reviewer for the positive and valuable feedback.

W1: Computationally Intensive Energy Optimization Process

In this paper, we primarily focus on the challenge of modeling constitutive relations from observations. The energy-based simulation is used solely as a tool to solve the dynamics constrained by our EUNet. Consequently, accelerating the energy optimization process is beyond the scope of our work.

W2: Extensive Experimental Validation

As requested by the reviewer, we implement HOOD [1] for further comparisons. Moreover, the baselines we have chosen are representative: MGN is the pioneering approach for animating mesh-based cloth, LayersNet achieves state-of-the-art performance for garment animation trained in a supervised manner, and MGN-S+PHYS employs a recent advanced unsupervised learning scheme as shown in SNUG [2] and HOOD. The main difference between MGN-S+PHYS and HOOD is that HOOD adopts a hierarchical graph neural network. Please refer to the additional PDF in the global rebuttal for more details. Simulators constrained by our EUNet achieve superior performance over other baselines.

In addition, all garments in Cloth3D [3] differ from each other in terms of length, size, and topology. Each category in Table 2 of the main manuscript includes several subclasses. For example, the "T-shirt" category also includes "Jackets" as shown in Figure 1 of the main manuscript, and the "Dress" category includes both short and long dresses. "Jumpsuits" and "Dresses" are combinations of upper and lower garments.

[1] Artur Grigorev, et al. HOOD: hierarchical graphs for generalized modelling of clothing dynamics. CVPR2023

[2] Igor Santesteban, et al. SNUG: Self-Supervised Neural Dynamic Garments. CVPR2022

[3] Hugo Bertiche, et al. CLOTH3D: Clothed 3D Humans. ECCV2020

Q1: Efficiency and Accuracy Comparing with Physics-based Models

As shown in global rebuttal, our EUNet is comparable to the traditional clothing model in terms of speed. While our EUNet contains two branch: $\Phi(\cdot)$ for potential energy and $\Phi(\cdot)$ for dissipation, each branch is slightly faster than the traditional clothing model.

In terms of accuracy, as shown in Table 2 of both the main manuscriptand the global PDF, simulators constrained by our EUNet achieves higher accuracy than those constrained by physics-based clothing models. Furthermore, our method reduces the need to carefully select different types of physics-based clothing models and estimate the corresponding parameters. Instead, our EUNet directly captures the observed constitutive laws, enabling learning-based simulators to achieve superior performance in garment animation.

Q2: Limitations of Using A Piece of Cloth

One limitation is that a single piece of cloth may not encompass all possible deformations and corresponding dynamics. For example, shear deformation may be less obvious, and interactions among different layers of clothing are unavailable. A simple solution to enrich the deformations and dynamics is to apply known forces, such as those controlled by robots, and to use multi-layered clothing during data generation to explore these interactions

On the other hand, we intentionally design the training data to be as simple as possible to ensure its applicability in real scenarios. An example is the data from [1], which can be directly adopted by our model but currently unavailable.

[1] David Blanco Mulero, et al. Benchmarking the Sim-to-Real Gap in Cloth Manipulation. IEEE Robotics Autom2024

Limitations: Comparison with HOOD

Please refer to W2.

We thank the reviewer for the detailed feedback.

W1: Insufficient Literature Review

We argue that our focus is quite different from Physics Parameter Estimation. The misunderstanding may come from the similarity in data format, where a piece of hanged cloth deforms given external forces. Unlike physics parameter estimation, our EUNet aims at learning the unknown ENERGY FUNCTION that describes the constitutive laws. While in physics parameter estimation, the constitutive models are known a priori and carefully chosen.

Specifically, with known constitutive models, physics parameter estimation aims at learning physics parameters, such as the lame constant in St. Venant-Kirchhoff elastic model for the target material. Complex devices and procedures, such as KES-F [1], the FAST system [2], and the Cusick drape test [3] are commonly required to estimate the parameters that best fit the analytical cloth models or simulators. Moreover, for a specific material, such as cotton, different clothing models or simulators require separate procedures or pipelines to estimate different sets of physics parameters.

In contrast, our EUNet directly captures constitutive behaviors of different materials, which takes deformation attributes (i.e., delta of length and bending theta) and material attributes (i.e., stiffness and damping coefficients) as input, and outputs the corresponding potential energy. The material attributes remain constant in our study. As discussed at L84-88, our EUNet does not need existing analytical physical models and physics parameter estimation.

In terms of literature review, as mentioned above we focus on learning to animate garments where modeling constitutive behaviors is the key challenge, we thus mainly discuss the garment animation and constitutive laws in the literature review section at L108-126 and L127-136 respectively. Energy-based physics simulation serves as a tool to generate dynamics constrained by our EUNet, and we briefly introduce the corresponding techniques at L137-144.

We will clarify the relationship between our focus and physics parameter estimation in the revision.

[1]. Kawabatak, et al. Fabric performance in clothing and clothing manufacture. Journal of the Textile Institute. 1989

[2]. Minazio, et al. FAST–fabric assurance by simple testing. IJCST1995

[3]. Cusick, et al. The dependence of fabric drape on bending and shear stiffness. Journal of the Textile Institute Transactions 1965.

Q1: Literature Positioning

Please refer to W1.

Q2: Physics Simulation/Graphics Community References

As discussed in W1, our focus is on how to learn the constitutive laws, which is discussed at L127-136 and involves several references from graphics community. We verify our EUNet by applying existing physics simulation techniques, which we briefly discussed at L137-144.

Q3: Experimental Validation

As discussed in W1, our work is orthogonal to physics parameter estimation, thus comparing with methods mentioned by the reviewer is unnecessary.

In addition, as discussed in W1 and Q2, our goal is to faithfully animate garments with observed materials where modeling constitutive relations is the key challenge. We thus verify the effectiveness of our EUNet by integrating with simulation techniques and evaluate the garments animations on Cloth3D [1], which is a large scale dataset and commonly used in garment animations, such as DeePSD [2] and MotionGuided [3].

[1] Hugo Bertiche, et al. CLOTH3D: Clothed 3D Humans. ECCV2020

[2] Hugo Bertiche, et al. DeePSD: Automatic deep skinning and pose space deformation for 3D garment animation. ICCV2021

[3] Meng Zhang, et al. Motion guided deep dynamic 3D garments. TOG2022