NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory

We thank the reviewer Gekm for the detailed comments. The reviewer notes that our cloth modelling “offers infinite resolution”, and the method “does not suffer from numerical locking issues” like classical mesh-based methods. We now address the remaining concerns:

Distinction from PINN-approaches for elastodynamics

Thanks for suggesting the paper by Rao et al. It is relevant, and we will include it in the related works. We agree that we are not the first (or the only) ones to apply neural implicit representation for elastodynamic problems. At the same time, our setting addresses thin-shell and, in particular, realistic cloth simulation that the earlier volumetric elastic works do not [Rao et al. 2021; Zehnder et al. 2021]. Volumetric modelling will often lead to ill-conditioned optimisation when one dimension is thin, necessitating the modelling of thin-shell kinematics. Moreover, to the best of our knowledge, prior works do not tackle geometric nonlinearity: Formulating non-linear strain (Eq.(5)) is crucial for large deformations and rotations arising in cloths (Sec. E.3 ablation). In addition, separating bending and stretching deformation modes allows us to better integrate data-driven non-linear anisotropic models (Secs. 4.3, B.3), unlike the linear elasticity used in Ref. [Rao et al. 2021; Zehnder et al. 2021]. Further, our extensions, such as material conditioning, non-analytical reference geometry and simulation editing, present many opportunities for computer graphics and vision.

We agree that we did not clearly highlight the differences. In the revised version, we suggest to add the following statement: “While previous works like Rao et al. and Zehnder et al. applied neural implicit representations for volumetric elastodynamic problems, our approach focuses on realistic thin-shell and cloth simulation. It addresses important cloth simulation aspects such as geometric non-linearities and the integration of non-linear anisotropic models, which are crucial for simulating large deformations and rotations.”

Figure 6

In this experiment, we show that the classical mesh-based simulators are sensitive (produce different folds and wrinkles) to the discretisation of the reference geometry. We use the discrete meshes as inputs for both competing methods [36,38] and ours for a fair comparison. Thus, for NeuralClothSim, we use the discrete meshes (I, II, and III) to train the initial geometry MLP using the process described in L152-158. The exact discretisations of the initial shape and additional comparison to DiffCloth[36] are visualised in Fig. XII-appendix; Fig. 6 is a shortened version of Fig. XII-appendix due to space constraints of the main paper.

Consistency evaluation of classical simulators

For FEM-solvers, we agree and expect the convergence to a continuous solution upon refinement. We tried ARCSim[46] for three slightly perturbed initial mesh discretisations (a similar setup as our Fig. 6 comparison) at increasing resolutions and observed improvements. For a simulation of the napkin with 10k vertices and a runtime of 18 minutes (a few minutes higher than our napkin example), we visualise the results in Fig. 1 of the rebuttal pdf. Indeed, there is an improvement in the degree of consistency in the higher-resolution case (Fig.1-rebuttal) compared to Fig. 6/XII's result of 400 vertices. However, the results still contain noticeable inconsistency, we think it could be due to several operations that are highly discretisation-dependent (such as the bending model relying on the dihedral angles) [46,60]. In contrast to FEM-based solvers, our method produces consistent results and is much less sensitive to discretisations of the initial states as shown in Fig. 6 and Fig. XII.

References

[Rao et al., 2021] Rao, Chengping, Hao Sun, and Yang Liu. "Physics-informed deep learning for computational elastodynamics without labeled data." Journal of Engineering Mechanics (2021).

[Zehnder et al., 2021] Zehnder, Jonas, et al. "Ntopo: Mesh-free topology optimization using implicit neural representations." NeurIPS (2021).

We thank the reviewer BLQv for their comments. The reviewer nicely summarizes our paper and notes that our method "leverages a principled and sophisticated thin shell theory", "may spur further work in neural cloth simulation", and that our presentation is "pretty thorough". We now address the points raised in the review.

Initial fit and multiple panels

Kindly see our general comment G3.

Inverse design

Thanks for suggesting this interesting point, which could further expand the usefulness of NeuralClothSim. Although we did not attempt the inverse design scenario of accurately estimating forces/materials, we made initial attempts for NeuralClothSim as a physics-based prior for ill-posed inverse problems. Using the thin-shell hyperelastic energy as a loss function in addition to data terms will have a spatial regularisation effect for improving the physical plausibility (force-material ambiguities would remain in this ill-posed setting). Such ideas are used in earlier works [Kairanda et al., 2022; Yang et al., 2023] with mesh-based physics simulators. These could benefit from our continuous representation and memory adaptivity. While a thorough investigation of this direction would be a standalone research project, we already have promising results. In Fig. 3 of the rebuttal pdf, we visualise fine-grained surface reconstruction from monocular video using NeuralClothSim as a thin-shell physical prior.

References

[Kairanda et al., 2022] Kairanda et al. "f-sft: Shape-from-template with a physics-based deformation model." CVPR 2022.

[Yang et al., 2023] Yang, Gengshan, et al. "Ppr: Physically plausible reconstruction from monocular videos." ICCV 2023.

We thank the reviewer bD8m, for their comments. The reviewer notes that our "method is technically sound", and the "results are compelling". We will update the draft to include minor comments, such as copy editing. We now address the points raised in the review.

Discretisation-independence

Our statement at L49-50 needs more elaboration, and we will include the following in the revised version. The statement includes two observations regarding consistency that our method offers, but the finite-element methods lack: 1. consistency with respect to the discretisation/meshing of the initial geometry (Sec. 2, Figs. 6, XII), 2. consistency with respect to multi-resolution simulation (Sec. H.2, Fig. XI), also well-explored in [Zhang et al., 2022]. Regarding the latter, there are different paradigms for speed vs. quality trade-offs for FEM-based cloth simulators and NeuralClothSim. FEM-based methods can have increased speed by reducing the spatial resolution. We highlight that such a trade-off is conceptually not possible for NeuralClothSim as we model the cloth as a continuous surface throughout the entire training. Instead, we can reduce the training time for partial convergence as we did in Fig. X. Thus, a one-to-one comparison is conceptually difficult, which we also elaborate on in Sec. H.2 (supplement).

Regarding sensitivity to initialisation, while FEM-based cloth simulators are designed to be deterministic, in practice, there are several factors (such as numerical precision and parallel computing) that can lead to slight variations in the simulation results between runs. This inconsistency is not problematic as cloth simulation doesn't have a single ground truth; rather, it can have multiple equilibria solutions under the same input parameters (template, material, and boundary conditions). We observed that a mesh-based simulator running the same simulation scenario on different machines generates non-identical results but leads to reproducible results on the same machine. This is indeed the case for ours as well. We conducted two experiments: 1) We can obtain reproducible results if we set the random seed leading to the same network initialisation (Fig. 5-(left) in rebuttal pdf), and 2) we observe non-identical results if we do not set the random seed (Fig. 5-(right)). Our results are indeed somewhat sensitive to the initialisation of the neural network weights, similar to the classical simulators are sensitive to hardware- and parallelisation-related effects.

Arbitrary rest shape and topology

Kindly see our general comments G3.

Non-boundary constraints

Yes, we support point and shape constraints in the interior of the cloth. For those, no change in the method is necessary, i.e., $\partial \Omega$ in Eq. (4) can well be a set of boundary points inside the domain. In Fig. 4 of the rebuttal pdf, we show a simulation of non-boundary constraints. Moreover, when the initial geometry is provided as a mesh (instead of analytical definition), mesh vertices can be specified as point constraints $\partial \Omega$ where $(\xi^1_{\partial \Omega}, \xi^2_{\partial \Omega})$ now correspond to curvilinear coordinates of the fixed vertex. We have demonstrated such results in Fig.6/Fig. XII.

Derivatives

We agree with the observation that the mixed partial derivatives of $\mathcal{F}_\Theta(\boldsymbol{\xi})$ with respect to $\Theta$ and $\boldsymbol{\xi}$ are required to exist. We will update the draft.

Extension to dynamics

Kindly see our general response G1.

Sampling

During training, we sample the surface with a stratified/jittered sampling technique (points perturbed within a uniform grid). We resample at each training iteration to continuously explore the parametric domain. The number of training samples is typically determined by the available GPU memory. At test time, we sample regular grid points so that it is easy to triangulate a deformed mesh. Moreover, samples at inference can be generated at much higher resolution as it requires a single forward pass, unlike the expensive derivative computations of physical quantities that are required during training.

References

[Zhang et al., 2022] Zhang, Jiayi Eris, et al. "Progressive simulation for cloth quasistatics." ACMTOG 2022.

[Müller et al., 2022] Müller, Thomas, et al. "Instant neural graphics primitives with a multiresolution hash encoding." ACMTOG 2022.

[Xie et al., 2024] Xie, Tianyi, et al. "Physgaussian: Physics-integrated 3d Gaussians for generative dynamics." CVPR 2024.