Gaussian-Augmented Physics Simulation and System Identification with Complex Colliders

We thank the reviewer for their valuable feedback. Specifically, we appreciated the comments on quality, clarity and recognizing the value of handling different primitives (i.e., mesh and 2D Gaussians) in our framework.

In this rebuttal, we provide additional experiments on various aspect of the colliders, evaluate our method on additional material types and shapes, and provide additional details on the training time.

Below, we address every point raised by the reviewer.

[Q1]. Per Weakness 1, could the authors present more results [...]

(1) Move collider to the side

We conducted additional experiments in which the collider is moved to the side, so that the material first impacts the ground and then interacts with the collider as it spreads along the surface. We evaluated our AS-DiffMPM across nine experiments—comprising three material types and three colliders. For brevity, we report the results per material type, averaged over the three colliders (Box, Bunny and Armadillo)

This configuration appears to introduce significant challenges in physical parameter estimation, likely due to this ground-first-collider-next complex interactions between the material and the collider. However, a more extensive evaluation would be needed to draw definitive conclusions.

Newtonian

Non-Newtonian

Granular

(2) Colliders of different size

For this analysis, we used AS-DiffMPM and ran 18 experiments: three material types with three colliders, each tested at two scales (0.5 and 1.5), resulting in 3 × 3 × 2 = 18 experiments. The results are averaged over the colliders.

No significant differences were observed compared to the results in Table 1 of our manuscript, although a more extensive evaluation would be needed to confirm this observation.

Newtonian

Non-Newtonian

Granular

(3) Bowl collider

We imported a bowl-shaped collider into our framework and conducted a comparative study of our AS-DiffMPM method against the baselines. The bowl was positioned such that the falling material impacts its open surface—causing some particles to enter the bowl while others fall outside of it. We conducted three experiments, one for each material type. The results are reported below.

(4) Multiple colliders

We conduct three experiments—one for each material type—in scenes containing multiple colliders. Specifically, we place the Box, Bunny, and Armadillo colliders side by side and below the falling material, so that it interacts with all three as it spreads. These experiments are performed using our AS-DiffMPM method. The results are presented below.

(5) More material shapes

We thank the reviewer for the constructive feedback in the limitations section. We agree that evaluating system identification on a wider variety of material shapes would strengthen our empirical analysis. To this end, we use 9 shapes and their associated material types from previous work [1]. We evaluate our AS-DiffMPM method and the baselines. Each method is tested on all 9 shapes, with each shape colliding against the three colliders, resulting in 27 experiments per method. The results, averaged over the colliders, are presented below.

(6) Further experiments

Moreover, please refer to answer to point Q3 of reviewer rXMc for results on planar colliders.

[Q2]. Per Weakness 2, could the authors present results of system identification on elastic and plastic objects?

We present below the results for elastic and plasticine materials using our AS-DiffMPM method and the baselines. Each method was evaluated on six experiments: two material types and with three colliders. Results are averaged over the colliders. Interestingly, all methods generally perform well on the elastic material. This may be due to the simpler nature of the interaction between the elastic material and the collider—since the material tends to bounce away upon contact, the particle-wise collision handling mechanism of AS-DiffMPM has less impact. Indeed, in these scenarios, no fine-grained interaction (e.g., material flowing around sharp corners) occurs.

[Q3]. Per Weakness 3, [...]

Please see answer to point W1 of reviewer dazL.

[Q4].How long does it take (on average) to finish [...]

Please see answer to point Q4 of reviewer rXMc.

[1] Xuan Li et al. PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification. In International Conference on Learning Representations (ICLR) 2023.

Non-Newtonian material with parameters:
μ       = 57033.1 
κ       = 139650.4  
τY      = 7919.1  
η       = 25.1

After 1 second:
PSNR  = 33.2 ± 4.1
SSIM  = 0.994 ± 0.004
LPIPS = 0.019 ± 0.001

After 2 seconds:
PSNR  = 28.9 ± 6.6
SSIM  = 0.988 ± 0.011
LPIPS = 0.024 ± 0.011

After 3 seconds:
PSNR  = 25.3 ± 7.8  
SSIM  = 0.966 ± 0.013
LPIPS = 0.035 ± 0.012

We thank the reviewer for the insightful feedbacks on our work. We are happy to know that you recognize the contribution for handling complex geometries within the MPM framework and the results on system identification from both particle trajectories and visual observations.

In this rebuttal, we address concerns about the theoretical properties and computational efficiency of AS-DiffMPM. We clarify the distinction between particle-wise and grid-level collision handling, present timing benchmarks across methods, and include a convergence analysis of gradients of physical parameters during optimization.

We address below every point raised by the reviewer, and we welcome any follow-up question.

[W1]. MPM is fundamentally a Galerkin-based method—yet the paper does not analyze whether the differentiable formulation preserves energy conservation, admits closed-form solutions, or under what conditions it remains stable.

We thank the reviewer for raising this important point regarding the theoretical properties of our differentiable MPM framework. We have observed empirically stable behavior across all evaluated material types (Newtonian, non-Newtonian, and granular) and collider geometries (Box, Bunny, and Armadillo).

Our implementation builds upon PAC-NeRF [1] and the DiffTaichi framework [2], inheriting their default simulation settings: a timestep of Δt = 0.0002 and a grid spacing of Δx = 0.02. Our code explicitly monitors numerical stability after every simulation step by checking for the Courant–Friedrichs–Lewy (CFL) condition: v_max × Δt ≤ Δx, where v_max is the maximum particle velocity detected at the current timestep. This condition checks if a particle travels more than one grid cell per timestep. If violated, the common approach is to halve the timestep Δt and retry the simulation step.

[W2]. All P2G/G2P and collision computations occur on the background grid; the results are therefore highly dependent on grid resolution, with coarser grids potentially introducing aliasing or inaccuracy.

We apologize for the confusion. In our paper, collision computations (both collision check and velocity corrections upon collision) occur entirely on the background grid for the GOP-DiffMPM baseline and in scenarios where the collider has a simple, analytically defined shape (e.g., a plane, as described in lines 122–126 in our manuscript and adopted in previous work [1]). These approaches are highly dependent on grid resolution. In practice, when a collider surface intersects a grid cell, all particles within that cell receive the same velocity update—regardless of their position relative to the surface—which can lead to artifacts such as penetration. These artifacts tend to worsen as the grid resolution decreases.

In contrast, our AS-DiffMPM framework can partially mitigate these issues through particle-wise collision handling. Although we still use a background grid to determine which nodes are inside or outside a collider (i.e., collision check using the Collision Grid), velocity corrections are applied at the particle level during the G2P stage, rather than during the GOP stage. This enables control over each particle's interaction with the collider, allowing us finer control such as resolving penetration artifacts (see lines 183–185 in our manuscript). Because these corrections happen on a per-particle basis, AS-DiffMPM is less sensitive to grid resolution compared to grid-level methods.

We hope this response addresses the reviewer's concern. If further clarification is needed, we would be happy to provide it. We will also clarify this point in the revised manuscript.

[W3]. The paper positions CPIC-based collision handling as a more efficient alternative to hard-clamp methods, compensating for approximation errors with speed. However, it never quantifies whether the computational effort saved by avoiding per-particle constraints is fully realized once the overhead of scatter/gather operations and backpropagation through the grid is accounted for. This raises the question of whether any real efficiency advantage survives in practical, high-resolution scenarios.

We thank the reviewer for raising this important point on computational efficiency, which we acknowledge was not sufficiently analyzed in the original submission.

To address this, we provide below a performance comparison of the collision-handling strategies used in our work: GOP-DiffMPM (grid-level collision handling), RP-DiffMPM (collider represented as rigid particles), and AS-DiffMPM (our particle-wise CPIC-based strategy). Timings are measured per simulation step using a 64x64x64 background grid, 25K particles and during collisions with the Box collider.

Timing comparison

Moreover, we remind that our AS-DiffMPM extend the MPM by (1) constructing a Collision Grid (Sec. 4.1.1), (2) transfering the Collision Grid to material particles (Sec. 4.1.2), (3) adding compatibility checks during P2G and (4) particle-wise velocity corrections during G2P (Sec. 4.1.3). We report below the time required for these operations.

Overhead Breakdown for AS-DiffMPM

Although AS-DiffMPM introduces a modest overhead compared to the baselines, it obtains superior physical accuracy for system identification with complex colliders (see Table 1 in our manuscript), due to its fine-grained collision handling mechanism.

Regarding the mention of “hard-clamp methods”: we clarify that our work does not compare against true hard constraints, but rather grid-level collision correction (as in GOP-DiffMPM), which may lead to simulation artifacts (e.g., penetration) in high-curvature scenarios. Our contribution is not focused on raw speed improvements, but rather on enabling a more accurate collision handling through particle-wise velocity corrections.

[W4]. There is no discussion of the method’s numerical properties: e.g., under what time step or grid refinement regimes the optimization converges, whether the gradient signals remain well-conditioned, or if any spurious oscillations arise during backpropagation.

We acknowledge that our original submission did not include an analysis on optimization convergence, and we agree that such analysis would strengthen the empirical evaluation of AS-DiffMPM.

Below, we provide an analysis of the gradient of physical parameters over 150 steps (a subset of steps is visualized) during system identification using AS-DiffMPM. Specifically, we report results from 3 representative experiments, one per material, colliding against a Box shape.

Newtonian material

Non-Newtonian material

Granular material

The tables above illustrate how the gradients of the physical parameters consistently decrease in magnitude during optimization, suggesting that no problematic oscillations arise to prevent smooth training. While we acknowledge that a more in-depth numerical analysis would be beneficial, we did not observe any stability issues during our empirical evaluations. We therefore leave this study to future work.

[1] Xuan Li et al. PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification. In International Conference on Learning Representations (ICLR) 2023.

[2] Yuanming Hu et al. DiffTaichi: Differentiable Programming for Physical Simulation. In International Conference on Learning Representations (ICLR) 2020.

We thank the reviewer for the detailed reading of our paper and constructing suggestions. Specifically, we appreciated the recognition of the impact of the proposed method for handling complex colliders and the extensive evaluation.

Briefly, in this rebuttal, we provide additional experiments in the planar setting, clarify the distinct challenges posed by non-Newtonian versus Newtonian and granular materials, and present a computational efficiency analysis of different rendering backends.

We hope our responses below adequately address your concerns and we are open to further clarification.

[W1]. [No comparison with existing method on planar colliders] [...]

Please see Q3.

[W2]. [Lack of qualitative comparison with other methods] [...]

We thank the reviewer for the suggestion and agree that qualitative side-by-side comparisons would further support our claims. As per NeurIPS rebuttal guidelines, we cannot include additional media at this stage, but we plan to add such visualizations to the supplementary material.

[Q1]. In Table 1 (System Identification from Particle Trajectories), AS-DiffMPM appears to [...]. What characteristics of Non-Newtonian materials make AS-DiffMPM particularly effective?

We agree with the reviewer that this analysis requires further clarification. Firstly, we remind that GOP-DiffMPM operates at the grid level during the Grid Operations (G-OP) stage. It applies a single, averaged velocity correction to all particles within a grid cell that is detected to be inside a collider. This is a coarse approximation that may overlook fine-grained interactions between the material and the complex collider.

In contrast, our AS-DiffMPM performs particle-wise collision handling during the Particle-to-Grid (P2G) and Grid-to-Particle (G2P) stages. It identifies individual particles near a boundary and applies particle-wise velocity corrections based on their specific relationship with the collider surface. These differing collision strategies have practical implications for system identification, particularly depending on the material behavior.

For fluid-like materials (e.g., Newtonian fluids or granular materials), where particles tend to slide along the surface or flow away upon impact, the smoothing effect of GOP-DiffMPM’s grid-based velocity correction is often sufficient. Although it may introduce minor inaccuracies at sharp corners (e.g., slight particle penetration), the approximation can contribute to smoother gradients and potentially help stabilize the optimization process. This may partially explain why GOP-DiffMPM performs competitively—or even slightly better—in some Newtonian and granular settings. However, for non-Newtonian materials with higher yield stress (τY) [1], which exhibit solid-like behavior, this averaging becomes problematic. In such cases, particles are more prone to adhere to surfaces or accumulate near sharp corners. GOP-DiffMPM’s grid-level smoothing tends to blur these interactions, causing the simulated material to partially "bleed" into the collider geometry. This results in reduced accuracy when estimating physical parameters. In contrast, AS-DiffMPM’s particle-wise correction accurately resolves per-particle contact behavior, particularly around complex or high-curvature boundaries, resulting in better system identification results.

We acknowledge that such nuances were not mentioned in the original manuscript and we will revise it accordingly.

[Q2]. In Table 3 (Geometric Error Evaluation), AS-DiffMPM only outperforms baselines in the case of Granular materials, which does not fully align with the trends observed in Table 1 for System Identification [...]

We understand the reviewer's concern and acknowledge that this analysis was not throughly discussed.

There is a conceptual distinction between the metrics used in Table 3 (geometric error) and those in Table 1 (parameter estimation error), which explains why they can lead to seemingly different conclusions.

The Chamfer Distance (CD) and Earth Mover’s Distance (EMD) in Table 3 measure the geometric similarity between predicted and ground-truth particle configurations. These metrics compute spatial correspondences between particles in one point cloud and particles in the other point cloud, and quantify how closely the predicted point cloud matches the reference one. However, a low geometric error does not necessarily imply that the correct physical parameters were identified. Two simulations may produce geometrically similar shapes despite being governed by different underlying physics. Specifically, there can be similar shapes but with different particle trajectories. Therefore, upon convergence of the physical parameters estimation, the model might have different particle trajectories but similar shape with respect to the reference, thus better geometric similarity but worse parameters estimation.

We acknowledge that the results and analysis in Table 3 are not properly discussed and we will revise this in our manuscript.

[Q3]. The proposed method is titled Any-Shape Differentiable Material Point Method [...]

We thank the reviewer for this observation and agree that evaluating AS-DiffMPM under planar collider conditions provides valuable insight into its performance in standard benchmarks. To this end, we conducted additional experiments comparing our AS-DiffMPM with both the baselines used in our paper and the Standard MPM (S-MPM) collision strategy adopted by prior works [2, 3, 4]. In S-MPM, collisions are resolved at the Grid Operations stage (Sec. 3.1): for a planar surface defined by point x_b and normal n_b, the nodal velocity v_g is set to zero for all grid nodes x_g such that (x_g - x_b) · n_b ≤ 0. In contrast, AS-DiffMPM represents the same planar collider as a mesh surface positioned at x_b, and applies particle-wise collision handling during both P2G and G2P steps. We performed three system identification experiments using particle trajectory supervision under planar collider conditions. The results are summarized in the following table, reporting the absolute error in physical parameter estimation.

The S-MPM achieves slightly lower errors for some parameters, likely due to its use of analytically defined boundary conditions, which may offer improved numerical stability in simple geometries. However, a more systematic analysis would be required.

[Q4]. The method is evaluated in conjunction with several rendering pipelines (DVGO, 2DGS, and MDyn-3DGS) [...]

We report the average time for the following components, measured on an NVIDIA RTX 3080 GPU: (1) full training, (2) a forward pass (i.e., particle advection using MPM followed by rendering), and (3) a backward pass. Full training consists of 150 optimization steps. At each step, we perform a forward and backward pass over all 16 frames in the video, followed by an update of the physical parameters (gradient optimization step).

While MDyn-3DGS has generally demonstrated higher performance than the other methods (see Tab. 2 in our manuscript), it comes at a significantly higher computational cost. This is primarily due to its particle infilling strategy, which increases the number of particles in the MPM simulation, and the use of a loss function that combines both image reconstruction error (between rendered and ground truth masks) and geometric error on the particles. Additionally, the reported timing for MDyn-3DGS does not include the preliminary dynamic scene reconstruction stage required before physical parameter optimization. This preprocessing step is briefly discussed in lines 297–299 of our manuscript; for further details, we kindly refer to [4].

[1] Yonghao Yue et al. Continuum foam: A material point method for shear-dependent flows. In ACM Transactions on Graphics (TOG) 2015.

[2] Xuan Li et al. PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification. In International Conference on Learning Representations (ICLR) 2023.

[3] Takuhiro Kaneko. Improving Physics-Augmented Continuum Neural Radiance Field-Based Geometry-Agnostic System Identification with Lagrangian Particle Optimization. In Computer Vision and Pattern Recognition (CVPR) 2024.

[4] Junhao Cai et al. GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation. In Advances in neural information processing systems (NeurIPS) 2024

We thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of our paper’s quality, clarity, technical soundness and experimental evaluation. Below, we address each of the reviewer’s points in detail.

In summary, this rebuttal clarifies the limitations and open challenges that need to be addresses to extend our approach to real world scenarios, and provides additional experiments on novel material types and object shapes.

Please let us know if there is anything we can clarify further.

[W1]. experiments are only on synthetic sequences, would you think it generalizes to real scenes? Should be included in limitation section.

We agree that the limitations section should discuss this point. Specifically, a key difficulty lies in the nature of the materials we target — Newtonian, non-Newtonian, and granular — which lack a fixed rest shape. This differs from previous work on deformable solids [1], where shape reconstruction and parameter estimation can be decoupled—for example, by first reconstructing the object's rest shape and then estimating physical parameters based on observed deformations. Instead, when dealing with fluid or granular materials in real-world settings, these tasks must be addressed simultaneously, as the object's shape evolves continuously during interaction. For example, estimating the viscosity of a fluid being poured requires reasoning jointly over its shape and dynamics.

Our proposed AS-DiffMPM supports such scenarios (e.g., pouring into complex containers; see supplementary video at supp/SecC2/glass_newtonian.mp4), and we believe it represents a step forward toward system identification in uncostrained environments (i.e., involving complex colliders). However, to fully achieve system identification of fluid or granular materials in the real world, joint reconstruction and identification from real world video is required—an open challenge that lies beyond the scope of this work. We acknowledge that this limitation was not stated in the original manuscript and we will update the limitations section to reflect this point.

[W2]. It would be better to up-front explain that the focus is on Newtonian fluids, non-Newtonian fluids, and granular media. From the intro I was more expecting deformable solid objects to be tackled, as it was very general.

We thank the reviewer for pointing this out. We acknowledge that the introduction frames the work around the general challenge of system identification, without clearly stating our specific focus on fluid and granular materials. We will revise the introduction to explicitly clarify this scope.

[W3]. in the eperiment section authors refer largely to prior work. To be self contained, it would be beneficial to know how close the initial parameter guesses are to the GT etc.

For a fair analysis, we generated data and ground truth values using the same procedure as in previous work [2]. Therefore, the conditions (i.e., cameras, initial guess, ground truth values) are the same as in [2]. For completeness, for each material, we show below the initial guess and the different ground truth physical parameters for each sequence. We remind that there are 25 different sequences and 3 colliders, totaling 75 sequences for our experiments. We will revise the experiments section to clarify this point.

Newtonian (initial guess: µ=10, κ=10000)

Non-Newtonian (initial guess: µ=100, κ=100000, τY=10, η=1)

Granular (initial guess: θ_fric=10)

[Q1]. how important are tne #of sequences used, when does it degrade?

We would kindly ask the reviewer to clarify what is meant by “number of sequences used”. As described in Sec. 5 "Datasets" and "Physical Parameters", our dataset follows the protocol of [2]: we use 10 sequences each for Newtonian and Non-Newtonian materials, and 5 for Granular, repeated across 3 colliders, totaling 75 sequences. Each sequence is a multi-view video recorded from 11 cameras, and the optimization is performed independently per sequence. If the question refers to how performance scales with fewer camera views, we agree this is a relevant research direction. However, this analysis is beyond the scope of our current work. We kindly refer the reviewer to [3] for such analysis.

[Q2]. are there other limitations, have other material types been tested? Would certain shapes be problematic?

Please refer to our answers to reviewer CpqV for additional experiments in this direction. Specifically, we address material shape variation and further material types in answers to points Q1 and Q2 of reviewer CpqV.

[Q3]. why is the collider shape imported and not reconstructed from multi-view?

We thank the reviewer for raising this concern. Sec. 5.1 and 5.2 aims to quantitatively assess the system capabilities in recovering physical parameters. For a fair analysis, all other sources are set at their ideal condition. Thus, we import the collider shape instead of reconstructing it from multiple views since the reconstructed collider might be imprecise, this affects the particle trajectories and thus the estimation of physical parameters. We invite the reviewer to check Sec. B in the supplementary PDF, as we believe it addressess the concern raised here (i.e., it analyzes the performance drop when using the reconstructed collider instead of the imported one). We are open to further clarification if needed.

[1] Licheng Zhong, Hong-Xing Yu, Jiajun Wu and Yunzhu Li. Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians. In European Conference on Computer Vision (ECCV) 2024.

[2] Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang and Chuang Gan. PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification. In International Conference on Learning Representations (ICLR) 2023.

[3] Takuhiro Kaneko. Improving Physics-Augmented Continuum Neural Radiance Field-Based Geometry-Agnostic System Identification with Lagrangian Particle Optimization. In Computer Vision and Pattern Recognition (CVPR) 2024.