MaNGO — Adaptable Graph Network Simulators via Meta-Learning

We thank the reviewer for their time and for re-engaging with our submission. We appreciate the opportunity to respond to the points raised and provide clarification where needed. While we understand that parts of the submission may be familiar, we respectfully acknowledge that each venue offers a fresh review process, and we are grateful for the paper being considered within the context of NeurIPS.

On End-to-End Latent Inference vs. Two-Stage Regression

The primary motivation behind our approach is the joint shaping of the latent representation through end-to-end training. While we acknowledge that a two-stage pipeline (e.g., directly regressing material parameters and conditioning on them) may work well on simpler problems, we argue that our method promotes a more expressive and integrated latent space. This space captures not only the identity of the physical parameters but also their functional impact on the system’s dynamics.

This is exemplified in Figure 5b, where the learned latent representations exhibit a clear and meaningful separation—for example, between positive and negative Poisson’s ratios, corresponding to expanding and contracting materials. This reflects the latent’s ability to encode task-relevant structure, rather than merely regressing known quantities.

On the Use of Deep Sets Instead of Message Passing in the Encoder

We appreciate the reviewer’s suggestion and agree that message passing is an intuitive choice for many graph-based problems. In fact, we implemented and tested a message-passing-based encoder early in development. However, we found no significant improvement over the simpler and more efficient Deep Sets-based encoder.

We hypothesize that this is due to the global nature of the material parameters being inferred. Since the encoder has access to node positions, Deep Sets can effectively represent the necessary global characteristics. In contrast, message passing is particularly suited for modeling localized interactions, which are less relevant when estimating global latent variables such as material properties.

On the Finite Receptive Field of Temporal Convolutions

We acknowledge the concern regarding the limited receptive field of 1D CNNs, especially compared to EGNO's use of global Fourier-based convolutions. However, our design intentionally leverages local temporal modeling, which we find more effective for capturing short-horizon dynamics, particularly in the presence of contact interactions and localized temporal events.

Unlike EGNO, our model operates directly in the time domain, enabling it to learn localized temporal dependencies without assuming global periodic structure. In all experiments, we ensure that the CNN’s receptive field fully covers the sequence length used for training and evaluation, mitigating the risk of incomplete temporal context.

On Memory Usage and Trajectory Length

We agree that ingesting full trajectories introduces a higher memory footprint, which is a known trade-off of non-autoregressive models. In MaNGO, memory usage scales linearly with the number of predicted time steps, as the network retains activations across the full temporal sequence:

Memory(MaNGO)≈Memory(MGN)×Number of predicted steps

That said, this design enables significant inference speed-ups by eliminating the need for sequential rollout.

On the Sphere-Cloth Coupling benchmark:

• The CNP encoder takes approximately 6 ms to compute the latent representation rrr,
• The MaNGO decoder predicts the full trajectory in ~13 ms,
• By comparison, the autoregressive MGN requires ~500 ms, as it generates one step at a time.

To address the memory challenge, we propose a hybrid decoding strategy as a promising extension. Instead of predicting the full sequence at once, the model could generate smaller temporal windows (e.g., 10–20 steps) in sequence. This would substantially reduce peak memory usage while maintaining the benefits of batched inference.

We appreciate the reviewer’s insights and believe the above clarifications help explain the reasoning and trade-offs behind our architectural choices. We hope this response provides useful context for evaluating the submission.

We thank the reviewer for their detailed and thoughtful feedback. Below, we address each of the raised points:

Missing Comparison with Autoregressive Models

We would like to clarify that we do include a direct comparison with a meta-learning version of MGN (Meta-MGN) in our experiments. This baseline is shown in orange in the plots (e.g., Figure 4), and is discussed throughout the experimental section.

• MaNGO consistently outperforms Meta-MGN both quantitatively and qualitatively across all benchmark tasks. Visual comparisons can also be found in the appendix visualizations.
• In terms of runtime, MaNGO demonstrates significant efficiency due to its non-autoregressive architecture. Comparing on the Sphere Cloth Coupling task, we get:

  • CNP Encoder: ~6 ms

  • MaNGO Decoder (full trajectory): ~13 ms

  • Meta-MGN (autoregressive): ~500 ms

    All models were trained for the same number of steps to ensure a fair comparison.

These results support our claim that MaNGO offers a favorable trade-off: higher memory usage, but significantly faster inference and better accuracy.

Choice of Deep Set Aggregation in the Encoder

We fully agree with the reviewer that a message-passing-based encoder may appear more expressive. In fact, we experimented with message-passing architectures during development. However, we found that they offered no significant advantage over the simpler and more efficient Deep Set encoder.

We hypothesize that this is due to the global nature of the latent variable being inferred (material parameters like Young’s modulus and Poisson’s ratio are global properties). Deep Sets, especially when provided with node coordinates and features, appear sufficient for capturing these latent structures. In contrast, message passing typically excels at modeling localized spatial interactions, which may be unnecessary for globally inferred variables.

Memory Complexity and Scalability of MaNGO

MaNGO’s memory consumption scales linearly with the number of predicted future time steps, due to the retention of intermediate activations:

Memory(MaNGO)≈Memory(MGN)×Number of predicted steps

While this can increase GPU memory usage relative to step-by-step models, it allows for highly parallelized inference, offering substantial gains in runtime performance.

Practical Memory Limitations

There is no sharp "tipping point" in MaNGO’s memory profile. The practical limit depends on the available computing resources and the length of the trajectory to be predicted. In high-resolution or long-horizon simulations, the memory cost may become non-negligible.

To address this, a hybrid decoding strategy can be adopted: instead of predicting the entire trajectory at once, the model predicts shorter temporal windows (e.g., 10–20 steps) in sequence. This reduces memory consumption without reverting to fully autoregressive decoding and is a promising direction for future extensions.

Experimental Setup and Data Partitioning

We thank the reviewer for this important question and are happy to clarify our meta-learning protocol:

• We randomly sample material parameters (e.g., Young’s modulus, Poisson’s ratio) from a predefined distribution to define each task.
• For each material setting, we generate 16 simulation trials with diverse initial conditions (e.g., varying object positions or velocities).
• We apply a random split over material parameters to form the meta-training and meta-testing sets. Thus, material properties in the test set are never seen during training.
• During meta-testing:
  • We select 1 to 8 trials from a test task to serve as the context set.
  • We evaluate the model on the remaining 8 unseen trials, ensuring a clean separation between training, context, and evaluation.

This setup ensures no data leakage and measures the model's ability to adapt to entirely new material parameters with minimal supervision.

Please let us know if additional clarification or detail would be helpful. We greatly appreciate the reviewer’s engagement and insights.

We thank the reviewer again for the thoughtful and constructive feedback.

We hope our responses have addressed the reviewer's concerns. Given the limited discussion period, we would appreciate it if the reviewer could briefly confirm whether any issues remain unresolved. We would be glad to provide further clarification if requested.

We thank the reviewer for their consideration and support.

We thank the reviewer for their thoughtful and constructive feedback. We address the raised points below:

Clarity: "Figure 5" in line 290 should actually be "Figure 6"

Thank you for pointing this out. You are correct about this typo, and we will correct it in the camera-ready version.

Synthetic Data and Limited Graph Scale

We appreciate the reviewer’s concern regarding the use of synthetic datasets and relatively small, fixed-topology graphs (≤500 nodes). This was a deliberate design choice to systematically assess MaNGO’s meta-learning capabilities in controlled environments where:

• Ground-truth simulation parameters are available,
• Noise levels can be precisely controlled,
• Comparisons to oracle and ablation baselines are reproducible.

We agree that scaling to real-world data and variable topology is an important next step. While we have not yet conducted experiments on meshes with >10k nodes or on scenarios involving fracture, contact-breaking, or remeshing, we note the following:

• MaNGO’s architecture is computationally scalable and can, in principle, be applied to significantly larger graphs. The primary challenge is the memory footprint incurred when predicting full trajectories in a single forward pass. To address this, a promising extension is a hybrid decoding strategy, where instead of predicting all remaining time steps at once, the model predicts smaller temporal windows (e.g., 10–20 steps) sequentially. This would reduce peak memory usage by limiting the temporal span of each prediction and retain the advantages of batched inference. Alternatively, a more engineering-focused solution using multi-GPU parallelism is also a viable direction to handle large-scale setups efficiently.
• To support dynamic topologies, MaNGO could be extended to rebuild the graph connectivity at each time window, using dynamic neighborhood graphs (e.g., k-NN or radius-based methods). While promising, this introduces additional design challenges such as temporal consistency and remeshing dynamics, which we consider outside the scope of the current work, but worth exploring in future research.

Training resources

We appreciate the reviewer’s concern regarding the computational resources used in our experiments.

We agree that the total training and hyperparameter tuning cost—approximately 8,500 GPU-hours for training and 6,000 GPU-hours for search—is considerable. Upon reviewing our pipeline, we observed that data loading and preprocessing could still be optimized. The GPU usage logs indicate that improving these components could further reduce idle time and accelerate training.

That said, it's important to highlight that Graph Network Simulators (GNS) are generally resource-intensive to train, especially on mid-scale physics environments with complex interactions. This characteristic applies equally to the baseline models, which are also non-trivial to train.

Importantly, we trained all models under identical hardware and time constraints, ensuring a fair comparison in terms of real-world training cost. Within those constraints, MaNGO consistently achieved strong generalization and fast inference, which we believe justifies the computational investment.

Inference-Time GPU Memory and Runtime on SCC Benchmark

• Runtime efficiency: One of MaNGO’s key advantages is its ability to predict entire trajectories in a single forward pass, enabling effective batched inference over time. This leads to a significant speed-up at inference time compared to traditional autoregressive next-step simulators.

  On the Sphere-Cloth Coupling benchmark:

  • The CNP encoder takes approximately 6 ms to compute the latent representation $r$,
  • The MaNGO neural operator decoder simulates the full trajectory in just 13 ms,
  • In contrast, the MGN next-step simulator requires approximately 500 ms, as it predicts step-by-step over time.

• GPU Memory: Memory usage scales linearly with the number of predicted time steps, as the model retains intermediate activations across the temporal dimension. In our experiments: Memory(MaNGO)≈Memory(MGN)×Number of predicted steps

  While this can become a bottleneck for long sequences or large meshes, the hybrid decoding strategy discussed above offers a clear path forward, without requiring major architectural changes.

Once again, we sincerely thank the reviewer for their thoughtful and encouraging feedback. We appreciate the opportunity to clarify key aspects of our work and are glad that the contributions of MaNGO were well recognized.

Dear Authors,

Thank you for addressing my concerns in detail.

I appreciate the rationale for starting with carefully controlled, fixed-topology benchmarks and agree that the proposed hybrid-window decoder and dynamic-neighbourhood graphs are promising avenues for scaling. Your clarification that memory, rather than graph operations, is the principal bottleneck is also persuasive.

That said, the substantial training time—roughly 8.5 k GPU-hours for training plus 6 k GPU-hours for hyper-parameter search—remains a significant practical limitation. Until this cost can be reduced or better justified, it tempers the immediate applicability of the method. Nevertheless, I find the conceptual contribution both novel and compelling, and I look forward to seeing MaNGO further developed and applied to broader settings.

Consequently, I maintain my overall recommendation of "borderline accept" and leave my category scores unchanged.

We thank the reviewer for the positive feedback and their interest in our work. The reviewer asked about the parameter count and runtime efficiency, which we are happy to clarify below.

• Parameter Counts: All models use the same Graph Neural Network (GNN) backbone to ensure a fair architectural comparison. The full network comprises approximately 3 million parameters.

• Runtime Efficiency: A key advantage of MaNGO is its ability to predict entire trajectories in a single forward pass, enabling efficient batched inference over time. This results in significant inference-time speed-ups compared to traditional autoregressive next-step simulators.

  On the Sphere-Cloth Coupling benchmark:

  • The CNP encoder takes approximately 6 ms to compute the latent representation.
  • The MaNGO decoder simulates the full trajectory in just 13 ms.
  • In contrast, the MGN next-step simulator requires approximately 500 ms, as it predicts one step at a time.

All models were trained with the same computing resources to ensure a fair comparison.

These results demonstrate that, although neural operator approaches like MaNGO incur a higher memory footprint (due to storing temporal activations), they offer substantial improvements in inference speed, making them highly suitable for applications requiring fast rollout generation.

Once again, we thank the reviewer for their time and are happy to provide further clarifications if needed.