Arbitrarily-Conditioned Multi-Functional Diffusion for Multi-Physics Emulation

We thank the reviewer for their insightful comments and valuable suggestions.

It would be interesting to see how uncertainty intervals or estimates obtained from this method compare to those obtained from other capable methods, such as simformer.

R1: Thank you for the great suggestion — we completely agree. We have added the evaluation and analysis of uncertainty quantification. Please see our response R1 to Reviewer bmeB for details.

The most important work in this domain is perhaps [1], with others including [2,3]. These methods need not be compared with experimentally, but it would be interesting to discuss the advantages and differences between these approaches

R2: Thank you very much for providing these excellent references! We will certainly cite and discuss them.

The key difference lies in the training and inference paradigms. The referenced works [1–3] focus on training an unconditional diffusion model, and only at inference time do they incorporate the likelihood of observations or measurements to guide sampling from the conditional distribution. The advantage of this approach is that it only requires a standard diffusion model --- potentially even a pre-trained one --- without the need to address conditional sampling tasks during training. Indeed, our baseline MFD-Inpaint (see Section 5.2) follows this paradigm.

However, the referenced methods may have limitations when applied to our scenarios:[1] requires a closed-form expression of the likelihood or forward model $p(y|x_0)$, or at least an efficient way to compute its gradient (or sensitivity) --- which is often infeasible in multi-physics systems governed by complex differential equations. The work [2] further restricts the applicability by assuming that the forward model is linear. [3] is more flexible and can handle nonlinear forward models, but it requires a "pseudo-inverse" operator for the forward process, which might not be easy to obtain.

Our approach takes a different route: during training, we explicitly incorporate conditionality by randomly sampling masks and training our diffusion model to handle a wide range of conditional sampling tasks (including the unconditional case). At inference time, we do not incorporate any likelihood scores --- instead, we directly sample from our learned denoising model. Empirically, we found that this conditional training strategy yields large improvements over unconditional training approaches like MFD-Inpaint across a variety of tasks (see Table 3 and our response R1 to Reviewer BtjJ).

The arbitrary choice of h being Bernoulli-distributed with p=0.5 is not justified... Have you experimented with varying p through training?

R3: Thank you for the insightful question. When there is no prior knowledge about whether a function value should be conditioned on or be generated, we suggested using $p = 0.5$ to avoid bias --- giving each component an equal chance of being conditioned or sampled. This is actually a uniform prior. We do agree, the mean of the proportion of the masked components is 0.5; but the standard deviation is also 0.5 ($\sqrt{p\times(1-p)}=0.5)$, which is the maximum possible, indicating that the actual masked proportion can vary widely. In our experiments, we used $p = 0.5$ to maintain a neutral setting. However, our method places no restrictions on the choice of $p$; it can be adjusted based on domain knowledge or specific application needs.

We have now included results for alternative values of $p$: 0.2, 0.4, 0.6, and 0.8. As shown below, the performance for $p=0.4$ and $p=0.6$ is comparable to that of $p=0.5$, demonstrating a certain degree of robustness to the choice of $p$. However, when $p$ is set to more extreme values, such as 0.2 or 0.8, the performance degrades markedly.

We will add these results and discussions into our paper.

Relative $L_2$ error

“We used FNO to construct our denoising network” What does this mean? You share the same architecture as FNO?

R4: Great question. Yes, we use the same architecture as FNO - specifically, a sequence of Fourier layers — to construct our denoising network. We will make this point clearer in the paper.

We thank the reviewer for the many constructive comments.

Consider adding ablation studies to demonstrate importance of each component (random masks, Kronecker structure)

R1: Great suggestion. Actually, we have already conducted studies on the effect of random masks. In Section 5.3, we compared our method (ACM-FD) with a variant trained without random masks (denoted as MFD-Inpaint) on the function completion task. As shown in Table 3, incorporating random masks during training significantly reduces the relative $L_2$ error --- by as much as 58% to 96%.

Here, we additionally include results for prediction tasks, presented below. Across various prediction settings, our model trained with random masks consistently outperforms the variant without them (denoted as MFD), achieving substantial error reductions ranging from 40.9% to 96.2%.

Relative $L_2$ error

We have also supplemented the ablation study on the use of the Kronecker product structure. Please see R2 to Reviewer bme8 for details.

discuss [1,2]

R2: Thank you for the excellent references. We will definitely cite and discuss them. A key distinction is that the masks used in [1,2] are fixed and derived from the sampling patterns inherent to the medical imaging data. As a result, these works focus on a single inverse task --- either recovering high-frequency regions [1] or unsampled measurements [2]. In contrast, our method repeatedly samples random masks during training, enabling it to jointly handle a wide variety of conditional sampling tasks, encompassing all kinds of forward prediction, inverse prediction, and completion tasks.

Another critical difference lies in the functional space formulation of our model: the noises used during both training and inference are stochastic functions sampled from Gaussian processes. Our method is designed to generate functions either unconditionally or conditioned on other function samples. This motivation leads to fundamentally different designs in model architecture, training loss, and computational techniques (e.g., the use of Kronecker product).

Add more details about the training process and hyperparameter selection

R3: Thank you for the great suggestion. We actually have provided detailed information in Appendix Section B, including the set of hyperparameters, their ranges, implementation details and libraries used for each method, as well as the validation settings. Due to space constraints, only a subset of this information is included in the main paper, with a reference to the appendix (see Line 366). In response to your feedback, we will further expand and enrich Appendix Section B to improve clarity and completeness.

Add statistical significance tests for the performance comparisons

R4: Thank you for the excellent suggestion. Based on a $z$-test conducted on the prediction errors with associated error bars, our approach outperforms the baseline methods in the vast majority of cases at the 95% significance level. We will include the results in our paper.

Include more details about the training data generation process

R5: We actually have included these details in Appendix Section A, which provides the governing equations, numerical simulation library used, simulation procedures, and data collection process. We believe this information is sufficient to generate all the data for our experimental settings. Additionally, we will publicly release our experimental data.

Provide visualization of the generated functions and their uncertainty estimates

R6: Thank you for the great suggestion. We have added the suggested visualizations and analysis --- please see our response R1 to Reviewer bmeB for details.

Lack of comparison with traditional numerical methods

R7: In fact, we did compare with traditional numerical methods, as all our test data (as well as the training data) are generated using them. We consider the outputs of the traditional methods as the gold standard, and our goal is to train a surrogate model that closely approximates their results. The relative $L_2$ error reported in Table 1-3 quantifies the difference of our method's prediction as compared to the traditional methods.

We sincerely thank the reviewer for the thoughtful and constructive comments.

no analysis of UQ performance.

R1: Great suggestion. Here we add UQ performance evaluation & analysis results.

First, to evaluate UQ quality, we computed the emprical coverage probability [1]: $CP = \frac{1}{N} \sum_{i=1}^N I(y_i \in C_\alpha)$ where $y_i$ is the ground-truth function value, and $C_\alpha$ is the $\alpha$ confidence interval derived from 100 predictive samples generated by our method. We varied $\alpha$ from {90%, 95%, 99%}, and examined our method in eight prediction tasks across Convection-Diffusion (C-D) and Darcy Flow (D-F) systems. We compared against Simformer. Note that the other baselines are deterministic and unable to perform UQ.

C-D

D-F

In most cases our method achieves coverage much closer to $\alpha$, showing superior quality in the estimated confidence intervals.

Next, we visualized prediction examples along with their uncertainties, measured by predictive standard deviation (std). As shown here, more accurate predictions tend to have lower std, i.e., low uncertainty, while regions with larger prediction errors correspond to higher std — providing further qualitative evidence that our uncertainty estimates are well-aligned with prediction quality. We will include the results in our paper.

[1] Dodge, Y. (Ed.). (2003). The Oxford dictionary of statistical terms. Oxford University Press.

lacks a training-time comparison against baseline models.

R2: Excellent suggestion. We have added an ablation study to assess the impact of incorporating Kronecker product in our method, and compared both the training and inference time with other methods. For a fair and comprehensive evaluation, we first examined the per-epoch training time. For neural operator baselines, we measured their total per-epoch time across all the tasks.

Training time (per-epoch) in seconds

Training time (total) in hours

The results show that leveraging Kronecker product properties largely improves the training efficiency of our method. Our per-epoch training time is much less than all the competing methods except DON, which achieves exceptionally fast training by using PCA bases as the trunk net. However, diffusion models typically require far more training epochs than deterministic neural operators. For instance, FNO and GNOT converge within 1K epochs across all the cases, while our method typically needs around 20K epochs. Consequently, despite per-epoch efficiency gains, our method, as well as Simfomer --- another diffusion-based method --- still takes longer overall training time.

Inference time (per-sample) in seconds

Lastly, by using Kronecker product, our method achieves substantial acceleration in generation, with a 6.7x speed-up. During inference, the computational cost is dominated by sampling noise functions, whereas during training, a substantial portion of cost also arises from gradient computation. Consequently, the runtime advantage of using the Kronecker product is even more pronounced during inference. We will include these results and discussions in the paper.