Improved Sampling Of Diffusion Models In Fluid Dynamics With Tweedie's Formula

W4 & Q3. We thank the reviewer for their suggestion regarding the stability for longer rollouts of our approaches. In addition to time-averaged metrics, in Fig. 7 (Appendix C), we show how each of the top-performing models from Table 2 correlate with the ground truth solution trajectory over time by calculating the Pearson correlation coefficient for the absolute velocity $\rho (|u|)$ at each timestep $r$. Also, in Fig. 2 (b), we report $\rho$ for the time evolution of the domain-wide kinetic energy, which describes the total kinetic energy of the system. Additionally, as requested, we include more analysis with regards to the temporal stability of our methods compared to traditional ones. We estimate the temporal stability for both $Tra_{long}$ (as defined in [2]) and $Fturb_{long}$ (combines both $ext$ and $int$ regions but with $R = 120$ instead of $R = 30$, more details will be included in an additional appendix for detailing all datasets) by calculating the rate of change of each flow field $x$ using $||(x_r - x_{r-1})/\Delta \tau||$ [2] and comparing against the ground truth simulation (GT). In summary, for $Tra_{long}$, our stability parameter remains bounded within the range $[0.015, 0.02]$ for both the ground truth (GT) and all models. While DDIM, DDPM, and IR slightly exceed the lower bound, they maintain overall stability. For $Fturb_{long}$, the stability parameter for the GT and all models is constrained within $[0.1, 0.15]$, with the exception of a minor deviation observed in DDPM. The corresponding Figure will be included in our revised paper. Contrary to $\rho (|u|)$, this parameter is useful for long rollouts as it helps identify whether a solution trajectory remains physical even after no longer being correlated to the GT. These additional results show that our proposed methods exhibit temporal stability for simulation trajectories significantly longer than the trajectories used for training.

[2]Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation. In: ICML (2024).

W5 - Lack of a comparison to EDM. As requested, we trained EDMs for three out of our four datasets, namely $Tra$, $Fturb$, and $Air_{One}$, and compare their performance against the top-performing models reported in Tables 1 and 2 (left). We use the same training hyperparameters (see Table 3) and network architecture as in Table 4. Our implementation of EDMs is based on the work by Karras et al. (2022). We implement their Algorithm 1 (deterministic sampler) and Algorithm 2 (Stochastic sampler) using 1st-order Euler and 2nd-order Heun's methods with design choices from the last column of Table 1 (Karras et al., 2022). We also consider preconditioning and a weighted loss function using the parameters recommended by the authors. For each case, we ran the model using different combination of deterministic/stochastic sampler and Euler's/Heun's method. For the stochastic sampler, the parameters $\{S_{churn},S_{tmin}, S_{tmax}, S_{noise} \}$ were non-comprehensively tuned to attain the best possible results. TA-MSE vs NFEs ($\in [1,200]$) curves for these various samplers will be included in our revised manuscript. In the tables below, we provide the updated Tables 1 and 2 (left) with the additional best EDM model. We summarize these finds:

• $Tra$ - EDM outperforms all probabilistic and deterministic models with 4 NFEs only.TSM achieves nearly comparable accuracy while remaining the sole model capable of single-step inference. Given the negligible accuracy difference from the best-performing model, we hypothesize that a more accurately pre-trained DDPM could enable IR to achieve more competitive results with low NFEs.
• $Fturb$ - EDM outperforms DDPM and DDIM in both accuracy and NFEs but achieves comparable performance to TSM, despite being $10\times$ slower. IR, however, remains the most accurate model for this dataset.
• $Air_{One}$ - EDM performs poorly in comparison to all other models. We believe the reason for this suboptimal performance is that the hyperparameters (including loss function weighting, scaling, and diffusion-related parameters) might not be optimal for this case and thus would require comprehensive tuning for improved results. Indeed, we believe that these author-recommended settings for EDMs are not expected to perform well in all applications (e.g., see [3]).
• In conclusion, EDMs are a great alternative to standard DDPMs, yet they don't provide significant improvements across different fluid dynamics problems compared to other baselines and our proposed approaches. We believe an interesting avenue for future research would be the combination of our proposed methods with EDMs to potentially enhance their performance and most importantly facilitate single-step inference.

We report these new findings in our updated manuscript.

[3]Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. In: ICML (2024).

We thank the reviewer for their thorough review and the invaluable feedback and suggestions. We address their comments and questions below.

W1 & Q1 - Generalization to other settings : We thank the reviewer for their insights and suggestions. Regarding $Air_{multi}$, we see that our models at least maintain the accuracy of the baseline DDPM and significantly reduce NFEs from 100 down to 41 and 10 for IR and TSM, respectively. Thus, we believe that our proposed approaches are promising for application to datasets with complex distributions.
Furthermore, we agree that evaluating our approaches on systems with more complex nonlinear dynamics, such as the Kuramoto-Sivashinsky (KS) equation, would provide valuable insights, especially for capturing high-frequency components. While our work does not directly include KS, we address similar challenges in the $Tra$ case, where shock waves introduce sharp, localized discontinuities that correspond to high-frequency energy in the spectral domain, and the $Fturb$ case, which "exhibits extreme events in the form of bursts in kinetic energy and dissipation" [1]. The success of our methods in capturing these features suggests their potential applicability to similarly complex PDEs like KS, yet we still believe it would be interesting to evaluate the performance gains on this dataset and compare against benchmark results. However, extending our approaches to KS or similar systems would require careful optimization of DDPM, IR, and TSM parameters, which is beyond the scope of this rebuttal. However, we appreciate the suggestion and would be happy to explore this in the final version.

[1] Clustering-Based Identification of Precursors of Extreme Events in Chaotic Systems. In International Conference on Computational Science (pp. 313-327). Cham: Springer Nature Switzerland. (2023)

W2 & Q2 - Counter-intuitive comparison to DDPM. While we agree with the reviewer that our improvements in terms of both speed and accuracy in some experiments might be surprising given the strong theoretical foundations for general DMs, our improvements arise from domain-specific adaptations. For this reason, we chose our experiments to exhibit different levels of difficulty and complex dynamics in both steady-state and transient scenarios to provide adequate empirical evidence for the efficacy of our approaches. Furthermore, we believe that our baseline DDPMs are competitive and well-tuned for our datasets. In fact, in our methodology, we first optimize the best possible baseline DDPM in comparison to (benchmark) deterministic baselines, then we tune for $s$ in TSMs and {$\gamma$, $x_{init}$} for IR sampling. Therefore, we are confident that our DDPM results are fair and representative. Additionally, we include a comparison against EDMs in our response below to W5.

W3 - Lack of theoretical guarantees. We agree that deriving theoretical guarantees is challenging. Further, we provide practical heuristics that guide hyperparameter selection effectively across datasets for IR and TSM.

• Deterministic test cases
  • IR: we usually start with $x_{init} \sim \mathcal{N}(0,I)$ and run our greedy optimization algorithm with low N (with $N = |\gamma|$) to obtain an efficient $\gamma$ with low NFEs. As demonstrated in Figure 4(b), $N=5$ often yield very good results for transient cases. $N$ can then be gradually increased to explore other schedules that might improve the accuracy with little increase in NFEs.
  • TSM: For higher speedup, the search for the optimum $s$ value typically begins within the range $[0.5, 1]$, especially if a large number of diffusion steps $T$ is chosen. We believe that Fig (3) provides several insights for the optimal combination of $s$ and $T$. We first test for both extremes of $s$ and then follow a standard line search approach to arrive at an optimum value for $s$, requiring additional 2 or 3 evaluations at most. We also give priority for models with low $T$ to minimize the NFEs required for inference.
• Stochastic test cases
  • IR: $x_{init}$ is optimal when obtained through truncated sampling of a pre-trained DDPM with $s >= 0.5$. The output from this approach is usually highly accurate but exhibits clear noise; thus it usually takes $N < 5$ for $\gamma$ to arrive at a clear output while improving or retaining the accuracy of the noisy input.
  • TSM: We follow the same procedure as before; however, we refrain from extreme $s$ values. Therefore, our search is restricted to $s \in [0.5, 0.9]$, or smaller lower bound for low $T$ values.

We will dedicate an additional section in our main text for an elaborate approach to optimizing the hyperparameters related to our proposed approaches. Moreover, we acknowledge that our experiments don't guarantee universal applicability; therefore, we will revise the generalizing statement to reflect the scope of our findings in the updated version of our paper.

We thank the reviewer for their comments and we address their concerns below.

Superiority of the methods over other surrogates, including NOs. We thank the reviewer for their comment on benchmarking against neural operators. As shown in Tables 6, 7, and 8, we systematically compared our methods with various deep learning baselines, including neural operators (namely FNOs) and UNets. Notably, our transient cases feature periodic boundary conditions, a scenario where FNOs are expected to excel due to their inherent architecture. In Table 6, we outline how our methods clearly outperform these baselines. Since we find from the $Tra$ case that FNOs yield suboptimal results, as corroborated by [1-3], we don't consider them for subsequent experiments. Further, our primary objective is to enhance the performance of DDPMs to reduce the gap between DMs and deterministic baselines; therefore, our methods don't always outperform UNets, especially when trained with advanced learning techniques. However, we demonstrated that TSM surpasses the best UNets (Table 1 left) while IR outperforms the baseline UNet (Table 1 right). In both cases, TSMs and IR surpass DDPMs in terms of speed and accuracy.

[1] Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation. In: ICML (2024).

[2] Learned Simulators for Turbulence. In: ICLR (2022).

[3] PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers. In: NeurIPS (2023).

Comment regarding the title. We agree that the title may imply broader applicability than what is covered in our experiments. To address this, we will revise the title to emphasize our focus on fluid dynamics.

Effectiveness of the pre-trained DM in IR. We thank the reviewer for raising an interesting point and we assume that they mean $t = \gamma_{i,k}$, where $k$ denotes the $k$-th element in a refinement schedule $\gamma_i$. The effectiveness of a pre-trained DM in IR sampling at any noise level $t$ lies in its ability to approximate the posterior mean $\mathbb{E}[\hat{x_0}| x_t]$ using Tweedie's formula, which directly relates to the likelihood estimation. In addition, we agree that in some cases the distribution $q(x_{init}|x_t)$ might not match the distribution $q(x_{0}|x_t)$ when using a pre-defined forward diffusion ($q(x_l|x_t)$ where $t > l$). Consequently, this places a limitation for the choice of $x_{init}$ to ensure that the forward process posterior estimation is optimal at any noise level $t$.

Comment regarding Eq. (6). Eq. (6) is an idealization of a refinement schedule $\gamma$ for which each step supersedes the accuracy of the previous step and by extension all the preceding ones. This is not a hard requirement for IR sampling, but rather a consequence of the greedy optimization algorithm we employ for $\gamma$. While this doesn't guarantee that for the last refinement step the error will be smaller than a certain threshold $\epsilon$ as this depends on several factors, it is possible to provide a convergence guarantee. Our IR method is closely related to DDIM, and thus can be interpreted via ODE integrators and guaranteed to converge given an optimized refinement schedule $\gamma$. In fact, the recursive sampling formula for IR, as presented in Algorithm 2, can be recovered from the generalized generative process (see Eq. 7) that supports both Markovian and non-Markovian inference processes by using $\sigma_t = \sqrt{1-\bar{\alpha}_{t-1}}$. This essentially means that IR sampling is a special generative process similar to DDIMs (when $\sigma_t = 0$) and DDPMs, which can be recovered with a proper choice of $\sigma_t$. Thus, IR belongs to the non-Markovian family of generative processes, to which DDIM belongs.

Data consistency in IR. We appreciate the reviewer’s suggestion regarding data consistency to improve the accuracy of IR sampling. Although it is possible to apply data consistency after line 7 in Algorithm 2, our experiments demonstrate that the proposed IR method achieves sufficient accuracy without requiring this additional step, thereby maintaining low computational cost. Enforcing data consistency with respect to $\hat{x}_0$ would require ensuring that $\hat{x}_0$ is a physically meaningful, non-noisy prediction of the flow field, which is exclusively achieved for noise steps close to 0 [4]. Additionally, to enforce data consistency, an auxiliary network or an expensive calculation of the governing PDE residual through high-order derivative approximations would be required as detailed in [5]. Thus, by not requiring data consistency in IR, the algorithm gains flexibility in selecting the noise steps for sampling (i.e., choosing the refinement schedule $\gamma$) while still allowing for future extensions that incorporate data consistency, if desired.

[4] Freedom: Training-free energy-guided conditional diffusion model. 2023.

[5] A physics-informed diffusion model for high-fidelity flow field reconstruction. 2023.

Dear Reviewer,

Thank you for your earlier comments and insights.

We value your feedback greatly and want to ensure that our responses have addressed your concerns. To this end, we invite you to review the updated version of our paper, which includes new baselines, additional analysis, and improvements based on other reviewers' suggestions. If you have any further questions or additional feedback, we would be happy to address them.

We sincerely appreciate your time and valuable feedback in helping us refine our work.

We would like to thank the reviewer for their thorough feedback and comments regarding our work, especially for their appreciation of the experimental setup and our approaches for improved sampling.

W1 & Q1 (comparison to PDE-Refiner): We appreciate the reviewer’s comment and agree that explicitly contrasting IR with PDE-Refiner would strengthen the clarity of our method's novelty. Since PDE-Refiner shares a lot of similarities with DDPMs in general, it is expected to share key similarities with other models/samplers including, DDIM, EDM, and IR. We summarize the main differences between IR and PDE-Refiner in the following points:

• IR is a sampling algorithm that works with pre-trained DDPMs, while PDE-Refiner requires training from scratch for a fixed schedule and number of refinement steps, similar to DDPMs.
• PDE-Refiner predicts the state at the initial step, while IR focuses on predicting the noise throughout the entire refinement process, similar to ancestral sampling.
• The combination of a flexible $\gamma$ and $x_{init}$ sets IR sampling apart from PDE-Refiner as it can be treated as a standalone sampling algorithm when $x_{init} \sim \mathcal{N}(0,I)$, but can also be used to refine a noisy or a low-fidelity state.

Furthermore, we think that the main reason behind the poor performance by PDE-Refiner in the $Tra$ case is that it was found to be highly sensitive to its key hyperparameters (number of refinement steps and the minimum noise variance) as reported in Kohl et al. (2024). This makes an efficient hyperparameter tuning for this method difficult, leading to suboptimal results.

Moreover, while the refinement schedule $\gamma$ is one of the key novelties of IR, the greedy optimization algorithm is an established optimization approach; thus, we believe that the details regarding this algorithm are secondary to the main contributions which we include in the main text.

W2: We thank the reviewer for their suggestion. While we do include a comprehensive list of baselines in our full set of results in Tables 6, 7, and 8, we will make sure to include the top-performing of these baselines in our Figures as well for the updated version of our paper.

W3: We thank the reviewer for pointing out a similar approach adopted in Gupta et al. (2022) regarding our reformulated autoregressive problem. While it is true that conditioning on the prediction stride $j$ has been previously explored, the focus in the referenced study lies in evaluating models conditioned on multiple parameters, including $j$, across datasets. In contrast, our contribution emphasizes how this conditioning enables flexible sampling that supports parallelization and enables the prediction of intermediate states without compromising accuracy compared to next-step surrogates. Therefore, we will discuss the work of Gupta et al. (2022) in our revised manuscript and delineate the key differences between our objective and theirs.

Q2: We appreciate the reviewer’s perspective on reinterpreting the TSM algorithm as a modified noise schedule with equivalent noise steps. For diffusion models in general, optimizing the noise schedule is a critical task that shall be tuned on a case-by-case basis whether through fixed schedules (e.g., linear, cosine, or sigmoid) [1] or learned ones [2]. While the proposed equivalence between truncation and a modified noise schedule might hold conceptually, in our experiments, we reported that noise schedules with large steps often produce noisy outputs, as shown in Tables 6 and 7 for the "DDPM T20" model. Therefore, adopting a custom schedule with the same noise steps as in TSMs without using Tweedie's formula for the last step at low SNR instead of an iterative refinement step is highly anticipated to result in suboptimal, noisy results. We believe that the ability to take a significant step from a low SNR state directly to a clean sample is facilitated by Tweedie's formula as it estimates the posterior mean rather than following the solution trajectory of the probability flow ODE/SDE. While this approach aligns conceptually with methods that use Tweedie's formula for final denoising (e.g., [3]), the novelty in our work lies in applying Tweedie's formula to significantly noisier states, enabling efficient sampling while maintaining accuracy.

[1] Ting Chen. On the Importance of Noise Scheduling for Diffusion Models. 2023.

[2] Kingma et al. Variational Diffusion Models. NeurIPS 2021.

[3] Score-Based Generative Modeling through Stochastic Differential Equations. In: ICLR (2021).

We thank the reviewer for their comments and for highlighting the main advantages of our approaches in tackling efficient sampling with high accuracy and stability.

Mathematical interpretation of TSMs. We appreciate the author's comment regarding the theoretical roots of TSMs. Truncation at low $t$ (i.e., using $s<0.2$) even for pre-trained DDPMs mitigates computational overhead without accuracy loss because the retained steps sufficiently capture the large scale and fine structures of the flow fields. At high $t$ (i.e., using $s\gg0.2$), an accurate approximation for the posterior mean $\mathbb{E}[\hat{x}_0|x_t;\theta]$ is only achievable by a model well-trained to approximate the score function (i.e., the gradient of the log-likelihood) for all $t \in [s\cdot T, T]$ and a target data distribution that permits a relatively simple reverse process, enabling high $s$ values. In fluid dynamics, the target distributions are often unimodal, which reduces the risk of bias introduced by Tweedie's formula, since the dominant mode of the distribution is captured with a relatively coarse discretization of the reverse-time SDE. This property supports truncation as it simplifies the sampling process without compromising the ability to accurately represent the underlying physics.

Achieving a balance between speed and accuracy in IR/TSM. Thank you for raising this important point. Indeed, achieving a balance between speed and accuracy in IR sampling and TSMs requires careful tuning of the associated hyperparameters. These are dataset- and task-dependent parameters that need to be optimized on a case-by-case basis to serve the specific dynamics of the problem. Regarding the efficient optimization of these hyperparameters, we provide a heuristic approach supported by our results:

• Deterministic test cases
  • IR: we start with $x_{init} \sim \mathcal{N}(0,I)$ and run our greedy optimization algorithm with low N (with $N = |\gamma|$) to obtain an efficient $\gamma$ with low NFEs. As in Figure 4(b), $N=5$ often yield very good results for transient cases. $N$ can then be gradually increased to explore other schedules that could potentially enhance the accuracy with mild increase in NFEs.
  • TSM: For higher speedup, the search for the optimum $s$ value typically begins within the range $[0.5, 1]$, especially if a large number of diffusion steps $T$ is chosen. We believe that Fig (3) provides several insights for the optimal combination of $s$ and $T$. We first test for both extremes of $s$ and then follow a standard line search approach to arrive at an optimum value for $s$, requiring additional 2 or 3 evaluations at most. We also give priority for models with low $T$ to minimize the NFEs required for inference.
• Stochastic test cases
  • IR: $x_{init}$ is optimal when obtained through truncated sampling of a pre-trained DDPM with $s >= 0.5$. The output from this approach is usually highly accurate but exhibits clear noise; thus it usually takes $N < 5$ for $\gamma$ to arrive at a clear output while improving or retaining the accuracy of the noisy input.
  • TSM: We follow the same procedure as before; however, we refrain from extreme $s$ values. Therefore, our search is restricted to $s \in [0.5, 0.9]$, or smaller lower bound for low $T$ values.

While a rigorous closed-form mathematical framework remains challenging, our heuristic approach was found to provide adequate results for our experiments, minimizing typical exhaustive parameter search. We will expand on this in the revised paper to improve clarity.

Increasing breadth of experiments. We appreciate the reviewer’s suggestion to broaden our experimental scope. We have included an additional comparison against EDMs for 3 of our experiments. In summary, across different fluid dynamics problems, EDMs demonstrate varying performance. On the $Tra$ dataset, EDM achieves the best accuracy with only 4 NFEs, surpassing all probabilistic and deterministic models. TSM achieves nearly comparable accuracy while remaining the sole model capable of single-step inference. On $Fturb$, EDM outperforms DDPM and DDIM in both accuracy and NFEs but achieves comparable performance to TSM, despite being $10\times$ slower, whereas IR remains the most accurate model for this dataset. However, on $Air_{One}$, EDM underperforms compared to all other models, likely due to suboptimal hyperparameter settings, including loss weighting, scaling, and diffusion parameters, which may require extensive tuning. Please see our response to reviewer 6cX6 for additional details on the EDM comparisons.

Regarding additional PDE solutions, while we believe that the current experiments cover diverse and intricate fluid dynamics phenomena, including both deterministic and stochastic scenarios, we acknowledge the value of testing on broader PDE settings. We’d be happy to include an additional experiment on the 1D Kuramoto-Sivashinsky problem in the camera-ready version of our paper.

Dear Reviewer,

Thank you for your earlier comments and insights.

We value your feedback greatly and want to ensure that our responses have addressed your concerns. To this end, we invite you to review the updated version of our paper, which includes new baselines, additional analysis, and improvements based on other reviewers' suggestions. If you have any further questions or additional feedback, we would be happy to address them.

We sincerely appreciate your time and valuable feedback in helping us refine our work.

We thank the reviewer for their thoughtful comments and their interest in our approaches to reduce computational costs. We address their concerns below.

Use of Tweedie's formula and similarity to [Delbracio et Milanfar, 2024]. We thank the reviewer for highlighting the connection between our proposed approaches, leveraging Tweedie's formula, with [Delbracio et Milanfar, 2024]. As mentioned, the models are fundamentally different, yet the aforementioned work can still be linked to DDPMs when the low-quality, degraded state is pure Gaussian noise. Their main reason to add scaled white noise in every step of their inference algorithm is to convert a deterministic algorithm to a stochastic one that is capable of exploring multiple possible explanations for a degraded sample and thus lead to better perceptual quality, though their approach was shown to not be beneficial in all cases. Since both our approaches are inherently stochastic (i.e., fresh white noise is injected in every step), the potential gains from injecting further white noise to the predicted posterior mean might have little effect. Alternatively, we can enforce data consistency after applying Tweedie's formula to improve the overall accuracy of the methods similar to resampling methods (e.g., [1] and [2]). However, this approach comes with its own limitations as we described in the paragraph starting at line 284.

[1] Solving inverse problems with latent diffusion models via hard data consistency. In ICLR (2024).

[2] Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing. In: CoRR (2024).

Superior performance of TSM with $s=1$. We would like to clarify that single-step TSMs (i.e., using $s=1$) don't always outperform ancestral sampling. $s$ is a hyperparameter that is to be tuned on a case-by-case basis and it is not always guaranteed that extreme $s$ values will always lead to the best results as they can significantly impact the stochasticity of the method. Since complex datasets generally benefit from stochastic samplers [3], we see that $s = 1$ is only possible on the $Tra$ (see Table 6) case while the optimum results for the cases $Fturb$ and $Air$ are obtained using $s = 0.75$ (see Table 7) and $s = 0.6$ (see Table 8), respectively. This is due to the increasing level of difficulty in these test cases, requiring more steps of the reverse Markov chain. Therefore, the trade-off between accuracy and stochasticity would limit the practicality of perpetually using $s = 1$. The observed results are specific to the nature of fluid dynamics simulations and thus, a generalization is only possible when considering the features of the data distribution (as we explain in the paragraph beginning at line 406).

[3] Karras et al. “Elucidating the Design Space of Diffusion-Based Generative Models”. In: NeurIPS (2022).

TSM performance in image reconstruction. We appreciate the reviewer’s interest in benchmarking our approaches against traditional DDPM sampling on image reconstruction. However, our focus on fluid dynamics problems is intentional due to the unique properties of the data. Physics-based simulations involve deterministic and continuous systems governed by PDEs, which differ significantly from the more complex, often multimodal distributions encountered in generative modeling tasks. Our methods are tailored to exploit these characteristics present in our datasets to support fast and accurate sampling of DMs. Hence, comparing TSMs/IR against ancestral sampling for image reconstruction is beyond the scope of our research.

Models performance on noisy dataset. Our primary objective from this study is to evaluate the effectiveness of our proposed methods in capturing the underlying dynamics of fluid systems without introducing additional complexities. We believe that handling noisy datasets is an independent, challenging task, potentially requiring methodological modifications to the training or the denoising algorithms. For example, existing studies are dedicated to exploring analogous challenges, including noisy [4] or sparse [5] observations. One simple approach is to include the conditional information (i.e., the previous noisy state) in the iterative refinement process similar to [6], but the efficacy of this method in reducing the impact of added noise and its ability to recover a noise-free observation remains uncertain. Thus, we believe that out-of-the-box diffusion models are not expected to excel on noisy datasets without careful algorithmic modifications, which is an interesting topic for future work.

[4] Risk-Sensitive Diffusion: Robustly Optimizing Diffusion Models with Noisy Samples. 2024.

[5] DiffusionPDE: Generative PDE-Solving Under Partial Observation. 2024.

[6] Kohl et al. Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation. In: ICML (2024).