*Rebuttal

1. larger value for N in SD3?

Great question! We did conduct this experiment with SD3 and observed that using N = 4 resulted in even faster convergence. However, unlike the N = 2 case, we were unable to find a configuration that consistently outperformed the baseline in EDM2, as N = 2 did. To maintain consistency in presentation, we chose to omit the N = 4 results. Our objective is to provide a robust sampling method that performs reliably without requiring extensive tuning. Since you brought it up, here is the list of results for SD3 with N = 4 and it performs better for CFG > 1 case:

We are happy to present this in the appendix as well.

2. Lossless approximation for coefficients

Yes. Everything is exact and has a closed-form solution. The only approximation is for $x_{\theta}$ (data) prediction which is using a Neural Network as is classical for diffusion.

3. Estimation error for F in larger scale dataset

There will be no approximation error for $F_{\theta}$. See above clarification.

4. Welcome to join discussion

We further provide more interesting experiment results and analysis during the discussion with other reviewers. Feel free to have a look if you are interested in!

Rebuttal

We thank the reviewer for thorough comments on both the main paper and the appendix. Below are our clarifications.

1. Computation Complexity and memory complexity

both computation and memory complexity overhead are marginal.

Computation complexity

1. All computations depend solely on time and can be precomputed after discretizing the time horizon, prior to sampling.
2. Although operations involving $\mathbf{\mu}$ and $\mathbf{\Sigma}$ may look costly due to the Kronecker product with identity, the effective dimension is $N$ (the number of variables). Thus, we only need to store and compute with $N$-dimensional matrices/vectors.
3. The major computation, which is the Neural network, has the same input and output dimension, which is identical as baselines.

Overall, the computational overhead arises primarily in points 1 and 2, and both are theoretically and empirically negligible. We further validate this experimentally by measuing throughput speed on ImageNet with 10 NFE. As the network size increases the performance gap further narrows:

memory complexity

And here is practical memory usage for Imagenet 512 w/ Batch=64:

2. Missing ablation study for TADA

Here are ablation experiments, (we skipped order=3 since it has been reported in the paper)

3. Better Quality like SciRE-Solver

Yes. see right side of Fig.5.

4. Mechanistic understanding? Cifar10 experiments?

We address both questions jointly because the CIFAR-10 experiments partially verify the proposed mechanism.

On small-scale data such as CIFAR-10, once a high-capacity model is well trained (e.g., EDM), ODE sampling performs better than SDE sampling, regardless of the amount of injected noise (see Fig. 5b in the EDM paper). By contrast, in large-scale settings (more data and higher dimensionality) such as ImageNet,where training an equally strong model is harder, SDE sampling tends to outperform ODE sampling (see Fig. 5b–c in the EDM paper and Table 1 in the DDIM paper) when NFE is sufficient.

Our hypothesis is that TADA does not provide a fast-sampling mechanism. Instead, it leverages stochasticity to compensate for propagation inaccuracies just like SDE, though it is pseudo-noise/stochaticity. Since SDEs are often slow due to their sensitivity to time discretization, the pseudo-noise in TADA offers a practical middle ground: retaining SDE-like behavior while enabling low NFE sampling through an ODE solver, thanks to pseudo-noise.

This aligns with our experiments: TADA consistently outperforms baselines on large-scale ImageNet and SD3, and as expected, underperforms ODE sampling on CIFAR-10 with the EDM checkpoint. The results below illustrate this pattern.

This further supports our recommendation in Sec.5 to apply TADA to larger models, rather than focusing on small-scale ones.

We can highlight this property in the paper to provide more intuition!

5. 186% Faster claim

It is computed as the number of NFE used to achieve FID=2 in imagenet512.

TADA converges to FID=2 at NFE=12.

As Figure 6 shows, the other solvers never dip below FID 2 within 35 NFEs. To dig deeper, we ran a full ablation and found one UniPC setup (row 7 in the EDM2_unipc_ablation file) hits FID 2 at roughly 23 NFEs, about a 186 percent improvement.

However, This configuration falls well behind the paper’s version when the NFE budget is small (NFE in the range 5 to 20, see rows 2–6 in EDM2_unipc_abaltion file).

In short, we searched every UniPC variant and picked the one that reaches FID 2.0 with the fewest NFEs. We hope that is the fairest way to compare.

6. Why change title?

We found this title sounds better :)

7 Summary

We hope this rebuttal can more or less address your concerns. Please feel free to raise any additional questions! We view the rebuttal process as a valuable opportunity to improve the quality of the paper and to validate that our ideas and results are meaningful to the broader community.

Also, If all concerns are resolved, we would deeply appreciate a reconsideration of the overall rating :)

Thank you for your rebuttal. However, my core concerns remain unresolved. Your claimed "nearly 186%" speedup is misleading. Your own timing shows that TADA is actually about 12% slower per step, and the alleged acceleration comes from cherry-picking a specific UniPC configuration that requires 23 NFEs at FID=2 but performs poorly under a typical NFE budget.

I am also confused by the inconsistency in the experimental results provided in the paper and the rebuttal, which show acceleration on some datasets but not on others (e.g., CIFAR-10). If the method possesses "SDE-like properties," a more fundamental question is: why is it able to accelerate faster than an ODE solver?

While you claim that TADA achieves "SDE-like properties" through pseudo-noise (Proposition 3.3), the paper never compares it against actual SDE solvers, which is the most relevant benchmark to validate this claim. If TADA truly had the advantages of "SDE-like properties," it should achieve a quality improvement similar to that demonstrated in SEEDS [1], which achieved significant performance gains using a true SDE solver on the EDM model.

Furthermore, the paper lacks an ablation study across different time schedules (e.g., EDM, logSNR, time-uniform) and also lacks validation on classic SDE-based score-based DMs. This further weakens the reliability of its claim to be a general solution.

[1] Gonzalez, Martin, et al. "Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models." NeurIPS 2023.

Accordingly, these concerns make it difficult for me to improve my assessment for the current manuscript.

Thank you for the response. However, we feel there is some misunderstanding and you are ignoring some points in the paper and in the rebuttal. We are going to number our responses to ease the discussion.

Your claimed "nearly 186%" speedup is misleading.

R1: First we did not claim "nearly 186%", we said "up to 186%" in the abstract which is the best case, and this is done with ImageNet512.

Your own timing shows that TADA is actually about 12% slower per step,

R2: You quoted worst case 12% for ImageNet64. For ImageNet512, it is only 3.4% slower. You can notice that, as the scale increases, the gap decreases which is a very nice property of our method.

and the alleged acceleration comes from cherry-picking a specific UniPC configuration that requires 23 NFEs at FID=2 but performs poorly under a typical NFE budget.

R3: We did not search configurations for our method to artificially boost our results. We selected 2 as the FID since this is the point to which all solvers in this comparison converge asymptotically. Consequently, we would like to understand what do you mean by cherry-picking? Please clarify.

show acceleration on some datasets but not on others (e.g., CIFAR-10).

R4: We already answered this question in part 4 of the previous rebuttal. Let us reiterate: Cifar10 does not benefit from SDE solvers, as already demonstrated by the previous paper (EDM). Therefore, our results simply confirm this known fact. Did you miss this section in the previous rebuttal? Or did you misunderstand it? It is unclear from your feedback which it is.

If the method possesses "SDE-like properties," a more fundamental question is: why is it able to accelerate faster than an ODE solver?

R5: It is exactly the point and the novelty of our paper: by increasing the dimension of the noise, an ODE solver can produce the level of diversity associated with SDE.

While you claim that TADA achieves "SDE-like properties" through pseudo-noise (Proposition 3.3), the paper never compares it against actual SDE solvers, which is the most relevant benchmark to validate this claim.

R6: This is incorrect. Figure.7 and Section.4.2 show comparisons with the SDE solver, namely SA-Solver. For clarity, we even referred to SA-SDE in Figure.7.

If TADA truly had the advantages of "SDE-like properties," it should achieve a quality improvement similar to that demonstrated in SEEDS [1], which achieved significant performance gains using a true SDE solver on the EDM model.

R7: Actually, TADA drastically outperforms SEEDS: for Cifar10, TADA is 600% faster than SEEDS for the same FID (FID=2.04 in 21 NFE for TADA while FID=2.08 in 129 NFE for SEEDS). Furthermore, and this is an additional information that you did not request, it should be noted SEEDS does not even outperform the UniPC (ODE) baseline in our paper, and this is actually aligned with our analysis in part 4 of the rebuttal: The SDE Solvers cannot outperform ODE solvers for small scale case.

Furthermore, the paper lacks an ablation study across different time schedules (e.g., EDM, logSNR, time-uniform) and also lacks validation on classic SDE-based score-based DMs. This further weakens the reliability of its claim to be a general solution.

R8: This is incorrect. We stated in the paper that we selected the best configuration for baselines and we also provide comprehensive experimental results in the supplementary materials. For TADA, we made a conservative choice by selecting the most common schedule (quadratic), this demonstrates the robustness of our method since we did not have to resort to the schedule exploration to artificially boost our results. That said: we could potentially further improve our results by carefully designing the schedules. However, our results are already good, we decided to leave such explorations for future work.

lacks validation on classic SDE-based score-based DMs

R9: What are the classic SDE-based score-based DMs? If the reviewer considers neither EDM, EDM2, nor Flow Matching to be “classic SDE-based score-based DMs,” may I respectfully ask what is meant by “classic” and what defines an “SDE-based DM”?

Accordingly, these concerns make it difficult for me to improve my assessment for the current manuscript.

R10: We hope we addressed your concerns. We noted that several of them stem from misunderstandings of points that were already addressed in the previous rebuttal and in the paper. As this was not acknowledged in your latest response, we kindly encourage you to have another look at them.

We remain available for any questions you may have, and we thank you for your continuous involvement in the rebuttal process.

[1] Gonzalez, Martin, et al. "Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models." NeurIPS 2023.

Thank you for taking the time to review our paper and rebuttal. Could you please confirm whether we addressed your concerns or not? We still have two more days for discussion if needed.

Rebuttal

We thank the reviewer for thorough comments on both the main paper and the appendix. Below are our clarifications.

1. Lines 30–33: Statement is overly strong

Agree. This should be the conditional statement.We will revise it to:

“... ODE solvers often yield lower‐fidelity results than SDE solvers as evidenced by the FID scores in [17] when sufficient function evaluations are performed and model capacity is a limiting factor.”

2. Shape mismatches in Appendix A

The shapes are correct because we abused the notation (sorry!) for all coefficient functions of $t$ by implicitly expanding them with the Kronecker product for simplicity. This is stated at the first line of the appendix. However we now recognize this is easy to miss and can lead to confusion. To remedy this, we will superscript all such variables with $\otimes$, e.g. $\mu_t^\otimes=\mu_t\otimes I_d$. Does this address your concerns?

3. Redundancy of the two state equations

This redundancy questions the need for an auxiliary variable during training. Our experiments, which sample from the momentum system using the pretrained diffusion model, confirm this observation, which means the auxiliary variable is indeed redundant during training.

Since the auxiliary variable truly offers no benefit during training, the next question is whether the momentum system improves sampling. Should the two formulations (momentum sampling vs vanilla diffusion sampling) remain mathematically identical, there would indeed be no justification,per the reviewer’s observation,for introducing an auxiliary variable.

Yet, as Proposition 3.3 shows, the momentum system solves an ODE with SDE-like diversity, which can potentially enhance sampling quality (i.e see Fig. 5b–c in the EDM paper and Table 1 in the DDIM paper). And we further verify it in Proposition 3.3 and the experiment section.

To summarize, we agree with the reviewer that introducing an auxiliary variable yields no benefit during training, as already advocated in the paper. However, this makes a meaningful difference during sampling.

We will emphasize this again in the introduction for clarity.

4. Exact expression for $\mathbf{r}_t$, $\mathbf{\mu}_t$ and $\mathbf{\Sigma}_t$

All closed-form expressions are provided in Appendix E.2. Here, we walk through them to aid the reviewer’s understanding and ensure clarity:

1. Given number of variable $N$, user can simply compute the $\mu^{(i)}_t$.

2. $\mathbf{\Sigma}_t$ is more complicated. We derive it from $\Phi(t,0)\mathbf{\Sigma}_0 \Phi(t,0)^T$ and $\mathbf{\Sigma}_0$ is the prior covariance matrix which is user's choice. The explicit expression of $\Phi$ is provided in Appendix E.2.

3. Here is a quick example for $N=2$ case. By simply plugging $N$ and $t$ in equation, we get $\mu_0^{(0)}=t^2x_1$,$\mu_t^{(1)}=2tx_1$, $\Phi(t,0)=[[1-t^2, t-t^2],[-2t,1-2t]]$. Asumme $\Sigma_0 = [[1,0],[0,1]]$, Then $\Sigma_t=\Phi(t,0)\Phi(t,0)^{T}$. Then the user are ready to compute SNR and $\mathbf{r}_t$ by using equations above.

4. After obtaining $\mathbf{\mu}$ and $\mathbf{\Sigma}_t$, the user can get $\mathbf{r}_t$ by $\mathbf{r}_t=\frac{\mathbf{\Sigma}_t^{-1}\mathbf{\mu}_t}{\mathbf{\mu}_t^T\mathbf{\Sigma}_t^{-1}\mathbf{\mu}_t}$, meanwhile the effective SNR is the denominator $\mathbf{\mu}_t^T\mathbf{\Sigma}_t^{-1}\mathbf{\mu}_t$, and the user can use this value to match the SNR of Flow matching and Diffusion Model.

5. Takeaway from Fig.2

Here are the motivations of Fig.2:

1. We want to emphasize that our model can produce diverse outputs from a single initial state, exhibiting SDE-like diversity—albeit without true stochasticity. Specifically, the diversity comes from the interaction of $N$ variables which is theoratically proved in Prop.3.3 and we name it pseudo-noise.

2. We include Fig.2 to demonstrate that our method is not merely an alternative time-discretization of the vanilla diffusion model. If it were, the generation trajectories along the SNR (or effective-noise) from the same starting point—would coincide with EDM-ODE. The fact that they do not overlap underscores the unique dynamics of TADA, which in turn contributes to improved generation performance.

6. Arbitrary choice of NFE for Heun?

511 NFE is the default value used in EDM paper for Stochastic sampling and we notice that this number performance best. Applying the same logic to EDM2, we use the 63-NFE schedule cited as that paper’s default;

7 Summary

We hope this rebuttal can more or less address your concerns. Please feel free to raise any additional questions! We view the rebuttal process as a valuable opportunity to improve the quality of the paper and to validate that our ideas and results are meaningful to the broader community.

Also, If all concerns are resolved, we would deeply appreciate a reconsideration of the overall rating :)

Rebuttal

We thank the reviewer for thorough comments on both the main paper and the appendix. Below are our clarifications.

1. CLIP and FID on COCO

We still have marginal improvement in terms of FID on COCO validation set especially for small NFE. We found that standard CLIP tends to favor blurry images as long as they are semantically aligned with the text, which can result in inaccurate evaluations. Instead, we chose to report HpsV2, a fine-tuned variant of CLIP, as it offers more reliable and meaningful assessments in our setting. if the reviewer believes that including the standard CLIP score is necessary, we are happy to incorporate those results in future discussions.

2. Lack of Definitions in Fig.1

Thanks for the feedback. We will add explanations for each variable in the caption and also hyperlink to the exact equation to increase the accessibility.

3. Align the formulations of Eq. (3) and the MDM training objective in the proposition.

We are going to switch all the notation from $\epsilon$ prediction to $x_1$ prediction.

4. Indicate force reconstruction

This is indeed a good suggestion for clarity of the paper! We will have explanation on how to reconstruct $F_{\theta}$ in Prop.3.1 and also provide a reference to section 3.2.

5. Lack of Explanation of Prop.3.3

Good catch! We sincerely appreciate the reviewer’s effort in working through the detailed Appendix B. Our goal was to keep the main paper accessible by moving most theoretical material to the appendix, but we now see that the main text may be overly concise.

We will therefore, at least, add an intuitive explanation of $\epsilon^{(i)}$ to the main paper.

7. Inconsistency references format.

We sincerely thank the reviewer for pointing out this seemingly minor but significantly important point for readability. We will make sure to address it in the revision!

8. Welcome to join discussion

We further provide more interesting experiment results and analysis during the discussion with other reviewers. Feel free to have a look if you are interested in!