We extend our sincere appreciation to Reviewer bLLA for the thoughtful consideration of our manuscript and the valuable insights provided. We address each query in detail as outlined below.

Q1：Use some metrics to demonstrate diversity, such as Inception Score and Precision

Thanks for your constructive suggestion! In order to demonstrate diversity, Fig.6 - Fig.10 in Appendix D compare the sampling results between stochastic samplers（ER-SDE-Solvers） and deterministic samplers on various datasets when fixing the random seed. As stochastic samplers introduce stochastic noise at each step of the sampling process, the generated images exhibit greater diversity, which becomes more pronounced in higher-resolution images.
In addition to the FID metric already presented in our paper, we appreciate your suggestion and have included additional metrics to further showcase the superiority of ER-SDE-Solvers. Specifically, we have chosen four metrics: Inception Score, Precision, Recall, and sFID. The relevant results are provided in Appendix E. Experimental findings consistently indicate that ER-SDE-Solvers perform optimally across all metrics, highlighting superior diversity and fidelity across all stochastic samplers.

Q2：Could $\phi(x)$ be adaptive rather than manual?

Thank you for this constructive suggestion! Currently, there lacks a comprehensive theoretical explanation regarding the selection of the noise schedules. Previous studies [1][2][3] have predominantly relied on manually designed noise schedules. [4] argued that the implicit choice for noise schedules enjoys no special properties and the optimal amount of stochasticity should be determined empirically.
Compared to entirely empirical choices, we have taken a significant stride in the study of noise schedule principles. Our criterion for selecting the noise schedules (noise scale functions) is based on the analysis of the First-order Euler Integral (FEI) coefficient, coupled with relevant experimental observations, rendering it practical (see Appendix A.8). The customizable $\phi(x)$ provides practitioners with a broad spectrum of choices, allowing them to make diverse selections based on specific application scenarios. This flexibility enables a fine balance between the efficiency of sampling and the diversity of generated images.
Different noise scale functions $\phi(x)$ correspond to different solutions, collectively forming the solution space of the ER SDE. This aligns with the focus of this study, which is to elucidate the solution space of the ER SDE. Therefore, this paper only explores some of the noise scale functions for the reverse process, providing examples of excellent performance (such as ER SDE 4 and ER SDE 5). Of course, whether an optimal and adaptive $\phi(x)$ exists is worth further investigation.

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript.

[1]Denoising diffusion probabilistic models, Ho, J. et al.
[2]Improved techniques for training score-based generative models, Song, Y. and Ermon, S.
[3] gDDIM: Generalized denoising diffusion implicit models, Zhang et al.
[4] Elucidating the design space of diffusion-based generative models, Karras et al.

We sincerely appreciate you for the thoughtful consideration and valuable insights.We address each query in detail as outlined below.

Q1:Discuss SDE with uncorrelated g(t) in [1,2]

Thanks for highlighting the relevance of SDE with uncorrelated g(t) in [1,2].Our paper differs in the following aspects:

1. Though [2] used $\tau(t)g(t)$ when modeling the reverse process, they took $\tau(t)=\tau$to a constant function for computational convenience in experiments(Sec.6).This implies [2] essentially introduced a parameter to control the extent of noise. However, the form of the noise scale is associated with g(t). In contrast, ER SDE proposes an entirely new noise scale h(t) for the reverse process, suggesting h(t) may not necessarily correlate with g(t).
2. Due to the challenges in solving ER SDE,[1,2] didn't offer specific implementable solvers. To our knowledge, our paper is the first to offer exact analytical solution for ER SDE.
3. While [1,2] generalized SDE and ODE,as noted by Reviewer yrnV,our paper tackles the ratio of how the solver should work as an SDE or ODE solver.

In the revised version,Sec.3.1 provides discussion with[1,2],significantly enhance the motivation for our paper.

Q2:Ito-Taylor expansion rather than Taylor expansion

Thanks for pointing it out. It is appropriate to use Ito-Taylor expansion when deriving the approximation error for SDE.The key difference between Ito-Taylor and Taylor expansion is the infinitesimal operators. However, we’ve dropped the high-order terms,aligning with common practice in[2,3].

We sincerely apologize for the oversight. It is noteworthy the approximated solutions derived from Ito-Taylor and Taylor expansion are identical after dropping residual terms.As a result, our conclusions remain valid, supported by extensive experiments.Corrections have been made in Appendix A.3&A.5.We deeply appreciate your feedback again.

Q3:Utilizing the existing semi-linear structure lowers the novelty

Though the utilization of the semi-linear structure in analytical solutions has been established for ODE[4] and SDE[2,3,5],distinctions are as follows:

1. Equations being solved are different.[4] is based on ODE and[2,5] are based on the original SDE, while we use the semi-linear structure to solve ER SDE. Compared to the former, solving ER SDE is more challenging[2].Although [3] and our paper are contemporaneous, we are open to comparisons. SDE in [3] is $dx_t=[A(t)x_t+b(t)\nabla_x\log p_t(x_t)]dt+g(t)dw_t$, while ER SDE is $dx_t=[f(t)x_t-\frac{g^2(t)+h^2(t)}{2}\nabla_{x}\log p_t(x_t)]dt+h(t)dw_t$.We can see these two SDEs are distinct.
2. Using the semi-linear structure to solve differential equations is common, dating back to [6].However,we aim at elucidating more comprehensive general solutions for SDEs. Based on it,we find the superior performance of ODE solvers for fast sampling over SDE solvers, and the equal performance of VP solvers with VE solvers.

We hope our response clearly articulates the unique contributions of our paper,addressing your concerns effectively.

Q4:Design theory-motivated noise schedule principles

Thanks for constructive suggestion!Now, there lacks theoretical explanations for selecting noise schedules.Previous studies[7,8,9] have mainly relied on manually designed noise schedules.[1] argued the choice for noise schedules enjoys no special properties and the optimal amount of stochasticity should be determined empirically.
Compared to purely empirical choices,we've taken a significant stride in the study of noise schedule principles. Our criterion for selecting noise schedules is based on FEI coefficient and experimental observations, which is practical.Customizable $\phi(x)$ offers practitioners diverse options,enabling a balance between sampling efficiency and image diversity tailored to specific application scenarios.
Different noise schedules $\phi(x)$ correspond to different solutions, collectively forming the solution space of the ER SDE,which aligns with the focus of our paper: elucidating the solution space of ER SDE. Therefore,we only explore some of $\phi(x)$ for the reverse process (ER SDE 4&5).The optimal $\phi(x)$ is worth further investigation.
We hope our responses meet your expectations,and we'd be grateful if these clarifications positively contribute to assessing our paper.

[1]Elucidating the design space of diffusion-based generative models.
[2]SA-Solver:Stochastic Adams Solver for Fast Sampling of Diffusion Models.
[3]SEEDS:Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion.
[4]DPM-Solver:A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps.
[5]DPM-Solver++:Fast Solver for Guided Sampling of Diffusion Probabilistic Models.
[6]Numerical Solution of Stochastic Differential Equations.
[7]Denoising diffusion probabilistic models.
[8]Improved techniques for training score-based generative models.
[9]gDDIM:Generalized denoising diffusion implicit models.

We extend our sincere appreciation to Reviewer yrnV for the thoughtful consideration of our manuscript and the valuable insights provided. We address each query in detail as outlined below.

Q1：Why does the low FEI coefficient lead to high sampling performance in low-NFE regime?

Thank you for your question! In our paper, Sec.3 demonstrates that the exact solution of the ER SDE comprises three components: a linear function of the data variables, a non-linear function parameterized by neural networks and a noise term. The linear and noise terms can be precisely computed, while discretization errors are present in the non-linear term. Due to the decreasing error as the order increases (see Table1), the first-order error predominantly influences the overall error. Therefore, we exemplify the case with order k = 1 for error analysis. It can be observed from Eq.(16) and Eq. (17) that the error arises from the second term \Big{[}1-\frac{\phi(\sigma_{t_{i}})}{\phi(\sigma_{t_{i-1}})}\Big{]} x_\theta(\tilde{x} _ {\sigma_{t_{i-1}}},\sigma_{t_{i-1}}), as $x_\theta(\tilde{x} _ {\sigma_{t_{i-1}}}, \sigma_{t_{i-1}})$ is a neural network trained in the forward process to estimate data, whose precision is fixed. To reduce discretization error, it is necessary to minimize the coefficient $1 - \frac{\phi(x_t)}{\phi(x_s)}$, referring as the First-order Euler Integral (FEI) coefficient. Therefore, when using the same model weights, a lower FEI coefficient leads to smaller discretization error. In turn, the images generated in the reverse process more similar to real images in the forward process, resulting in higher image quality and a smaller FID.
In summary, our ER-SDE-Solvers directly utilize raw information without retraining, offering broad applicability and high flexibility. We hope our explanation addresses your concerns.

Q2：What is the advantage of the stochastic solver compared to the deterministic solver?

Thanks for your insightful question!  

1. Stochastic samplers can produce higher-quality images with a minimal increase in NFE(refer to Table 2, Table 9).
2. stochastic samplers with classifier guidance exhibit superior performance in rapid sampling compared to deterministic samplers. We appreciate your suggestion and have incorporated this result into the main material to further substantiate the benefits of ER SDE 5.
3. SDE-based stochastic samplers exhibit an enhanced capability to generate variable images by injecting additional noise into the data state at each sampling step. This conclusion is further supported by Fig.6-Fig.10 in Appendix D, where stochastic samplers demonstrate a greater variability in generated images when the random seed is fixed.

Q3：What is $\phi(x)$ used for experiments in ER-SDE-Solvers on ImageNet64?

$\phi(x)$ used for experiments in ER-SDE-Solvers on ImageNet64 is $φ(x) = x(e^{x^{0.3}} + 10)$（ER SDE 5）. As outlined in Sec.4.2, we select ER SDE 5 as the noise scale function by default if there is no special indication. We hope our clarification addresses your concerns.

Q4：Compare ER-SDE-Solvers to PNDM and DEIS

Thank you for your suggestions! One of the contributions of our paper is to yield mathematical insights elucidating the superior performance of ODE solvers over SDE solvers in terms of fast sampling. Building upon it, we design a stochastic sampler with rapid sampling capabilities comparable to ODEs. To demonstrate the effectiveness of our sampler, we select only a subset of representative ODE-based samplers for comparison due to space constraints.
PNDM and DEIS are also ODE-based samplers with superior performance. Following your suggestion, we compare our method with them in Table 10. It can be observed that ODE-based samplers demonstrate superiority in rapid sampling, which further validates our standpoint.

Q5：What does the 'step' in Figure 3 stand for, in both FEI coefficients and FID scores cases?

Your opinion is valid. The steps stand for the sampling step within the whole 200 steps of the reverse process, and the number of function evaluations (NFE) for FID scores. Thank you for your thorough review!

Q6：Align some of the large-scale results to the main material

Appreciate your suggestion! Stochastic samplers guided by classifiers demonstrate superior performance in rapid sampling of high-resolution images compared to deterministic samplers. We have incorporated this result into the main material (Table 3) to further substantiate the benefits of ER SDE 5.

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript.

Dear reviewer,

We sincerely hope that our recent response and revisions have successfully addressed your concerns. Your feedback is invaluable to us, and if you find that our efforts have met your expectations, we would be immensely grateful if you could reconsider and possibly elevate the score for our work.

If you have any additional questions or suggestions, we would be happy to have further discussions.

Best regards,

The Authors

We extend our sincere appreciation to Reviewer RiM8 for the thoughtful consideration of our manuscript and the valuable insights provided. We address each query in detail as outlined below.

Q4：What is the order of the algorithm used to report Fig.3?

We sincerely apologize for confusion. In the experiments depicted in Fig. 3, we utilize the first-order ER SDE Solvers. The relevant experimental details have been supplemented in the legend of Fig. 3. Thank you for pointing it out!

Q5：The claim of outperforming all other stochastic samplers is not well supported？Can ER-SDE-solver further improve sample quality by increasing NFE?

Thanks for your question! The purpose of our paper is to accelerate sampling for diffusion models, so we limit our experiments to a finite number of steps (<=50 NFE), which is consistent with many works in the field [4][5][6]. For a fair comparison, FID metrics should be compared under the condition of fixed NFE. When NFE=50, ER SDE 5 achieves 2.24 FID on ImageNet64, while EDM-Stochastic only achieves 2.49. Therefore, we claim that ER-SDE-Solvers achieve state-of-the-art performance across all stochastic samplers in the context of rapid sampling.
In fact, ER-SDE-Solver can enhance sample quality by increasing NFE. Experimental findings indicate that judiciously introducing additional stochastic noise can effectively rectify errors in the early sampling steps, thereby generating high-quality samples in prolonged sampling scenarios. However, blindly increasing NFE for better FID might not be practical in real-world applications. Therefore, our paper focuses on rapid sampling rather than achieving absolute high-quality samples. Leveraging ER-SDE-Solvers for high-quality sampling can be considered for future work.

Q6：Spelling correction

We sincerely apologize for the spelling error. We have rectified it in the revised version. Thank you for pointing it out!

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript.

[4] Fast sampling of diffusion models with exponential integrator, Zhang et al.
[5] DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps, Lu et al.
[6] DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models, Lu et al.

We extend our sincere appreciation to Reviewer RiM8 for the thoughtful consideration of our manuscript and the valuable insights provided. We address each query in detail as outlined below.

Q1：The discretization error discussed in Sec.4.1 needs further clarification

We sincerely apologize for confusion! Sec.3 demonstrates that the exact solution of the ER SDE comprises three components: a linear function of the data variables, a non-linear function parameterized by neural networks and a noise term. The linear and noise terms can be precisely computed, while discretization errors are present in the non-linear term. Due to the decreasing error as the order increases (see Table1), the first-order error predominantly influences the overall error. Therefore, we exemplify the case with order k = 1 for error analysis.
It can be observed from Eq.(16) and Eq. (17) that the error arises from the second term\Big{[}1-\frac{\phi(\sigma_{t_{i}})}{\phi(\sigma_{t_{i-1}})}\Big{]} x_\theta(\tilde{x} _ {\sigma_{t_{i-1}}},\sigma_{t_{i-1}}), as $x_\theta(\tilde{x} _ {\sigma_{t_{i-1}}}, \sigma_{t_{i-1}})$ is a neural network trained in the forward process to estimate data, whose precision is fixed. To reduce discretization error, it is necessary to minimize the coefficient $1 - \frac{\phi(x_t)}{\phi(x_s)}$, referring as the First-order Euler Integral (FEI) coefficient. Therefore, when using the same model weights, a lower FEI coefficient leads to smaller discretization error. In turn, the images generated in the reverse process more similar to real images in the forward process, resulting in higher image quality and a smaller FID. We hope our explanation addresses your concerns.

Q2：It is tricky to find N-point and noise scale function for different models in practice

Thank you for your thorough review! We conducted ablation experiments in Appendix C.3 to investigate the choice of the numerical integration points $N$. To strike a balance between efficiency and performance, we fixed $N = 100$ for all experiments in our paper. Although the ablation experiments were conducted on CIFAR-10 dataset using the pretrained EDM, our experimental results on other datasets such as FFHQ, ImageNet, and LUSN, using Guided-diffusion as the pretrained model, indicate that the chosen integration point $N=100$ still performs well. Therefore, practitioners do not need to excessively adjust the number of numerical integration points in practice.
Regarding the noise scale function, our experiments were conducted using the pretrained EDM on CIFAR-10 dataset (See Fig.3). Extensive experimental results indicate that our designed ER SDE 5 can generalize to all datasets and pretrained diffusion models. However, we still provide practitioners with the opportunity to customize the noise scale function according to specific application scenarios. Compared to entirely empirical choices [1][2][3], we have taken a significant stride in the study of noise schedule principles. Different noise scale functions $\phi(x)$ correspond to different solutions, collectively forming the solution space of the ER SDE. This aligns with the focus of our study, which is to elucidate the solution space of the ER SDE. Therefore, we only explores some of the noise scale functions for the reverse process, providing examples of excellent performance (such as ER SDE 4 and ER SDE 5). Of course, whether an optimal and adaptive $\phi(x)$ exists is worth further investigation.

Q3：It is difficult to see the difference from Fig. 3 (b)

We apologize for the lack of clarity in the figures. We have provided enlarged details for Fig. 3(a), Fig. 3(b), Fig. 5(a) and Fig. 5(b) in the revised version. As observed, the ODE solver exhibits minimal discretization error. ER SDE 4 exhibits discretization error that closely adheres to the behavior of the ODE. ER SDE 5 demonstrates elevated error in the initial 100 steps and gradually converges to the ODE’s error profile. Both ER SDE 4 and ER SDE 5 exhibit comparable efficiency to the optimal ODE solver. Image quality deteriorates for ill-suited noise scale functions (like ER SDE 2).

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript.

[1]Denoising diffusion probabilistic models, Ho, J. et al.
[2]Improved techniques for training score-based generative models, Song, Y. and Ermon, S.
[3] gDDIM: Generalized denoising diffusion implicit models, Zhang et al.

We extend our sincere appreciation for your valuable insights. We address each query in detail as outlined below.

Q1：Precisely define discretization error

The discretization error as defined by us does not refer to the local error $\tilde{x}_t-x_t$, but pertains to the error introduced when discretizing the nonlinear integral term $\int_{\sigma_t}^{\sigma_s}\frac{\phi^{(1)}(\sigma)}{\phi^2(\sigma)}x_\theta(x_\sigma,\sigma)d \sigma$. The discretization error we define directly influences the local error, as Section 3 demonstrates that the exact solution of ER SDE consists of three components: a linear function of the data variables(1st term), a non-linear function parameterized by neural networks(2nd term), and a noise term(3rd term), i.e.,

The linear and noise terms can be precisely computed, while local errors are present in the non-linear term. Due to the decreasing error as the order increases (Table1), the first-order error predominantly influences the overall error. Therefore, we exemplify the case with order k = 1 for error analysis. It can be observed that the discretization error arises from the second term is a neural network trained in the forward process to estimate data, whose precision is fixed. To reduce discretization error, it is necessary to minimize the coefficient $1 - \frac{\phi(x_t)}{\phi(x_s)}$, referring as the First-order Euler Integral (FEI) coefficient. Therefore, when using the same model weights, a lower FEI coefficient leads to smaller discretization error. We hope our explanation addresses your concerns.

Q2：Why does ER-SDE 5 have better results than ER-SDE 4 on CIFAR10 and ImageNet64?

Thanks for your question! In fact, the sampling performance of ER-SDE 5 is inferior to that of ER-SDE 4 when NFE is relatively small. Table 4 and Table 5 in Appendix D present the image generation quality of different noise scale functions on ImageNet 64 × 64 and CIFAR-10. We cite here to demonstrate clearly:

Table 4: Sample quality measured by FID$\downarrow$ on ImageNet $64\times64$ for different noise scale functions with VE ER-SDE-Solver-3.

Table 5: Sample quality measured by FID$\downarrow$ on CIFAR-10 for different noise scale functions with VE ER-SDE-Solver-3.

Due to the discretization errors of ER-SDE 4 being sufficiently close to the minimal error (see Fig. 3), it exhibits outstanding FID scores on CIFAR-10 and ImageNet 64×64 (NFE≤20). This result once again confirms the validity of the relationship between FEI and discretization error. In the experiments on ImageNet 64×64, despite the initial significant discretization error in ER-SDE 5, its later-generated image quality (NFE≥20) is comparable to ER-SDE 4. This is attributed to the fact that ER SDE involves implicit Langevin diffusion, which can effectively correct any errors in the early sampling steps [1]. Consequently, the early errors in ER-SDE 5 are rectified in the later stages.

Q3：Why not just do ODE instead of ER SDE?

Thanks for your insightful question! As highlighted in Sec.1, SDE-based stochastic samplers exhibit an enhanced capability to generate diverse images by injecting additional noise into the data state at each sampling step. This conclusion is further supported by Fig.6-Fig.10 in Appendix D, where stochastic samplers demonstrate a greater variability in generated images when the random seed is fixed. This advantageous feature becomes particularly prominent in the context of high-resolution image generation, effectively meeting the demands of industries such as art creation and game development.
In addition, stochastic samplers with classifier guidance exhibit superior performance in rapid sampling compared to deterministic samplers(see Table 3). This is attributed to the customized noise injected into the sampling process, which mitigates the inaccuracies in data estimation introduced by classifier gradient guidance.

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript. Thank you for your time and consideration.
[1] Elucidating the design space of diffusion-based generative models, Karras et al.

Thanks for your detailed reply.

R1: Could you provide a precise mathematical definition of the discretization error discussed in the paper and clearly show how FEI controls the discretization error? The current response seems to repeat what you wrote in the paper.

R3: Thanks for referring to the Fig.6-Fig.10. However, these are just some generated samples. A more quantitative comparison of sample diversity is needed if the focus is to enhance diversity. For example, report Recall on ImageNet256 and compare with existing methods like EDM, DPM solvers. Right now, this is not strongly supported by the experiments. For example, in Table 13, ER-SDE-Solver has worse Recall than the deterministic DPM-solver, which is an important metric for measuring sample diversity.

We extend our sincere appreciation for your valuable insights. We address each query in detail as outlined below.

Q4：Why Eq.(9) is linear in the narrow sense?

Thanks for your question! Although $x_\theta$ is a nonlinear neural network, it is pre-trained during the forward process. Eq.(9) represents a stochastic differential equation with the data state $x_t$ in the reverse process as the variable. According to the discussion in Section 4.2 of [2] regarding Eq.(2.3), we refer to Eq.(9) as linear in the narrow sense.

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript. Thank you for your time and consideration.

[2] Numerical Solution of Stochastic Differential Equations, Peter E Kloeden and Eckhard Platen.

We extend our sincere appreciation for your valuable insights. We address your query in detail as outlined below.

Q: The diversity of generated images may not be strongly supported by the experiments?

Thanks for your insightful question! We suppose that the Recall metric may not adequately reflect the diversity of generated images when the Number of Function Evaluations (NFE) is small. For instance, when employing DDIM ($\eta=1$)(Stochastic Sampler), the Recall is lower than that of DDIM (Deterministic Sampler), and there exists a significant gap between them. However, when NFE<=20, the Recall of the ER-SDE-Solver is slightly lower than that of the DPM-Solver by 0.01-0.02. This demonstrates that our approach indeed enhances the diversity of generated images while ensuring efficient sampling performance. This may be attributed to the lower quality of images when NFE is low, which could potentially impact the Recall metric.

Additionally, the pre-trained model employed in Table 13 is Guided-diffusion[1]. According to the results reported in [1], when NFE is around 250, Guided-diffusion achieves its best Recall metric at only 0.65 (Table 5 in [1]). In contrast, the ER-SDE-Solver already reaches a Recall of 0.66 at NFE=30. Due to limitations in the model's fitting capacity, even with further increases in NFE, it is challenging to achieve significant improvements in Recall.

We sincerely hope that our responses align with your expectations, and we would be extremely grateful if these clarifications contribute positively to the assessment of our manuscript. Thank you for your time and consideration.

[1] Diffusion models beat gans on image synthesis, Dhariwal P, Nichol A.

Concern 1:

I'm not sure how you apply the Taylor expansion here. But I feel the second equation should be $x_t = \frac{\phi(\sigma_t)}{\phi(\sigma_s)}+\phi(\sigma_t) \frac{\phi^{(1)}(\sigma_s)}{\phi^2(\sigma_s)}x_0(x_{\sigma_s}, \sigma_s)\int_{\sigma_t}^{\sigma_s}\mathrm{d}\sigma + ...$ Can you provide intermediate steps to derive that?

Putting this derivation aside, suppose your derivation is correct. If I'm not mistaken, you define the $x_0$ to be the data state (some form of score function) of the actual training data distribution. The error you defined is the approximation error that arises from the neural network rather than the discretization error. The discretization error is from discretizing a continuous function, which has nothing to do with the accuracy of the neural network. Controlling the approximation error is not a meaningful goal for the sampling problem of the pre-trained diffusion model. The goal here is sampling from the distribution defined by the trained diffusion model rather than sampling from the training data distribution.

Concern 2

Thank you for explaining your motivation in detail. However, I find the paper confusing now, as I'm not sure about its key message. Here are my observations:

1. In section 4.1, the paper claims that the ODE sampler is superior to the stochastic sampler for rapid sampling based on its error analysis.

2. In contrast, the authors propose a specialized stochastic sampler, ER-SDE-solver, for rapid sampling, which seems contradictory to the previous claim. The motivation behind this proposal is not clear.

3. On page 2, the last sentence of the first paragraph suggests that stochastic samplers are more promising for larger sampling steps. However, the paper focuses on rapid sampling that uses small steps. During the rebuttal, the authors stated that their focus is on rapid sampling rather than achieving the highest quality samples. However, they also claim that the ODE sampler is better than the stochastic sampler in the rapid-sampling region. This raises my question of why not just use an ODE sampler instead of proposing a new stochastic sampler.

4. In response to this question, the authors claim that SDE-based stochastic samplers have the ability to generate diverse images by injecting additional noise into the data state at each sampling step. However, this claim is not well-supported by either empirical or theoretical results. Also, the proposed stochastic solver doesn't even outperform the deterministic DPM-solver on the only metric report in the paper for measuring diversity.

I think this paper might need major revisions to address these concerns. I'll change the rating to 3.