Rethinking the Noise Schedule of Diffusion-Based Generative Models

We thank the reviewer for the positive and comprehensive feedback. We are encouraged that the reviewer found our WSNR noise schedule is beyond adjusting noise schedule intuitively. In the following, we provide a response to the questions in the review.

Q1: Why should we align the schedule of high-resolution image generation to 64 x 64? Whether the proposed metric could shed light on optimal schedule for image generation?

A1: We appreciate the reviewer’s query regarding our anchor noise schedule choice.

We chose the 64x64 noise schedule as our anchor training noise schedule because the diffusion model is well-trained at this resolution. However, the choice of resolution for alignment is flexible.

Our WSNR metric lays the foundation for finding the optimal schedule for image generation, reducing our exploration to just one set of hyperparameters. This is because the WSNR-Equivalent Training Noise Schedule consistently achieves superior results with a single set of hyperparameters. In contrast, other noise schedules fail to consistently achieve desired performance across multiple resolutions in both RGB and latent spaces under a single hyperparameter set.

Furthermore, we believe that our proposed data-driven sampling schedule can aid in exploring the design of training noise schedules. Since probabilistic generative models are essentially fitting data distributions, we hold that the optimal noise schedule should be data-driven. We leave this exploration to our future work.

Q2: Is the newly proposed sampling strategy highly related to the WSNR metric? Can we adjust the sampling schedule based on other metrics such as PSNR?

A2: We are very grateful to the reviewer for raising such insightful questions, which have sparked our thinking about how to use WSNR to uniformly design both training and sampling noise schedules.

Our proposed WSNR metric can be used to uniformly represent noise levels across different scales, meaning that a data-driven sampling schedule can be designed and expressed through WSNR. We found that for the FFHQ-64 and FFHQ-256 datasets, the ODE integration intervals should be [0.002, 98.96] and [0.008, 394.62], respectively. When represented in terms of WSNR, these intervals become [117166875, 0.0478] and [117166875, 0.0481], showing that FFHQ 64 and 256 are similar in terms of the WSNR metric.

We have not explored replacing WSNR with PSNR, because WSNR is a promising metric. We will leave the exploration regarding PSNR for future work.

---

We would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

Your further guidance and feedback will be important to us and we are committed to making any additional improvements as needed.

Thank you very much for your time and consideration. We look forward to your response.

Q1: The concurrent work [1] also discusses the noise schedule from the spectra view. The authors are encouraged to discuss the differences.

A1: Thank you for bringing concurrent work [1] to our attention. We acknowledge that [1] also explores the noise level from a spectral perspective in RGB space.

However, our work and [1] have several distinct differences:

1. Different Approaches: Our work is dedicated to designing a robust training noise schedule applicable across different resolutions and latent spaces. [1] mainly focuses on the relay connection within their proposed RDM, enabling a cascading sampling process from 64x64 to 256x256 in a single pass. It seems that they do not contribute to the training noise schedule. If my understanding of [1] is incorrect, please kindly correct me.
2. Different Analysis Metrics: We propose WSNR as a consistent quantization metric, ensuring consistent quantization results at the same noise level across different resolutions. In contrast, [1] uses the SNR quantization metric, which is not consistent in different resolutions.
3. Different Analytical Perspectives: We analyze the average power of images and Gaussian noise from a power spectrum perspective, highlighting that the average power of Gaussian noise is uniform across all frequency components. This conclusion is not stated in their work.

Additionally, it's worth noting that we have conducted theoretical analysis on the sampling noise schedule and proposed improvements. Moreover, [1] is also a submission to ICRL 2024. We believe these are two independent pieces of research work, and this should not be considered a weakness or even a reason for rejection.

Q2: The noise schedule is discussed in previous works from different perspectives.

A2: We appreciate the reviewer's comment regarding the discussion of noise scheduling in prior works. Since our work is titled "Rethinking the Noise Schedule of Diffusion-Based Generative Models", we have acknowledged that some previous works have explored noise schedule design. However, our paper has unique contributions:

• We propose a novel WSNR metric to consistently quantify the noise level across multiple resolutions in RGB space and latent space for the forward process of diffusion models.
• We significantly improve the performance of diffusion models, via our novel WSNR-Equivalent training noise scheduler. Specifically, our method decreases the FID score by 3.6 on FFHQ-256x256 dataset, and achieves the same FID scores on ImageNet-256 and 512 for UViT-M (#Params: 287M) as those of UViT-L (#Params: 287M).
• We analyze that the ODE sampling noise schedule should be data-driven. Consequently, we propose estimating the integration interval based on the average data distance.
• We explore the relationship between the number of evaluation points and generation quality. Our findings lead us to develop a NFE-guided sampling noise schedule.
• Our sampling schedule refines the FID of pre-trained CIFAR-10 and FFHQ-64 models from 1.92 and 2.45 to 1.89 and 2.25, respectively, utilizing 35 network evaluations per image.

As the reviewer did not provide further explanation and references about the "previous works" and "different perspectives", we will discuss the recent representative research works [2,3,4,5,6], which proposed handcrafted training noise schedules to improve the performance of diffusion models. However, the proposed schedules are based on the SNR metric, which is neither consistent in different resolutions, nor consistent in latent space.

That means their schedules fail to achieve desired performance consistently on various across multiple resolutions in RGB space and latent space under a single set of hyperparameters. In contrast, ours can achieve superior results consistently with just one set of hyperparameters.

Besides, our analysis and improvement on ODE sampling noise schedule, an aspect not addressed by the other methods. This unique focus not only demonstrates our method's innovative approach but also highlights its ability to fill a critical gap in the current research landscape. By delving into areas previously unexplored, our work contributes significantly more to the advancement of the field, offering novel insights and practical improvements that set a new standard for future research in this area.

Q3: The highest resolution is 256x256. Experiments with a higher resolution are highly recommended.

A3: We thank the reviewer for highlighting the importance of conducting experiments at higher resolutions, especially given our paper's focus on improving noise scheduling in high-resolution contexts.

Due to the limited time for rebuttal and our computational resources, we trained a network on ImageNet 512x512 to demonstrate the effectiveness of our training schedule. Specifically, we use UViT-M as network architecture, similar to the experiment in our draft on ImageNet 256x256, keeping the hyperparameters unchanged. The results, as shown in the table below, indicate that UViT-M, utilizing our WSNR-Equivalent noise schedule, achieves a superior FID score compared to the UViT-L model, which has more than double the parameter count of UViT-M.

We understand the importance of ensuring that our paper meets the high standards of ICLR and would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

Your further guidance and feedback will be important to us and we are committed to making any additional improvements as needed.

Thank you very much for your time and consideration. We look forward to your response.

[1] Teng, Jiayan, et al. "Relay diffusion: Unifying diffusion process across resolutions for image synthesis." arXiv preprint arXiv:2309.03350 (2023).

[2] Nichol, Alexander Quinn, and Prafulla Dhariwal. "Improved denoising diffusion probabilistic models." International Conference on Machine Learning. PMLR, 2021.

[3] Kingma, Diederik, et al. "Variational diffusion models." Advances in neural information processing systems 34 (2021): 21696-21707.

[4] Choi, Jooyoung, et al. "Perception prioritized training of diffusion models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[5] Chen, Ting. "On the importance of noise scheduling for diffusion models." arXiv preprint arXiv:2301.10972 (2023).

[6] Hang, Tiankai, et al. "Efficient diffusion training via min-snr weighting strategy." arXiv preprint arXiv:2303.09556 (2023).

Dear Reviewer pugY,

We appreciate the diligent efforts put into reviewing our work.

We have compared our novel training and sampling noise schedule with both previous and concurrent works. Besides, we conducted experiments on a higher resolution. Furthermore, we improved the writing quality in our revised manuscript.

We would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

We sincerely look forward to your response and would happily address any other questions.

Thank you,

The authors

Dear Reviewer pugY,

Just a reminder, there are only 4 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

Dear Reviewer pugY,

Just a reminder, there are only 3 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

Dear Reviewer,

Just a reminder, there are only 2 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

We are grateful to the reviewer for the comprehensive feedback. We are encouraged by the recognition that our WSNR metric is both novel and well-motivated, and that the proposed sampling schedule is efficient and effective. Below, we respond to the questions raised in the review.

Q1: The motivation or justification for rethinking the noise schedule of diffusion-based generative models is not clearly explained in the introduction. How Figure 1 is related to the main motivation of the paper?

A1: We appreciate the reviewer’s feedback regarding the clarity of our paper’s motivation and the role of Fig 1. We acknowledge an error in our initial submission regarding the referencing of figures. This analysis is actually presented in Fig 2. We apologize for any confusion this may have caused and have corrected this reference in our revised manuscript. Fig 1 is intended to illustrate the training noise schedule and sampling noise schedule.

We have revised the introduction section to explain the motivation clearly. Here, we summarize our motivation briefly:

1. As shown in Fig. 2, significant variation in noise levels across images of different resolutions when exposed to Gaussian noise with identical standard deviations, leads to a discrepancy in the noise levels focused on by manually designed training noise schedules in different RGB space resolutions and latent space. Motivated by this observation, we propose a novel metric WSNR to quantify the noise level consistently across multiple resolutions in RGB space and latent space. We further propose a novel training noise schedule based on the WSNR metric.

2. As illustrated in Fig. 4 and elaborately explained in Sec. 5, the diversity of the generated data is jointly influenced by the initial Gaussian distribution at the start point and the Euclidean distance from the data points in the dataset. Therefore, we propose a data-driven sampling noise schedule to determine the integration interval of the diffusion ODE.

3. As demonstrated in Tab. 4, an increased number of evaluation points (the unique time steps at which model makes predictions) leads to better results when the step size is relatively large. Conversely, with smaller step sizes, the advantage of adding more evaluation points becomes less significant. Drawing on these findings, we propose a strategy for dynamically selecting numerical methods according to computational constraints, aiming to optimize generation quality.

Q2: From Eq. (1), it seems to implicitly assume the variance exploding (VE) case, but it is not clear what happens in the variance preserving (VP) case.

A2: Our proposed methods are applicable to VP case. As shown in Tab.3, our data-driven sampling noise schedule works when it is applied to DPM-Solver, which is based on VP ODE.

Besides, we conduct experiments on VP cases to evaluate our WSNR-Equivalent training noise schedule. The FID score on FFHQ dataset in the following table show that our method further improves performance on VP cases.

Actually, the denoiser D_theta (x) in Eq. 1 can be used to explain VP, VE and EDM cases. As shown in Eq. 7 in [1], we can write the $D_\theta(x; \sigma)$ in the following form:

$D_\theta(x; \sigma) = c_{\text{skip}}(\sigma) x + c_{\text{out}}(\sigma) F_\theta(c_{\text{in}}(\sigma) x; c_{\text{noise}}(\sigma))$ For VP, $c_{\text{skip}}(\sigma)=1, c_{\text{out}}(\sigma)=-\sigma, c_{\text{in}}(\sigma)=\frac{1}{\sqrt{\sigma^2 + 1}}, c_{\text{noise}}(\sigma) = (M-1)\sigma^{-1}(\sigma)$. This implies that our Eq.1 is compatible with the VP case and our sampling noise schedule can be evaluated with the pre-trained VP model.

As for the training process, we follow the training loss in [1], setting the training target as:

$E_{\sigma, y, n} \left[ \overbrace{\lambda(\sigma) c_{\text{out}}(\sigma)^2}^{\text{effective weight}} \left\| \overbrace{F_\theta \left( c_{\text{in}}(\sigma) \cdot (y + n); c_{\text{noise}}(\sigma) \right)}^{\text{network output}} - \overbrace{\frac{1}{c_{\text{out}}(\sigma)} (y - c_{\text{skip}}(\sigma) \cdot (y + n))}^{\text{effective training target}} \right\|^2 \right]$

Therefore, we believe that our training process is compatible with interpreting the VP case.

[1] Karras, Tero, et al. "Elucidating the design space of diffusion-based generative models." Advances in Neural Information Processing Systems 35 (2022).

We understand the importance of ensuring that our paper meets the high standards of ICLR and would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

Your further guidance and feedback will be important to us and we are committed to making any additional improvements as needed.

Thank you very much for your time and consideration. We look forward to your response.

Dear Reviewer 9Ms4,

We appreciate the diligent efforts put into reviewing our work.

We have tried to answer all your question about the Eq.1 and would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

As you recognize so many strengths of our work, we sincerely look forward to your response.

We would happily address any other questions.

Thank you,

The authors

Dear Reviewer 9Ms4,

Just a reminder, there are only 4 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

Dear Reviewer 9Ms4,

Just a reminder, there are only 3 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

Dear Reviewer,

Just a reminder, there are only 2 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

We are thankful to the reviewer for their very positive and comprehensive feedback. We feel encouraged by their acknowledgment that our WSNR metric effectively quantifies the noise level, independent of image resolution, and is also applicable to latent space.

Q1: How would the metric depend on the local properties of the image, such as the presence of flat regions or textures?

A1: We are very grateful to the reviewer for raising such insightful questions, which is an important property for both neural networks and image processing.

The WSNR metric, designed based on power spectrum analysis, quantifies noise levels at the image level. The frequency characteristics of local areas influence the overall power spectrum, thereby affecting the WSNR.

Local properties are a common and crucial property for neural networks. We can further use Short-Time Fourier Transform (STFT) to analyze the power spectrum of local areas more directly and likewise use WSNR to represent the noise level in these areas. We plan to include this analysis in the final version of our work.

Thank you very much for your time, consideration and support.

We thank the reviewer for the positive and comprehensive feedback. We are encouraged that the reviewer found our WSNR metric represents noise level well and our adaptive sampling scheme balances efficacy and efficiency well. In the following, we provide a response to the questions in the review.

Q1: The proposed WSNR-Equivalent training noise schedule and data-driven sampling noise schedule seem to be independent of each other, which weakens the focus of this paper.

A1: We respectively disagree with this viewpoint.
Firstly, the training and sampling processes of diffusion models are strongly interrelated. Diffusion models are trained independently for each noise level. The inference stage can be seen as refining the generated images multiple times at various timesteps using the network. We believe that optimizing both the training and sampling stages simultaneously highlights our contribution.

Secondly, exploring the training and sampling processes of diffusion models within a single work is not a weakness. In the NIPS 2022 Outstanding Paper [1], the authors similarly improved the training and sampling processes of diffusion models, which was not considered a weakness by the reviewers, area chairs, or senior area chairs.

Finally, combining our methods will further enhance the performance. We evaluate our trained model on FFHQ-256 using our proposed sampling method. The table shows that our sampling method consistently lowers the FID score of the trained model, indicating improved performance since a lower FID score is desirable.

[1] Karras, Tero, et al. "Elucidating the design space of diffusion-based generative models." Advances in Neural Information Processing Systems 35 (2022): 26565-26577.

Q2: Experiments in Table 1 and Table 2 compare with only EDM training noise schedule. The authors are suggested to compare with more training noise schedules to further verify the effectiveness of training noise schedules.

A2: We appreciate your suggestion to include comparisons with more training noise schedules to further validate the effectiveness of our approach.

Due to the limited time for rebuttal and our computational resources, we conduct experiments on VP cases to evaluate our WSNR-Equivalent training noise schedule. The results in the following table show that our method further improves performance on VP cases.

We understand the importance of ensuring that our paper meets the high standards of ICLR and would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

Your further guidance and feedback will be important to us and we are committed to making any additional improvements as needed.

Thank you very much for your time and consideration. We look forward to your response.

Q3: Are u and v in the frequency domain? What technique is used to convert images into frequency domain?

A3: Yes, u and v are in the frequency domain. Because $P$ represents the power spectrum. The power spectrum is defined as $P(u,v) = |F(u,v)|^2$ where $F(u,v)$ is the 2D discrete Fourier transform (DFT). Please check the Eq. (4-67) and (4-89) in [2], which is a textbook widely adopted by university computer vision courses.

[2] Gonzalez, Rafael C. Digital image processing 4th Edition.

Q4: Given a finite dataset, an ideal solution for the denoiser D(xt) can be found as the weighted sum of all clean data in the dataset. The ideal solution is not proved.

A4: We appreciate the reviewer's point regarding the lack of a formal proof for our assertion that an ideal solution for the denoiser can be represented as a weighted sum of all clean data in the dataset.

Here, we provide proof to make it clear: Given the standard deviation of Gaussian noise $\sigma$ and the clean data probability distribution $p_{\text{data}}(x) = \frac{1}{Y} \sum_{i=1}^{Y} \delta(x - y_i)$, the probability of the noisy data $p(x; \sigma)$ can be expressed as:

$ p(x; \sigma) & = p_{\text{data}} * \mathcal{N}(0, \sigma^2 \mathbf{I}) \\ & = \int_{\mathbb{R}^d} p_{\text{data}}(x_0) \mathcal{N}(x; x_0, \sigma^2 \mathbf{I}) dx_0 \\ & = \int_{\mathbb{R}^d} \left[ \frac{1}{Y} \sum_{i=1}^{Y} \delta(x_0 - y_i) \right] \mathcal{N}(x; x_0, \sigma^2 \mathbf{I}) dx_0 \\ & = \frac{1}{Y} \sum_{i=1}^{Y} \int_{\mathbb{R}^d} \mathcal{N}(x; x_0, \sigma^2 \mathbf{I}) \delta(x_0 - y_i) dx_0 \\ & = \frac{1}{Y} \sum_{i=1}^{Y} \mathcal{N}(x; y_i, \sigma^2 \mathbf{I}) \\ $

The loss of the Denoiser is:

$$\mathcal{L}(D; \sigma) = E_{y \sim p_\text{data}} \mathbb{E}_{x \sim p(x, \sigma)} \| D(x; \sigma) - y \|_2^2$$

$\mathcal{L}(D; \sigma) = \int_{\mathbb{R}^d} \frac{1}{Y} \sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) \| D(x; \sigma) - y_i \|_2^2 dx$

The solution is obtained by setting the gradient w.r.t $D(x; \sigma)$ to zero:

$$0 = \nabla_{D(x;\sigma)} \mathcal{L}(D; x, \sigma)$$ $$0 = \nabla_{D(x;\sigma)} \left[ \frac{1}{Y} \sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) \| D(x; \sigma) - y_i \|_2^2 \right]$$ $$0 = \sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) \nabla_{D(x;\sigma)} \left[ \| D(x; \sigma) - y_i \|_2^2 \right]$$ $$0 = \sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) \left[ 2 D(x; \sigma) - 2 y_i \right]$$ $$0 = \left[ \sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) D(x; \sigma) \right] - \left[ \sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) y_i \right]$$ $$D(x; \sigma) = \frac{\sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I}) y_i}{\sum_{i=1}^Y \mathcal{N}(x; y_i, \sigma^2 \mathrm{I})}$$ $$D(x; \sigma) = \frac{\sum_{i=1}^Y \mathrm{exp}(\frac{||x-y_i||^2}{-2\sigma^2}) y_i}{\sum_{i=1}^Y \mathrm{exp}(\frac{||x-y_i||^2}{-2\sigma^2})}$$ $$D(x; \sigma) = \sum_{i=1}^Y \mathrm{softmax}(\frac{||x-y_i||^2}{-2\sigma^2}) y_i$$

As shown in the last Equation in this comment, the ideal solution for the denoiser $D(x)$ can be found as the weighted sum of all clean data in the dataset.

Q1: Figure 1 has nothing to do with resolutions.

A1: Thanks for pointing it out, we acknowledge an error in our initial submission regarding the referencing of figures. Specifically, the discussion of resolutions is visualized in Fig 2 instead of Fig 1. We apologize for any confusion this may have caused and have corrected this reference in our revised manuscript. We appreciate the reviewers bringing this to our attention and ensuring the accuracy of our work.

Q2: This paper is not the first to quantify the noise level of the forward process of the diffusion model. What is the contribution of this paper compared to [1]?

A2: We have revised the relevant claim in the introduction section. Thanks for your suggestion. We note that prior work, P2 [1], proposed a simple heuristic partition of noise levels based on SNR metric: coarse (SNR 0 ~ 1e^-2), content (1e^-2 ~ 1), clean-up (1 ~ 1e^4).

Unlike our WSNR metric, SNR is neither consistent in different resolutions, nor consistent in latent space. Our WSNR is the first power spectrum based metric, which is consistent for noise level across multiple resolutions in RGB space and latent space.

However, our quantization metric is different from P2. We summarize the difference as follows:

Besides the WSNR metric, our paper contributes uniquely by:

• We significantly improve the performance of diffusion models, via our novel WSNR-Equivalent training noise scheduler. Specifically, our method decreases the FID score by 3.6 on FFHQ-256x256 dataset, and achieves the same FID scores on ImageNet-256 and 512 for UViT-M (#Params: 287M) as those of UViT-L (#Params: 287M).
• We analyze that the ODE sampling noise schedule should be data-driven. Consequently, we propose estimating the integration interval based on the average data distance.
• We explore the relationship between the number of evaluation points and generation quality. Our findings lead us to develop a NFE-guided sampling noise schedule.
• Our sampling schedule refines the FID of pre-trained CIFAR-10 and FFHQ-64 models from 1.92 and 2.45 to 1.89 and 2.25, respectively, utilizing 35 network evaluations per image.

Thus, our paper offers new insights and advancements in the study of the noise schedule of diffusion models.

[1] Choi et al., Perception Prioritized Training of Diffusion Models, CVPR2022

Q5: The proposed method is hardly reproducible.

A5: We will release code to reproduce the results.

Q6: Typos and connection. Use one-letter variables in algorithms for readability. Put titles on the axes in the figures for readability.

A6: In the revised version of the draft, we fix typos and improve the connection between sentences. Besides, we make the algorithm and figures more readable. We will keep polishing our draft for the final version. Thanks for your suggestion.

Q7: What is the number of evaluation points?

A7: Evaluation points are the unique timesteps at which the diffusion model makes predictions. The 2nd Heun's adopts the start point and end point to estimate the 2nd derivative. Therefore, the end point of the current timestep is identical to the start point of the next timestep. For example, when employing the 2nd Heun method for sampling, which uses 18 timesteps, the Number of Function Evaluations (NFE) is calculated as 2x18 - 1 = 35. Please refer to this code segment for more details. However, the model actually performs evaluations at only 18 unique timesteps. We refer to these 18 points as evaluation points. In the case of the midpoint method, all NFEs are performed at unique timesteps, so NFE equals the Number of Evaluation Points (NEP).

Q8: How much is the difference in wall clock between 35 network evaluations with the proposed method and typical number of network evaluations with other methods?

A8: There is no difference in wall clock between 35 network evaluations with the proposed method and typical number of network evaluations with other methods.

For a UNet with a base channel of 128 and channels per resolution set to 1-2-2-2, having 4 ResNet blocks at each resolution, the computational cost for each neural network evaluation is 41.72 GFlops when the output shape is (1, 3, 64, 64). The remaining computational cost involves less than 100 instances of element-wise addition or multiplication operations, amounting to less than 0.0012288 GFlops.

Therefore, our wall clock time is the same as other methods under the identical NFE.

Q9: This paper proposes a training method and a sampling method. How do they affect performance when applied together?

A9: To prove the compatibility of our training and sampling methods, we evaluate our trained model on FFHQ-256 using our proposed sampling method. The table shows that our sampling method consistently lowers the FID score of the trained model, indicating improved performance since a lower FID score is desirable.

We understand the importance of ensuring that our paper meets the high standards of ICLR and would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

Your further guidance and feedback will be important to us and we are committed to making any additional improvements as needed.

Thank you very much for your time and consideration. We look forward to your response.

Dear Reviewer d8X8,

We appreciate the diligent efforts put into reviewing our work.

Like you, we believe that the writing improvement and readability are important.

We have revised the manuscript and would greatly appreciate it if you could confirm whether the revisions made align with your expectations and satisfactorily resolve the issues previously highlighted.

We would happily address any other questions.

Thank you, The authors

Dear Reviewer d8X8,

Just a reminder, there are only 2 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors

Dear Reviewer d8X8,

Just a reminder, there are only 4 hours left for the rebuttal.

After this time, we will not be able to participate in the discussion any further.

We are waiting for your response. We would happily address any other questions.

Best Regards,

The Authors