Unified Perspectives on Signal-to-Noise Diffusion Models

1. The reported improvements in performance, such as slight reductions in FID scores, are minimal in many cases (e.g., 3.39 vs. 3.43 on FFHQ). These marginal gains raise concerns about whether the proposed methodologies justify the increased complexity introduced by the framework.

2. The experimental validation relies on relatively low-resolution datasets, including CIFAR-10 (32×32), FFHQ (64×64), AFHQv2 (64×64), and ImageNet (64×64). The limited resolution restricts the generalizability of the findings, as the proposed methods are not tested on higher-resolution or more complex datasets.

3. The paper is heavily builds on prior work by Kingma et al. (2021) and Song et al. (2020a). The reported advancements offer only minor improvements over these existing frameworks, leading to questions about the novelty and impact of the contribution.

• The main weakness of the paper is the lack of the authors' insights on the benefits of using the theoretical contributions and the lack of experimental validation of them.
• At this point, analysis of experimental results is lacking in some respects. For now, it looks like Section 4 is just paraphrasing facts that can be seen in the figures.
  • Could you provide a more detailed explanation of Figures 1, 2, and 3? Is the existence of a better hyperparameter an unconditional tendency over datasets/models? If not, what kind of conditions apply?
  • In theory, many previous studies have shown that this can be a special case, and as far as I know, those studies (e.g. VDM++) have reported performance on high resolution data. Are there any experimental results and analysis of hyperparameter search on high-resolution data?

• Despite the claim of presenting a comprehensive framework, the scope of the work appears quite limited. While the paper aims to present a unified perspective, the introduction of numerous variables and complicated notation makes the theoretical framework difficult to follow. Unfortunately, it does not seem to succeed in building a clear and cohesive theory. Additionally, there appear to be several similar studies that are not adequately referenced.

• At first glance, the work appears mathematically intensive, but the complexity seems to stem mainly from cumbersome notation and calculations rather than any mathematically nontrivial arguments. Decorative math without substantial purpose is unnecessary.

  • Edited after Chair's suggestions
    • For instance, SNR is defined as $\lambda(t)$, yet the paper also introduces another notation, $SNR(t)$. This redundancy adds unnecessary complexity and increases the cognitive load on the reader.
    • The origin or derivation of the transition function $\Phi(\tau, t)$ is unclear and needs further explanation.
    • Theorem 1 appears too trivial, serving merely as an introduction of notation. It seems to be a simple variable transformation and does not warrant being presented as a theorem. Instead, it could be streamlined into the main text without the excessive formality. The same critique applies to Theorem 2, which merely rephrases known results using the authors' notation, lacking any substantive contribution.
    • Another example is the discussion that describes the Euler-Maruyama method with time discretization. It would suffice to state that an SDE of the form $dz_t = f_t dt + g_t dB_t$ is discretized as $z_{t+1} - z_t = f_t \Delta t + g_t (w_{t+1} - w_t)$.
    • The authors claim that Theorem 3 provides a stronger result compared to existing work; however, the improvement is marginal and lacks nontrivial insights. The parameter $\rho$ is introduced, yielding a known result when $\rho = 0$ or $\rho = 1$. Yet, this formulation appears straightforward if one is familiar with the derivation process. In fact, I once have derived a similar formula several years ago. This generalization is not particularly challenging, and the significance of this generalization remains unclear in the paper, as it only elaborates on the specific cases of $\rho = 0$ and $\rho = 1$.
    • Despite the formality, the definition of signal-to-noise space is not given, which the authors claim the contribution of the paper.
    • Overall, the paper contains an abundance of mathematical ornamentation, where existing results are merely rephrased in the authors' notation. It is difficult to identify any substantial contributions, and the writing style would benefit from greater clarity and conciseness.

• The number of cited references is notably low compared to similar papers, and several important works on diffusion models are missing. In addition, more thorough citation of fundamental literature on SDEs, stochastic process or nonequilibrium thermodynamics would be appropriate. These references are arguably more important than citations like Bishop or Hyvarinen for the context of this work.

  • Edit after Chair's suggestions
    • For example, relevant literature can be found in the survey compiled at [https://github.com/chq1155/A-Survey-on-Generative-Diffusion-Model]. Some of the theoretical literature will be closely related to this paper.
    • See also https://arxiv.org/abs/2209.00796.
    • I think that Ho's work already provides a comprehensive general framework.

• The example images presented in the appendix might have been compelling in 2020, but in 2025, they no longer offer anything particularly interesting or novel.

• In its present form, the paper is not well-organized and reads disjointed. Without having intimate knowledge of each cited work, it is at many points unclear what parts are repeated from prior work and what is actually novel, since reproduced and (apparently) newly derived formulae are intermixed in writing without clear distinctions. For example, Theorem 2 / Eq. (4) is reproduced from Zhang & Chen (2022, Eq. 16), but is not clearly marked as such, instead it is only noted as being "inspired by (Zhang & Chen, 2022)", and the following formula Eq. (5) seems to be simple substitution of the authors' SNR-based expression, as far as I can tell. This could be improved by stating clearly which theoretical results are reproduced and which are novel, and what the motivation and purpose of extending, generalizing, or modifying each known result is.

• Furthermore, it is unclear what the goal of each section is, as many subsections of the main section 3 do not seem to be connected and do not form a cohesive story. Specifically, the authors derive (or reproduce) six theorems and one corollary in section 3, but in section 4, they only make use of the corollary (which is a corollary of theorem 3) to perform any experiments. This makes it unclear what the purpose of theorems 4, 5, and 6 in this work is, especially since they also do not seem to be connected to each other. This could be improved by rewriting sections towards a more coherent story linking the results, and by adding experiments related to theorems 4-6, if possible. For example, for theorem 5, the authors could actually list a grouping of S2N models that "induce the same diffusion model on the signal-to-noise space", as stated as a possibility in lines 378-380. For theorem 6, the authors could attempt an empirical comparative analysis of the mutual information of various S2N models, e.g. the pretrained diffusion models used for the other experiments.

• The SNR-based viewpoint, which according to the title should be a key feature of the paper, does not seem really necessary for theorems 1 to 3 and corollary 1 (which, as noted, are however the only theoretical results that are used in the experimental section), unless I am missing something. This viewpoint - as used in this work - also seems to generally increase the complexity of the involved expressions rather than reduce it, whereas the reduction of complexity was one key advantage of originally introducing this viewpoint in Kingma et al. (2022). Please clarify the use of this viewpoint for theorems 1-3 and corollary 1.

Reviewer comment: The SNR-based viewpoint, which according to the title should be a key feature of the paper, does not seem really necessary for theorems 1 to 3 and corollary 1 (which, as noted, are however the only theoretical results that are used in the experimental section). This viewpoint - as used in this work - also seems to generally increase the complexity of the involved expressions rather than reduce it, whereas the reduction of complexity was one key advantage of originally introducing this viewpoint in Kingma et al. (2022). Could the authors clarify how the SNR-based perspective contributes to each theorem and the corollary?

• Some expressions and derivations listed in the main text seem overly extensive, showing trivial steps that may better be left in the Appendix, for example in lines 253-269. Doing so could free up space for writing a more cohesive storyline and for easier understanding of the key contributions.

• The experimental section seems highly limited in scope and generalizability:

  • The authors only evaluate a small selection of models and use FID as the single metric to compare models. For models based on the datasets FFHQ and the conditional model of ImageNet, only marginal differences in FID are found (Table 2). Only for AFHQ, the proposed choice $\gamma = 0.026$ seems to make a real difference for FID. For CIFAR-10 32x32, the gains from $\gamma \neq 0$ are more clear (Fig. 1), but this dataset is very limited by its resolution and size, making it generally unsuitable for drawing deep conclusions.
  • The best choice of the hyperparameters $\gamma, \delta$ (and perhaps $\rho$) may well be very different for other datasets or even different network architectures, but the experimental section lacks any discussion of this.
  • The results in Figure 2 and 3, setting $\rho = 1$ (as in standard SDE sampling) and varying $\delta, \gamma$ are not significant at all, with an FID score of 1.68 vs. 1.70 for conditional ImageNet and between 2.36 and 2.40 for unconditional CIFAR-10.
  • This seems to suggest that, beyond results on the limited CIFAR-10 set (Figure 1), there are no relevant gains from the proposed sampler, and it requires expensive grid-searches for up to three hyperparameters, severely limiting its practical usefulness.
  • The trustworthiness of the presented empirical results could be improved by evaluating on other datasets or tasks, other pretrained models (not only those trained with the EDM formalism) and/or comparing against other samplers proposed in the literature, at similar NFE.