Provable Efficiency of Guidance in Diffusion Models for General Data Distribution

Thanks for your valuable questions. Below we provide a detailed point-by-point response.

Experiments on real dataset. We have added experiments on the ImageNet dataset to validate our theory. Please refer to our response to Reviewer EKNn of Experiments on real dataset.

Further explanations of the main theorem. We agree that a more detailed explanation of Theorem 3.1 would enhance clarity. In the revised version, we will add the explanation of main analysis idea (see ''Explanation of main analysis idea'' in our response to Reviewer 5U8H), detailed comparison with prior theory (see ''Comparison with prior works'' in this response), explanation of IS (see ''Issue of Inception Score'' in this reponse), and experiments on Imagenet dataset (see ''Experiments on real dataset'' in our response to Reviewer EKNn) to address this comment.

Clarification of the toy example. Originally, we state that ''$P(p_{1|X_0}(1|Y_0^w)\ge p_{1|X_0}(1 | Y_0^0))$ is less than $1$, which indicates the guidance may not achieve uniform improvement in classifier probabilities'' for the toy example. To make it more accurate, we will revise it to ''$P(p_{1 | X_0}(1| Y_0^w)\ge p_{1| X_0}(1| Y_0^0)) < 1$ for any $w < 10$, which indicates ...'', since we have numerically confirmed that this holds true for all $w < 10$, which covers the practical range of $w$ in typical applications.

Relation to prior works on toy examples. We agree with the existence of prior analyses of classifier guidance in GMMs. However, our work focuses on different aspects compared to these studies. We will include a detailed comparison in the revised version, as detailed in next bullet.

Comparison with prior works. Actually, we have cited and briefly compared with these existing works, which mainly focus on specific classes of distributions like GMMs. In contrast, our main contribution lies in providing a more general theoretical analysis. Nonetheless, we agree that a comparison between our theory--- when applied to specific distributions--- and prior works would further clarify our contributions. We will include a new section in the revised manuscript for comparison:

''Existing works focus mainly on specific classes of distributions like GMMs, while our work provides a more general theoretical analysis. Below, we compare our findings with prior works when restricted to specific distributions.

In [1], the authors demonstrate that $p_{c | X_0}(c | Y_1^w)\ge p_{c | X_0}(c | Y_1^0)$ holds under specific conditions, while we show that this inequality does not always hold. In addition, [2] argues that guidance can degrade the performance of diffusion models, as it may introduce mean overshoot and variance shrinkage. In contrast, our result shows that guidance can improve sample quality by generating more samples of high quality. Furthermore, [3] shows that classifier guidance can not generate samples from $p(x | c)^{\gamma}p(x)^{1-\gamma}$ for GMMs and establishes its connection to an alternative approach, i.e., the single-step predictor-corrector method, whose effectiveness in this specific setting remains unclear. In contrast, we directly analyze and demonstrate the effectiveness of CFG.''

Issue of Inception Score (IS). We offer two clarifications:

1. The reviewer is absolutely correct that IS is not a perfect metric and has known limitations in evaluating sample quality.

2. We disagree that ''IS is outdated'' and argue that IS is still one of the most widely used and informative metrics for assessing sample quality. The precise evaluation of sample quality remains an open problem. Despite the concerns raised in [4], IS continues to be employed as a key metric in recent works on classifier guidance [5] and classifier-free guidance [6] for diffusion models, due to the absence of an alternative metric that is demonstrably superior to IS. Our choice to consider IS aligns with the original studies that introduced diffusion guidance. In addition, most issues outlined in [4] come from the inaccuracy in the empirical estimation of IS, which doesn't apply for the theoretical consideration, as our results are derived using the true conditional probability.

We will clarify the reasonability for using IS in the revised manuscript:

''Although some practical limitations of IS have been identified [4], it remains a commonly used metric for evaluating sample quality in the study of diffusion guidance [5,6]. Moreover, in our theoretical analysis, we use the true conditional probability, which addresses the estimation issues discussed in [4].''

Typos. We have fixed them in the revised version.

[1] Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models

[2] What does guidance do? A fine-grained analysis in a simple setting

[3] Classifier-Free Guidance is a Predictor-Corrector

[4] A Note on the Inception Score

[5] Diffusion Models Beat GANs on Image Synthesis

[6] Classifier-free diffusion guidance

We thank the reviewer for the constructive feedback! Below, we provide a detailed point-by-point response.

Explanation of main analysis idea. In the revised version, we will add the following explanations to better communicate the high-level ideas behind the analysis:

''A glimpse of the main analysis idea. First, this result comes from the key observation that the function of reverse process, $p_{c|X_{t}}(c|X_t)^{\rm -1}$, forms a martingale, as stated in Lemma 3.2, which is established through a careful decomposition of $p_{c|X_{t}}$ and $p_{X_{\tau}|X_{t}}$. Next, the guidance term $s_t(x|c) - s_t(x)$ in classifier-free guidance (CFG) aligns with the direction of $-\nabla p_{c| X_t}(c| x)^{-1} = p_{c| X_t}(c| x)^{-1}[s_t(x|c) - s_t(x)]$, which makes us expect that adding the guidance at time $t$ can decrease $\mathbb{E}\_{x_{\tau} \sim X_{\tau}}\big[p_{c| X_{\tau}}(c| x_{\tau})^{-1} | X_t = x\big]$ for all $\tau \le t$. Finally, to achieve the desired result, particular care must be taken in handling first- and second-order differential terms with respect to $t$ for the process $p_{c| X_{1-t}}(c| Y_t^w)^{-1}$ due to its randomness nature, which is completed in Section 4.2 based on the technique of Ito's formula.''

Extension to DDIM. As pointed out by the reviewer, our analysis is specific to DDPMs. In particular, our analysis relies on the martingale property stated in Lemma 3.2, which relies heavily on the property of $p_{X_{\tau}|X_{t}}$ in DDPMs and can not be applied for DDIMs. Extending our framework to DDIMs remains an open question due to the absence of this key property. We will add a remark in Section 3.1 of the revised version to explicitly state this limitation.

Influence of discretization. We fully agree that our continuous-time analysis for CFG can not be immediately extended to the discrete-time setting with only small KL divergence error. As the reviewer suggests, if $p(c | x)^{-1}$ is uniformly bounded, one could establish the desired result. However, such an assumption is too strong. Instead, we adopt a weaker condition: we assume that $\mathbb{E}[(p(c | Y_1^w)^{-1}-1)1(p(c | Y_1^w)^{-1} > \tau)]$ is small for some threshold $\tau > 0$. We have also verified this assumption numerically on the Imagenet dataset. We will include the following new result in the revised version:

''The sampling process (5) with the learning rate schedule (19) satisfies

$

\mathbb{E}[p(c | Y_1^w)^{-1}] \le \mathbb{E}[p(c | Y_{\overline\alpha_1}^{w, \mathsf{cont}})^{-1}] + \mathbb{E}[(p(c | Y_{1}^{w})^{-1}-1)1(p(c | Y_{1}^{w})^{-1} > \tau)],

$

where $\tau$ is defined as the largest value satisfying

$

\mathsf{TV}(Y_{\overline\alpha_1}^{w, \mathsf{cont}}, Y_1^w) \le \mathbb{P}(p(c | Y_{1}^{w})^{-1} > \tau).

$

This further implies the following relative influence from discretization, the ratio between the improvements of $Y_1^w$ and $Y_{\overline\alpha_1}^{w, \mathsf{cont}}$ over $X_{\overline\alpha_1} = Y_{\overline\alpha_1}^{0, \mathsf{cont}}$, obeys

$

\frac{\mathbb{E}[p(c | Y_{\overline\alpha_1}^{0, \mathsf{cont}})^{-1}] - \mathbb{E}[p(c | Y_1^w)^{-1}]}{\mathbb{E}[p(c | Y_{\overline\alpha_1}^{0, \mathsf{cont}})^{-1}] - \mathbb{E}[p(c | Y_{\overline\alpha_1}^{w, \mathsf{cont}})^{-1}]} \ge 1 - \frac{\mathbb{E}[(p(c | Y_{1}^{w})^{-1}-1)1(p(c | Y_1^w)^{-1} > \tau)]}{\mathbb{E}[p(c | Y_{\overline\alpha_1}^{0, \mathsf{cont}})^{-1}] - \mathbb{E}[p(c | Y_{\overline\alpha_1}^{w, \mathsf{cont}})^{-1}]}.

$

For different values of $\mathsf{TV}(Y_{\overline\alpha_1}^{w, \mathsf{cont}}, Y_1^w)$, we empirically validate the aforementioned assumption on the ImageNet dataset. Here we use $\mathbb{E}[p(c | Y_{1}^{0})^{-1}]-\mathbb{E}[p(c | Y_{1}^{w})^{-1}]$ as an estimate of $\mathbb{E}[p(c | Y_{\overline\alpha_1}^{0, \mathsf{cont}})^{-1}] - \mathbb{E}[p(c | Y_{\overline\alpha_1}^{w, \mathsf{cont}})^{-1}]$. The relative error $\frac{\mathbb{E}[(p(c | Y_{1}^{w})^{-1}-1)1(p(c | Y_1^w)^{-1} > \tau)]}{\mathbb{E}[p(c | Y_{1}^{0})^{-1}]-\mathbb{E}[p(c | Y_{1}^{w})^{-1}]}$ is presented in the following table for various values of the TV distance and $w$. The results indicate that the relative error remains small, particularly for practical choices of $w \ge 1$.

$

\begin{array} \hline \hline \mathsf{TV} & w=0.2 & 0.4 & 0.6 & 0.8 & 1 & 2 & 3 & 4\\ \hline 0.30 & 0.447 & 0.196 & 0.115 & 0.085 & 0.029 & 0.006 & 0.006 & 0.002 \\ 0.10 & 0.440 & 0.194 & 0.114 & 0.085 & 0.029 & 0.006 & 0.005 & 0.002 \\ \hline \end{array}

$

''

Assessment on real dataset. We have added experiments on the ImageNet dataset to validate our theory. Please refer to our response to Reviewer EKNn of Experiments on real dataset.

Thanks a lot for the reviewer's helpful comments and valuable feedback. Below, we provide a point-by-point response, which has also been incorporated into the revised version of our manuscript.

Experiments on real dataset. Notice that classifier-free guidance (CFG) was originally validated on the ImageNet dataset [1], where Inception Score is typically computed using the Inception v3 classifier [2]. To further support the practical applicability of our method, we have included an additional numerical experiment using a pre-trained diffusion model [3] on the ImageNet dataset, which we believe provides stronger empirical validation than other datasets.

The results demonstrate that guidance improves sample quality by decreasing the averaged reciprocal of the classifier probability rather than achieving uniform improvement across all samples, thereby validating our theory. We adopt the guidance scale range in [1] from $0$ to $4$; specifically, in our experiments, we use $w = \{0.2, 0.4, 0.6, 0.8, 1, 2, 3, 4\}$. The numerical results are as follows:

$

\begin{array}{c|cccccccccccc} \hline \hline w & 0.2 & 0.4 & 0.6 & 0.8 & 1 & 2 & 3 & 4 \\ \hline {P(p_{c|X_0}(c|Y_1^w) \geq p_{c|X_0}(c|Y_1^0))} & 0.70 & 0.75 & 0.78 & 0.80 & 0.82 & 0.85 & 0.85 & 0.86 \\ -\mathbb{E}[p_{c|X_0}(c|Y_1^w)^{-1}] &-140 & -74 & -47 & -36 & -14 & -3.6 & -3.4 & -2.0 \\ \hline \end{array}

$

Paper structure. We agree that presenting the numerical experiments as a separate section would enhance readability. we will restructure the manuscript by placing the toy experiment and the newly added results on the ImageNet dataset in a separate section following the main results.

Practical implementation. One potential application of our theoretical analysis comes from its implication that guidance may reduce sample quality for a small subset of samples. This observation could motivate further research into adaptive guidance mechanisms that ensure a more uniform improvement in sample quality. We will include a remark following Theorem 3.1 to discuss the potential implementation of our theory:

''Theorem 3.1 states that guidance improves the averaged reciprocal of the classifier probability rather than the classifier probability of each individual sample. This suggests that while guidance improves overall sample quality, it may lead to a decline in quality for a small subset of samples. This insight encourages the development of adaptive guidance methods that address this issue and achieve more uniform performance gains, which is a potential practical application of our theory.''

Influence of guidance scale $w$. We agree that extremely large values of $w$ can degrade the performance of CFG. However, this phenomenon is consistent with both our theoretical and experimental results. In practice, the performance of a diffusion model is typically evaluated based on two key metrics: diversity and sample quality. CFG attains a trade-off between these two metrics, as noted in [1]. In this paper, our main focus is on the influence of guidance on sample quality, particularly in relation to the Inception Score. Our results align with practical observations in the sense that the generated samples are more closely adhere to the conditional distribution with larger $w$. In addition, previous studies have shown that an extremely large $w$ can severely reduce sample diversity and negatively impact realism. While this is an important consideration for practical applications, it is not the main focus of this work. We will add the following remark to avoid confusion:

''In practice, performance of diffusion models is commonly evaluated by two metrics: diversity and sample quality. This study primarily focuses on the sample quality measured in a similar way as the Inception Score, which increases with $w$. However, prior work [1] has demonstrated that large values of $w$ can significantly reduce sample diversity, leading to unsatisfactory performance in real-world applications.''

Related works. Reference [4] proposed to use a bad version of the model for guiding diffusion models. We will cite it properly in our revision.

[1] Classifier-free Diffusion Guidance

[2] Rethinking the Inception Architecture for Computer Vision

[3] https://github.com/CompVis/latent-diffusion

[4] Guiding a Diffusion Model with a Bad Version of Itself