Generalization for Discriminator-Guided Diffusion Models via Strong Duality

Thank you for your review and positive comments on our paper. Please see the comments regarding your questions:

Question: Although the theoretical results guarantee the convergence and generalization power of the diffusion model, it does not explain why the refined diffusion model is better than the non-refined diffusion model by comparing them with the same metric.

Answer: In Theorem 4, we have that the IPM $d_{\mathcal{H}}$ between refined SGM and the data is equal to $d_{f,\mathcal{H}}$ between SGM and the data minus a positive quantity. Note however that by Equation (8), $d_{f,\mathcal{H}} \leq d_{\mathcal{H}}$ is a strictly smaller divergence. Thus, we can write Theorem 4 and 6 as thinking as the refined SGM is closer to the data compared to just the SGM and the data in the same metric.

Question: It would be better if the authors could provide guidance for choosing the discriminator based on their theoretical results.

Answer: One direct implication from our results is to use discriminators whose lipschitz constants are small (close to 1). We do this since the Rademacher complexity of such a class can be bounded.

Question: Could the author explain why finding $h^{*}$ in the optimization of $\mathsf{D}_{f,\mathcal{H}}$ is equivalent to a binary classification problem?

Answer: This is due to the link shown in Equation (10) where $\mathsf{D}_{f,\mathcal{H}}$ is equivalent to minimizing a proper binary loss. Thus finding h^{*} is equivalent to finding the minimizer of the binary classification problem. We will make this link clearer in the updated version.

Thank you for your review and noting the strength of our contributions. Please see our response to each of the weaknesses described:

Question: “However, in the paper, the authors claim that their findings advocate for discriminator refinement. Can you be clearer on this point?”

Answer: We would like to clarify an important aspect of our paper mentioned above in the general comment (1): the distribution $\mu^{\mathcal{H}}$ is precisely SGM+D (from Kim et. al.) as shown in Example 2 of the paper. In particular, the framework for a general choice of $f$ and valid sets of discriminators $\mathcal{H}$ generalizes SGM+D. In Theorem 4 and Theorem 6, we can see that the left-hand side is IPM between SGM+D and data is upper bounded by SGM and data minus a positive term. Note that D_{f,\mathcal{H}} <= D_{\mathcal{H}} (Equation 8 in the submission) and thus under the same divergence D_{\mathcal{H}}, we can see that SGM+D is closer to the data than SGM is to the data. Thus, Theorem 4 can be abstracted as:

Gap between SGM+D and data = gap between SGM and data - positive quantity

Therefore advocating for using discriminator refinement. Thank you for this comment, we believe adding this discussion to clearly separate this point will clarify our contribution.

Question: Is this theorem only useful because it allows the decomposition and further analysis of the IPM

Answer: Yes the impact of Theorem 1 is the decomposition of IPM and the fact that the “refined” diffusion model (SGM+D) is the minimizer of the primal problem which is of independent conceptual interest.

Question:Could you derive practical recommendations from your theory? Does it give particular insights or ideas of improvements for SGM+D models? For example, could you predict the impact of the choice of $f$ on the behavior of the generative model, whether in terms of distribution fitting or generalization?

Answer: The main practical guidance is the fact that there is a trade-off of using discriminative models and generalization as spelled out by the equality from point (1) in the general comment. Thus, we recommend the use of regularized discriminators such as 1-Lipschitz functions to refine the diffusion model. The work of Kim et. al. do not discuss the effects of using restricted discriminator sets. Additionally, our framework allows us to derive new losses to refine diffusion models. We show that binary cross entropy loss corresponds to the framework of Kim et. al. however other proper composite losses such as the square loss.

Question: It is not exact and it should be clearly stated that you characterize "an exact distribution". Answer: Thanks for bringing up this point. In fact, from our proof we can see that this choice of $\mu^{\mathcal{H}}$ is indeed the unique minimizer. However we understand the inconsistency in our wording has it made it unclear, we will make this point more consistent throughout the paper.

Question: In the current version, the lack of structure makes the reading more complicated.

Answer: Thank you for these points, we will divide them into subsections as you suggest to help the reader better appreciate the purpose of each section.

Question: Theorem 2 and Theorem 3 should rather be Corollaries of Theorem 1. Calling them theorems is overselling since they are direct applications/reformulations of Theorem 1

Answer: Indeed, we agree and will name them Corollaries.

Thank you for the valuable suggestions you have given us to improve clarity in our paper. In particular, we believe that using the notation you suggest (SGM+D) vs (SGM) and abstracting Theorem 4 and 6 is an effective way to help the reader understand our work. We hope we have sufficiently answered your concerns and these updates to the presentation will compel you to increase your score.

We would like to thank the reviewer for providing their extensive feedback on our work and noting the importance of the problem we solve. We note that there are problems with the clarity and exposition with the paper, and would like to clarify some questions about the theoretical validity.

Question: “A typical example of this problem is Theorem 6. Firstly, since the choice of $\mathcal{H}$ influences the IPM on the left-hand side, it is difficult to compare different choices of $\mathcal{H}$.”

Answer: Thank you for pointing this out. Infact, the way generalization plays a role is more or less the same as the story in [1] where we have an IPM objective on the LHS and a Rademacher complexity on the right-hand side. In a similar vein to their discussion, we require $\mathcal{H}$ to be ‘discriminative’ so that IPM on the LHS is meaningful while ensuring the Rademacher complexity is bounded. Consider again the abstracted version of Theorem 6 from point (1) in our general comment:

D(refined SGM, data)<= D(SGM, data) - discriminative ability + Rademacher complexity

We can see that as $\mathcal{H}$ becomes more discriminative, the “discriminative ability” term will increase however the Rademacher complexity will increase. Thus, we have a trade-off narrated exactly by Theorem 6.

Question:”I would like the authors to clarify how this results "close[s] the generalization gap" (Section 1).”

Answer: When we refer to ‘closing the generalization gap”, we are referring to the fact that Theorem 1 proves that the gap between discriminator refined diffusion models $\mu^{\mathcal{H}}$ is at most the gap between the original diffusion model and the data (which existing work has bounded in many related work) minus the discriminative ability term + the Rademacher complexity.

Question: “The paper scarcely discusses the only prior work in generalization of diffusion models by Oko et al. (2023).”

Answer: The main theorem decomposes the IPM between refined diffusion model and the data into two terms. The first is a divergence between the data and the original diffusion model (before refinement). All existing work such as Oko et. al. decomposes this exact term (divergence between the data and the original diffusion model) whether it is for Wasserstein distance, KL-divergence or Total Variation. The divergence we show can be upper bounded by these as discussed in Section 5. Therefore, our results can be used in combination of all existing work to help decompose that term.

Question: “Furthermore, I find that the paper underplays in the introduction the original contribution of Kim et al. (2022) on discriminator guidance. They did theoretically show the relevance of their approach by showing that it closes a gap between the score estimation and the true score. While I believe that the results presented in the submission go further, I would like to better understand their added value.”

Answer: In the original paper of Kim et. al., while they do show a theoretical result, it focuses on recovery of the discriminator-refined model against the data samples - here there are two key differences. First, we show a connection to strong duality and how the refined distribution in their work coincides precisely with the optimal distribution in the primal problem of GANs and secondly, we show a generalization bound where the Rademacher complexity appears.

Question: “It seems implicitly accepted that $\mu^{\mathcal{H}}_{T,\upvartheta}$ is the same as $\mu^{\mathcal{H}}$ as in Theorem 2, why is it the case?”

Answer: $\mu^{\mathcal{H}}_{T,\upvartheta}$ is defined to be the refined diffusion model after first finding a binary classifier between the diffusion model at time $T$ and data $h^{*}$ and then using the transformation in Eqn (12) to construct it. Therefore, notationally it is the same as $\mu^{\mathcal{H}}$. Thank you for pointing this out, as we will make it much clearer in the updated version.

Question: Assumption 1 does not seem valid,

Answer: Assumption (1) - (3) are taken verbatim from [2] where it is stated that “Assumption 1 is standard and has been used in the prior works.” We would like to point out that this Assumption is only needed in the analysis of $D_{f,\mathcal{H}}$ which can be bounded by $D_{\mathcal{H}}$ or $D_f$ (Equation 8 from main paper) and thus we can utilize many of the existing works to bound this quantity instead including Oko (which holds when $\mathcal{H}$ is the set of $1$-Lipschitz functions).

We thoroughly appreciated your feedback, especially the updates on the clarity of Section 5. We hope that we have addressed your concerns sufficiently and compel you to increase your score.

[1] Zhou, Denny, et al. "On the Discrimination-Generalization Tradeoff in GANs." (ICLR 2018)

[2] Chen, Sitan, et al. "Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions." (ICLR 2023).

Thank you for your review and noting the generality of our results to other areas of ML.

We agree that the paper is quite dense and we will split Section 5 into subsections to better improve the flow of the paper. Thank you for pointing out the redundancies, we will incorporate this feedback to strengthen the paper. Regarding your question, the $f$-divergence quantity while makes a negative (since it is minus) quantity in lowering the upper bound, it is intractable and would be difficult to compute.