Generalization through variance: how noise shapes inductive biases in diffusion models

We thank the reviewer for their time and thoughtful comments. See below for a point-by-point response.

(i) The authors claim that generalization occurs if and only if V-kernel is nonzero since otherwise the reverse sampling process can only produce training examples. However, this definition seems restricted since when diffusion models generalize, they not only differ from the PF-ODE dynamics, but generate "high quality" samples as well. In other words, the V-kernel should have certain benign properties. This point is not adequately addressed in the current paper.

This is a great point, and we strongly agree that it is important to address. We will address this point in two ways. First, we will add a new section to the main text ("Generalization through variance: consequences and examples") that explicitly discusses various "benign" properties of the V-kernel that support reasonable generalization. These properties include the fact that it tends not to add noise far from training data, that it tends not to increase the effective dimensionality of the training distribution (e.g., if training data lie in a plane, to good approximation so will the generalized distribution), and that training data is more likely to be sampled than novel states. The new section will also contain various numerical experiments that validate our results in simple 1D and 2D settings.

Second, we will add a new appendix ("Benign properties of generalization through variance") that contains additional discussion of this point, including more math.

(ii) In section 5 the authors derive the analytical form of the V-kernels under three circumstances. However, the resulting equations (9), (13) and (17) are really hard for me to interpret. Can the authors provide more explanation on the unique properties of these different V-kernels and how they shape the distributions corresponding to the reverse sampling process?

We agree that the results of our original draft are fairly formal, and that it would be extremely helpful to illustrate them using a series of concrete examples. The new main text section we are writing shows precisely what the V-kernel looks like for a variety of simple models trained on 1D and 2D point clouds, and in particular shows how generalization through variance is affected by (i) training set structure (specifically, gaps between training data and duplications of training data), (ii) anisotropy in the forward process, and (iii) the feature set used (in the case of linear models). Throughout this section, we will provide additional commentary linking the experimental results to our math results.

(iii) As a researcher focuses on empirical works, I don't find the results of this paper very useful. I think the paper could benefit from a further discussion on how the results can help improving practical diffusion models.

Although our focus is more 'scientific' (i.e., understanding an observed phenomenon) than 'practical' (i.e., improving model performance), we plan to add comments related to improving practical models to the Discussion, and hope that the new experiments we add to the main text are also helpful. One finding of the experiments is that feature-related inductive biases appear to matter a lot in practice, with the choice of feature set, and how it interacts with the proxy score covariance, sometimes strongly affecting the form of generalization through variance.

We thank the reviewer for their time and helpful comments. Before we address them, we would like to note that there is something of a qualitative distinction between two questions regarding how diffusion models generalize:

1. Given a finite number of samples M from some data distribution, what do diffusion models learn? (Relatedly: do models learn to simply regurgitate one of the $M$ samples? Do they learn something simple, like a kernel density estimate, or not?)
2. In the limit as $M \to \infty$, do diffusion models learn the ground truth distribution? (Relatedly: if yes, how does convergence depend on $M$?)

These questions are both interesting and worthy of study. In our work, we focus almost exclusively on the first question. Our comment  “At present, there is arguably no theory that describes…” is intended to refer to the fact that, while a large number of papers address question 2 (including the Li et al. and Fu et al. papers you provide as examples), to our knowledge there is a theoretical gap regarding question 1.

The distinction between the two may be somewhat confusing, as there are two asymptotic limits one can consider. One is the limit in which the model has access to an infinite number of samples from the data distribution (e.g., an infinite number of pictures of real dogs). The other is the limit in which, during training, the model has access to an infinite number $P$ of training examples, which consist of noise-corrupted versions of data distribution samples. We take the $P \to \infty$ limit but not the $M \to \infty$ limit, whereas most authors take both to infinity. This drastically changes the nature of the corresponding mathematical problem, and we think that this emphasis contributes to the novelty of our work.

See below for a point-by-point response.

Weaknesses

1. The paper's main weakness is the lack of empirical validation of the theoretical claims. While the theoretical intuition is clear, the paper would be greatly strengthened by including simulations or experiments that demonstrate the practical impact of the "generalization through variance" phenomenon.

We agree that this is a major weakness, and are working on a new main text section ("Generalization through variance: consequences and examples") before the Discussion to remedy this. The new section considers the impact of generalization through variance (GTV) in the context of a variety of 1D and 2D examples. While these examples are admittedly toy given that real diffusion models typically involve high-dimensional data, these simple examples are easy to visualize, they make the effects of GTV easy to see and understand, and they allow many quantities of interest to be straightforwardly computed. (They are also simple enough that we can consider a large number of models, since training is not an issue.)

We show that generalization, as we claim, generally happens in regions where proxy score covariance is high, which corresponds to regions between multiple training examples. The details of this are modulated somewhat by the choice of feature maps in the linear case: for example, we show that rotating the data distribution and/or feature maps can yield a slightly different kind of generalization.

We thank the reviewer for their time and helpful comments. See below for a point-by-point response.

Weaknesses

1. The paper assures that the generated samples are different from the training samples due to the noise in the score proxy, but this does not seem to explain why the generated samples are meaningful in some sense (e.g., have similar features to the training samples). If one assumes a ground truth distribution then this is obvious, but the authors emphasize that they refrain from that, but it is not clear what replaces this assumption (beyond the generated sampled being different from the training samples).

This is a good point: how do we know if a given generalization of training data is 'reasonable' without a ground truth? In the revised version of the paper, we address this issue in three ways.

First, we describe various generic properties of the V-kernel that make it in some sense 'benign' (to borrow a phrase used by reviewer AyYq). These properties include the fact that it tends not to add noise far from training data, that it tends not to increase the effective dimensionality of the training distribution (e.g., if training data lie in a plane, to good approximation so will the generalized distribution), and that training data is more likely to be sampled than novel states. These properties are described both at the beginning of a new main text section ("Generalization through variance: consequences and examples") and a new appendix ("Benign properties of generalization through variance").

Second, in the aforementioned new main text section, we depict a number of examples of generalization through variance (GTV) in 1D and 2D settings. Although these settings are admittedly toy, they are useful because they are easy to visualize, the results are easy to interpret, and it is straightforward to numerically compute all quantities of interest. These examples provide empirical evidence that the generalization achieved is in some sense reasonable. We underline the point that there are multiple ways to reasonably generalize a point cloud, though, by showing different types of generalization of the same 2D point cloud depending on the model and hyperparameters used.

Third, we include a new appendix section (and limited discussion in the main text) on generalization error, as measured in terms of the KL-divergence between the ground truth distribution and the learned distribution, in settings where there is a known ground truth. It is hard to analyze except in special cases, but an interesting empirical insight is that error tends to fall faster as a function of the number of ground truth samples for diffusion models than comparable Gaussian-based kernel density estimates.

2. Except for the appearance of a non zero covariance matrix in noise of the reverse diffusion process, the expressions in Propositions 5.1 and 5.2 are somewhat implicit. For example, it is not obvious how easy it is to compute them, or how its form affects generalization.

This is a good point. We add additional commentary on computing them in practice as comments in the main text and relevant appendices. The V-kernel can be computed directly, since one only needs access to (i) the covariance of the proxy score (which we compute in Appendix B), whose computation involves computing a certain covariance; and (ii) the feature maps $\phi(\mathbf{x}, t)$ in the linear case. In the new main text section, we directly compute it in a number of cases. More generally, since the V-kernel has a form that depends on certain expectations, it is expected that it is also reasonably straightforward to compute in higher dimensions.

Questions

1. Line 67: What is exactly meant by “boundary regions” ? One example is regions that are close to multiple training points, but how these are defined in general ? As high likelihood regions ? This term is used repeatedly afterwards so it would be good to accurately define it.

More generally, we mean values of $\mathbf{x}$ and t for which $p(\mathbf{x}_0 | \mathbf{x}, t)$ is highly uncertain (above some threshold, say), which (via Bayes' rule) can be converted into a statement about the curvature/Hessian of the noise-corrupted likelihood (or log-likelihood) $p(\mathbf{x} | t)$. These are precisely regions between multiple training points in the discrete case, but this definition also makes sense for other types of training distributions.

We are adding a new appendix section describing what we mean by 'boundary regions' since the idea is central to our results, and since clarity about our usage of it will probably help other readers.