Losing dimensions: Geometric memorization in generative diffusion

1. The ideas in this paper—such as the spectral gap, critical time, "active" samples, generalization during sampling of memorized models, and geometric memorization—are too dense and interwoven, making the paper somewhat difficult to follow. Prioritizing key results could improve clarity.

2. The paper offers limited insight beyond the observed phenomena, as memorized diffusion models are impractical in real-world applications and have been extensively studied in prior work (such as https://arxiv.org/abs/2402.18491). This work appears largely as a technical extension of that research.

My main issue with the paper is that I don't understand what its contributions are. I do not even know what is being studied: is it the difference between the empirical and the population score? Is the spectrum of the score Jacobian? These two things seem connected in the authors' mind, but I did not see a clear connection. Several results are mentioned in the text and seemingly not used anywhere else. For instance, equation (9) comes from another paper (and in a different setting, as it assumes that $F$ is random, but this is not stated in the text of this paper) and it is not reused. It makes it difficult to see what is the big picture that supposedly emerges from the analysis.

The theoretical results are not clearly stated and do not support the claims made in the text.

• The authors state that "we find that, under some conditions, subspaces of higher variance are lost first due to memorization effects". I did not see where this was shown.
• Similarly, at several places in the text, it is mentioned that the $x$-dependence of the condensation time and other quantities are crucial. However, all numerical experiments are either on Gaussian data, for which the Jacobian is independent of the input, or show the average Jacobian.
• It is stated that "the experimental curves obtained from the empirical score look consistent with the theory". I disagree with this claim. First, the three panels of Figure 4 show qualitatively different behavior. Second, it is not clear to me what predictions the theory was making in the first place, apart from highlighting the presence of gaps at several dimensions (which can be predicted from simple linear algebra).
• In Figure 6, the authors write "The estimated diminsionality tend [sic] to increase with the dataset size, suggesting a phenomenon of geometric memorization." I do not see why an increase of dimensionality would be linked to memorization. On the opposite, shouldn't memorization be characterized by a collapse of dimensionality?

There are several typos. I also suggest using a "sequential" colormap for figures 5 and 6 to enhance readability.

1. The experiments are limited. In fact, I am slightly concerned about how non-linear manifolds behave in practice. Could you please provide the same analysis conducted in Figure 5, but instead of using linear manifolds embed that in a high-dimensional sphere?

2. The formal explanation of 5.1 can be greatly improved! Please explicitly define the higher and low-dimensional subspaces that are referred to in the rest of the text. In other words, please explicitly define what these subspaces are in terms of $F$. To my understanding, these subspaces correspond to the subspace obtained by eigenvectors of $FF^\top$ that have eigenvalues above different thresholds. Nonetheless, it is important to explicitly define it to avoid confusion.

3. For better accessibility for the ML community, this is a paper that can benefit a lot by having a section such as "a primer on random energy models". For example, being an ML researcher with little knowledge of statistical physics, I was unable to follow the exposition in Appendix C leading up to eq. (15) in the limited timeframe for review.

Extra Points

1. Figure 1 is not referenced. It would be useful to incorporate it into the explanation, especially when the latent manifolds are defined.

2. Please add the explicit derivations of eq. (8) to the Appendix of the paper. What I am getting is different. Also, this equation does not match eq. (14) in the supplementary material.

3. Related to 5.1, this is probably a comment for the supplementary provided, but Kamkari et al. [A] and Tempczyk et al. [B] do an experiment which is somewhat different but gets the same conclusion. Please refer to the paragraph "FLIPD is a multiscale estimator" in Section 4.1 and Appendix D.3 of [A] + Figure 1-b of [B], which is precisely an experiment on multivariate normal distributions with specific covariance eigenspectrums that matches the setup in 5.1. Please consider adding these two to the discussions in both papers.

4. Please add the explicit derivation that leads to eq. (19) to the appendix. Especially, the case where $F$ is non-diagonal.

References

[A] Hamidreza Kamkari, Brendan Leigh Ross, Rasa Hosseinzadeh, Jesse C Cresswell, and Gabriel Loaiza-Ganem. A geometric view of data complexity: Efficient local intrinsic dimension estimation with diffusion models. In Advances in Neural Information Processing Systems, volume 37, 2024.

[B] Tempczyk, Piotr, et al. "A Wiener process perspective on local intrinsic dimension estimation methods." arXiv preprint arXiv:2406.17125 (2024).

The analyze in section 5 is itself very interesting and meaningful, however, I have questions on some of the definitions. Furthermore, I think there exists huge gap between the theoretical results and the practice. i list my questions as follows:

(i) First of all, I have doubt on the definition of the manifold (equation 3). Can you explain why the manifold is defined as the points that has 0 score? This is a very restricted definition since in practice, the $x_t$ along the reverse sampling trajectory never satisfy this assumption unless $t$ is extremely close to 0. Shouldn't we define the manifold as the intermediate noisy distribution $p_t(x)$ ? For the experiment results on natural images, can you clarify whether you are calculating the Jacobians at $x_t$ that have zero score ?

(ii) Without this special manifold assumption, the singular vectors with zero singular values no longer represents the tangent space. Can you clarify whether/how the tangent space interpretation holds when analyzing models trained on natural images ?

(iii) In contrast, the work by Stanczuk does not have this special manifold definition and the dimension of the manifold is calculated in a very different manner rather than just counting the number of zero singular values. I suggest that the authors compare their approach more directly to Stanczuk's method, explaining the key differences and potential implications.

(iv) The definition of memorization is very different from what people care about in practice. In practice, people define memorization as diffusion models generate images that can be found in its training dataset [1]. To be more specific, given a noisy image $x^*_t$ and score function $s(x^*_t,t)$, the denoised version of $x^*_t$ can be written as $x^*_t+\sqrt{t}s(x^*_t,t)$. and memorization happens when $x^*_t+\sqrt{t}s(x^*_t,t)$ is close to training images of the diffusion models. Can the author address how their definition of memorization relates to or differs from the practical definition I've described ?

(v) I think the Jacobians matrix alone cannot fully explain the memorization phenomenon since it doesn't have enough information on the score function $s(x^*_t,t)$. Notice that equation (4) only holds when the deep network is bias free. In general, equation (4) should be written as:

$

s(x^+p,t)\approx s(x^,t)+J(x^*,t)p.

$

I think memorization depend more on the property of $s(x^*,t)$ itself, rather than its Jacobian matrix. Can the author explains how their choice of focusing on the bias-free deep network might limit the analysis?

(vi) The analysis on empirical score has huge gap between practice since recent work[2] has shown that denoisers in practice cannot learn such empirical score function. For this reason, the emprical score function is too ideal and cannot explain the empirical results on natural images (section 6.3).

(vii) The paper claims that memorization causes the model to losing some directions of the tangent spaces. They support this by observing that the time-dependcy gaps in the Jacobian spectrum are lost. However, it should be noted that for diffusion models trained on different number of images, the singular vectors of the Jacobians change significantly as well, which can be observed in the following link https://github.com/LabForComputationalVision/memorization_generalization_in_diffusion_models/blob/main/notebooks/Demo_UNet_CelebA80x80.ipynb. Therefore, "losing dimension" is not an accurate description since the subspaces themselves change significantly. The current theory cannot explain this change in subspaces.

[1] Wang, Wenhao, Yifan Sun, Zongxin Yang, Zhengdong Hu, Zhentao Tan, and Yi Yang. "Replication in visual diffusion models: A survey and outlook." arXiv preprint arXiv:2408.00001 (2024).

[2] Zeno, Chen, Greg Ongie, Yaniv Blumenfeld, Nir Weinberger, and Daniel Soudry. "How do minimum-norm shallow denoisers look in function space?." Advances in Neural Information Processing Systems 36 (2024