Towards a Theoretical Understanding of Memorization in Diffusion Models

I believe the paper is incorrect.

Unless I’m missed some assumption, proposition 1 is incorrect. Looking at the proof, it looks like a mistake occurs in eq78: the left hand side does not equal the covariance matrix of Z|Y since the expectation is taken over all of Z instead of Z|Y. Also a counterexample to the proposition exists: you can increase the variance of a set of points by removing samples at the mean.

I don’t see how Theorem 1 could possibly be correct in the general case. For example, let’s say a line in semantic space is mapped to a circle in sample space. Then if one travels along the straight line in semantic space, they will always recede in semantic space, while in sample space they will recede initially, but eventually return. The proof also appears to be problematic, in particular the change of variable looks incorrect (the Jacobians should definitely not cancel out when you perform a change of variable).

Again, perhaps there was some assumption I missed conditioned on which these theorems are true - if the authors can satisfactorily address this I am of course willing to reevaluate this paper. Otherwise, these errors cast serious doubts on the claims of this paper.

The paper has the following weaknesses:

1. Typos and Formatting Errors: The paper contains several minor typographical errors, such as incorrect numbering and missing spaces (e.g., "line 045" and "line 367"). These errors, while minor, could detract from the paper's professional quality and should be corrected to maintain the paper's credibility.

2. Clarification Needed on Downstream Impacts: The connection between the proposed method's impact on memorization and its benefits for downstream task performance (e.g., improved FID scores in image generation) is not well-established. Clarifying whether SIDE enhances performance in practical applications would strengthen the paper's impact and relevance.

Theoretical side:

1. The authors appear to assume $p_\theta$ is a multivariate normal throughout the proofs, which seems extremely inappropriate and is obscured by the theorem presentation. Please clarify if this is incorrect.

2. Eq. (1) is basically the sum of KL divergences between Gaussians centred at training points and the learned distribution. Let's assume that the training datapoints are well apart and the learned distribution is a mixture of Gaussians centred at each of the training datapoints (perfect memorization). Now let us take the limit $\epsilon \to 0$, while at around the memorized point, the term $p_\theta / q(x, x_i, \epsilon)$ converges to $1/|\mathcal{D}|$ at all other points it will blow up to infinity. Therefore, as $\epsilon \to 0$ the metric blows up to infinity and it cannot detect an obvious case of memorization. The metric, in and of itself, is useless (which is probably why the authors do not use it in practice). However, it can aid in proving Theorem 2, which is trying to show a much easier phenomenon: increased specificity in the input distribution around a cluster might lead to memorization.

3. The authors use the term "suffix training" without defining what it means. To my understanding, it follows from (6), which is basically maximum likelihood across all the conditions -- which is how conditional DPMs are trained --- but there is no explanation whatsoever, and there is no reason to believe that unconditional DPMs with "implicit" labels are trained this way.

4. What is the definition of the norm used in Eq. (8)? Upon reading it seems it is the matrix norm induced by the L2 norm, but it is not defined in the definition.

Practical Approach:

1. The metrics seem invented (AMS/UMS) and tuned with strange hyperparameters $\alpha$ and $\beta$. At least one additional metric which is widely accepted is required to determine whether SIDE is a valid approach.
2. The only baseline that is compared against is "random", at least the baselines from Wen et al. [A] or Ross et al. [B] should be added to the analysis to make SIDE convincing.

1. The theoretical part of the paper does not consider exact diffusion models but instead focuses on an encoder-decoder pair trained via maximum likelihood estimation. I believe there is a gap here that needs to be addressed. The authors' setting significantly simplifies score-based models. While I understand this simplification aids in proving theoretical results, discussing this gap is necessary.

2. There is a disconnect between the theoretical analysis and the proposed method. The theory mainly focuses on how informative labels (explicit or implicit) can lead to stronger memorization of training data (as discussed in Proposition 1 and Theorem 2). However, the SIDE method primarily aims to extract information from unconditional DPMs. This paper implicitly assumes that the training process of unconditional DPMs extracts classification information from the training data, forming implicit informative labels and lead to memorization. While this may be well-known from the authors' perspective (e.g., references [1] and [2]), I believe this logical premise needs to be explicitly discussed.

3. The experimental section lacks direct validation of the theoretical findings. Attacking unconditional DPMs by artificially learning class information of the training data does not convincingly demonstrate that unconditional DPMs acquire memorization through implicitly learning data labels. Since additional classifier information is introduced, such attacks may result from data leakage rather than solely from the memorization of DPMs. Therefore, additional experiments are necessary. I suggest that the authors should at least verify the impact of different labels on conditional DPMs using a small-scale dataset like CIFAR-10, exploring scenarios with normal labels, clustered labels (as in reference [3]), etc.

4. The setting of the proposed TDKD method is somewhat peculiar. The authors implicitly assume that the training dataset of the DPMs under attack is inaccessible, yet they assume that a pre-trained time-independent classifier on this training dataset is available. This seems like a deliberately crafted special scenario. Why is the time-independent classifier on this dataset available while the time-dependent classifier is not? In the experimental section, the original training data is fully accessible. I do not understand why the authors start from a time-independent classifier and use TDKD for distillation instead of directly training a time-dependent classifier. This raises doubts about the effectiveness of the SIDE attack. In diffusion models trained on non-public data, how can we obtain a time-independent classifier?

5. In the introduction, the authors state that one of their contributions is: "We introduce a novel metric to measure the degree of point-wise memorization in DPMs." However, I did not see the specific measurement methods for Pointwise Memorization and Semantic Memorization in the experimental section. Instead, the authors use Average Memorization Score and Unique Memorization Score. What prevents them from utilizing the metrics proposed in the theoretical section in practical experiments? This is crucial for evaluating the contributions of the theoretical part based on these metrics.

References:

1. Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
2. Your Diffusion Model is Secretly a Zero-Shot Classifier
3. Hierarchical Clustering for Conditional Diffusion in Image Generation