Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion

From the questions raised, it appears that there are some major misunderstandings. In what follows, we do our best to clarify them, and we urge the reviewer to reread some part of our work and reassess their evaluation. In our end, we will do our best to improve the presentation based on the Reviewer's feedback.

"Have you used any clean images?": Yes, we use clean data but only for times $t\leq t_n$ of the diffusion, where $t_n$ is a hyperparameter to be controlled. For times $t\geq t_n$, we only use corrupted data and the Ambient Score Matching objective. The corruption to time $t_n$ happens once, before training, to ensure that there is no information leakage about the clean images at these times.

"what is S?": The meaning of S is explained on page 7, Lines 377-382 in the first column. To reiterate, for each generated image, we measure its similarity, S, (inner product), with the nearest neighbor in the dataset. Tables 1 (and Table 2) report percentages of the dataset that have similarity S greater than certain thresholds. In our rebuttal, we further report mean and median values -- see response to Reviewer Einp.

"the relationship between FID and memorization issues and the selections of $t_n$ remains unclear". Figure 1 shows the achieved FID and memorization for different values of $t_n$. There is an extended discussion on the choice of $t_n$ on page 7 of our submission.

"but the true values of t_n are not provided": this is a critical misunderstanding. We have access to clean data, and we intentionally corrupt it for some diffusion times to avoid memorization. Please refer to Algorithm 1 and Section 3 (Method, page 4) of our submission, where we verbally explain this algorithm. See also Choices for $t_n$ on page 7.

About differential privacy-based methods: This is a great remark. We will add this discussion. Briefly, the notion of privacy is stronger than the notion of memorization. Privacy guarantees would mean that any adversary that has access to the model could not extract samples used for the training of that model. Our method does not guarantee that: we show that sampling with our model leads to outputs that resemble less to the training points. That does not exclude the possibility that an adversary with access to our model could recover the training points. Since we have relaxed expectations/guarantees, we also enjoy high performance.

On the Gaussian assumption in Lemma 4.2: The goal of Lemma 4.2 is to motivate why the ambient optimal score solution ``leaks” less information at time $t_n$ compared to running vanilla DDPM from time $T$ until time $t_n$. While we believe that it holds for more general distributions, generalizing the result comes with some technical challenges and does not add much additional value to our goal of motivating why the ambient optimal score solution leaks less information.

Section 4.2: Building a theoretical framework for understanding memorization in machine learning is quite challenging, and there is limited amount of work trying to address it. The seminal work of Feldman provides a natural instance (with some arguable caveats) of the learning problem where optimal classification requires memorization.

Our work aims to take a step in understanding memorization in diffusion models, not by solely using Feldman’s framework but by taking a two-step approach:

First, we adapt the setting of Feldman (which is a well-accepted) to more general loss functions (capturing e.g., standard score matching losses). The message of this general framework is roughly speaking the following: if the distribution of the frequencies of the subpopulations is heavy-tailed, then to achieve optimal generalization loss, the algorithm has to make the loss at all the points seen exactly once zero; otherwise, it pays a penalty \tau_1 for any not fitted point.

Second, to make use of this general framework, we rely on how diffusion models are trained. We view the above general framework in many noise scales. As we explain e.g., in Lemma 4.5, even if the original dataset has the heavy-tailed structure of Feldman’s model, as the noise scale increases (which is the core idea of diffusion-based generation), the tails will become lighter since the populations start to merge. This is where our framework is specialized in diffusion models and not in Theorem 4.3.

Combining the above two observations, the theoretical part of Section 4.2 gives evidence that avoiding memorization in the high-noise regime is feasible. Our main contribution in this part is the general observation that the tail of the distribution of the frequencies depends on the noise scale, and this is a property that depends crucially on how diffusion models are trained. This illustrates our conceptual contribution on why, for the high-noise regime, memorization could be avoided.

We hope this rebuttal clarifies things and we remain at the Reviewer's availability if there are further concerns.

We thank the reviewer for careful reading, for their feedback and suggestions!

Lemma 4.2 being about a single point: We preferred to present the statement for 1 point since it already presents the advantage of ambient diffusion compared to vanilla DDPM: Given a generation of m points from each of the two models, the mutual information is larger. We agree with the comment that a larger training set is desirable, and this is why we present such a result in the Appendix (we refer to it right after Lemma 4.2).

On the difference from Feldman’s work: We believe that our main contribution in this part is the general observation that the tail of the distribution of the frequencies depends on the noise scale. This observation is specific to the way that diffusion models are trained and does not appear in Feldman’s work because it focuses on the multiclass classification. To make this observation more rigorous, we adapted Feldman’s framework with a more general loss function (which should be independent of the mixing coefficients): in fact, this adaptation is not technically challenging and we do not claim novel technical challenges; it is instead a generalization of the previous proof. However, the novelty comes from a conceptual perspective. As the noise level increases, the penalty \tau_1 in the Informal Theorem 1 is decreased because the tails become lighter.

To summarize, this part of the paper illustrates our conceptual contribution on why for the high-noise regime, memorization could be avoided.

On the readability of appendix and main body: We thank the reviewer for that comment. Indeed, the lack of space made the presentation a bit dense. We will use the extra page in the Camera Ready to improve the presentation of the work.

We hope that our response addresses any remaining concerns and that the Reviewer will support the acceptance of our submission. We remain at the Reviewer's availability in case there are additional concerns to be addressed.

We thank the reviewer for their valuable reviews and suggestions!

On no of duplicates in figure 2: By the no of duplicates, we mean the percentage of generated samples whose similarity to their nearest neighbor in the training set is greater than 0.9. The nearest neighbor is found using the cosine similarity in the DINO space.

As per the reviewer’s request, we also calculated the mean and median of the similarity in the same setting as Figure 1 (300 training samples from FFHQ). We will include these results in the updated version.

On FID using Dinov2: As per the reviewer’s suggestion, we performed additional experiments to evaluate the quality of our model with FID using Dinov2. We will add these results with a plot in the updated version.

On clarification about the sampling process: Our method is a training-time mitigation strategy, and therefore, no modifications are made during the sampling. For $t\geq t_n$, we replace the original dataset with a noisy version of it where each datapoint has been replaced (once) with a noisy realization. This step corresponds to the creation of $S_{t_n}$ at step 1 of Algorithm 1 (Appendix page 12). The creation of $S_{t_n}$ is very important as it ensures the reduced information about the clean distribution for the learning at $t\geq t_n$ (see Lemma 4.2). This step leads to decreased memorization. For $t\leq t_n$, we train as usual. We will clarify this in the updated version.

On theoretical results: We will do our best to improve the presentation of our results, using the one additional page for the Camera Ready. Here we provide an intuitive explanation of the results, and will include such a discussion in the updated version to improve the readability.

Lemma 4.1 proves that the optimal score learned at time $t_n$ is a gradient field that points towards the noisy training set. This lemma is formal and can be seen as the analogue of the optimal score solution for standard DDPM in a noiseless dataset.

Lemma 4.2 motivates why this solution ``leaks’’ less information at time $t_n$ compared to running vanilla DDPM from time $T$ until time $t_n$. This property is formalized using the mutual information. Additionally, we needed the Gaussianity assumption to make it formal since for the Gaussian, we can derive explicit expressions for mutual information.

Section 4.2 explores connections between our work and the work of Feldman, which focuses on the multiclass classification. Theorem 4.3 presents the analogue of Feldman’s result with a more general loss function. However, the message is essentially the same: the more heavy-tailed the distribution of the frequencies, the higher the penalty of not memorizing (in our setting, memorization corresponds to making the score objective small). This is formally presented in Lemma 4.4, which comes from the work of Feldman and shows that the penalty term \tau_1 depends on the tail of the distribution of the mixing coefficients. Finally, we illustrate in Lemma 4.5 that the noise scale of the diffusion process affects the parameter \tau_1 since, as the noise increases, the subpopulations merge and the mixing coefficients add up (hence becoming lighter). To make this intuitive argument fully formal, we provide the result for the most standard mixture model, namely Gaussian mixture, and the result should extend to any mixture family since noise addition is a contracting operation.

On larger datasets: For the unconditional image generation setting, memorization becomes negligible for larger datasets. Per Reviewer’s Request, we trained on a slightly bigger dataset. For FFHQ 5k samples, we get the following results:

As seen, our method is still a little better in terms of memorization with the same FID as the Baseline, but the improvement is smaller compared to more data-limited settings.

For larger datasets, memorization is still an issue if there is text-conditioning. Our results in Section 5.2. show improvements over the baselines for this regime as well.

We hope our rebuttal clarifies the Reviewer's concerns and that the Reviewer will strongly support the acceptance of our work.

We thank the reviewer for their Review and suggestions.

On the blurry output of the diffusion models: Figure 3 contains a noisy image $x_t$ and the learned model’s approximation of $E[x_0 | x_t]$. Intuitively, this expectation represents an average over all possible clean images $x_0$ that could have produced the noisy image $x_t$. As a result, the predicted image is expected to appear blurry. This is validated by the second row (outputs of a diffusion model trained on 52k images): a model trained on the full dataset has blurry predictions, as it should. On the other hand, when a model memorizes the training data, it is not modeling correctly the conditional expectation – instead, it is overconfident, i.e., it believes that one of a few clean images could correspond to a given noisy input $x_t$. In this case, it outputs a sharper (cleaner) image, which reflects memorization. The figure shows that our algorithm (Row 4) matches the blurriness level of the oracle model (52k training images) even though our model is only trained with 300 images.

On combining two existing algorithm and novelty: The novelty of the algorithm is not in the loss, but in the way we create the data we apply the loss to. Let us clarify further. For times $t\geq t_n$, we replace the original dataset with a noisy version of it where each datapoint has been replaced (once) with a noisy realization. This step corresponds to the creation of the set $S_{t_n}$ at step 1 of Algorithm 1 (Line 612, Appendix Page 12). The fact that we create $S_{t_n}$ once before the launch of the training (instead of recreating at each epoch) is very important as it ensures that there is reduced information about the clean distribution for the learning that happens at times $t\geq t_n$ (see also Lemma 4.2). This step in the algorithm leads to decreased memorization.

Comparison with existing works:

• In the papers “Consistent Diffusion Meets Tweedie” and “How Much is a Noisy Image Worth?”, the authors are dealing with corrupted datasets and they are using Ambient Score Matching loss to train with the corrupted data.
• In DDPM, clean data is used for all times using the regular objective.
• In this paper, we have clean data, but we intentionally corrupt them for some times to reduce memorization. At the same time, we do not want to reduce performance, so we still use the clean data for certain diffusion times. The idea of intentional corruption (and the way its done in Algorithm 1) so that we achieve both reduced memorization and good performance is the algorithmic novelty of this work.

On the related work and additional baselines based on those works: We want to bring to the Reviewers’ attention that two out of the three papers that we are being asked to compare against were released last month (26 Feb'25, and 5 Feb'25). We submitted this work in January 2025, so we do not think it is fair to compare against these two baselines. Besides, neither of these two works has experiments with images. That said, we are happy to cite and discuss these works in the updated version, and we will do so.

We thank the Reviewer for bringing the paper “Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention” to our attention and will cite it in the updated version. We attempted to compare against this method, but the code for training-time memorization mitigation is not provided in the official repository. We even tried implementing the approach ourselves, but couldn’t replicate their results. We contacted two of the authors about the issue, but as of today we have not received a reply. If the code becomes available, we will gladly compare.

On integrating the method with other diffusion methods: Our method can work with any training architecture, training loss type (e.g., x0-prediction, epsilon-prediction, flow-matching, etc), and sampling method (EDM sampler, DDIM, DDPM sampler, etc). All the results in the paper are obtained by using the EDM codebase for all these components (architecture, training loss, sampler).

As per Reviewer’s request, we experimented with other pipelines, and we present the results for (FID, Memorization) below when the training is performed using 300 training images for the FFHQ data set.

In the light of these new experiments and clarifications, we hope that the Reviewer will support the acceptance of our submission.