EraseDiff: Erasing Data Influence in Diffusion Models

Thank you for your thoughtful feedback and suggestions, which we have now incorporated into the new revision of the manuscript. New results are included from pages 20 to 22, with changes highlighted for ease of tracking. We address your questions below. We are happy to continue the discussion in case of follow-ups.

Q5.1 Is there a theoretical guarantee to do the optimization alternatively? It seems not necessary in this setting, as we may do it completely in a separate way, i.e., do the maximization first to completely scrub the forgetting classes and then fine-tune to make up the performance on the remaining dataset. Which way would be better, and why?

The nature of first forgetting and then retraining suggests possible degradation as either the model is seriously damaged by the forgetting step or even the possibility of relearning the forgetting data as a result of retraining (especially for the classes with large resemblances). To put this to test, we performed an experiment on CIFAR-10 by first performing NegGrad and then relearning using the remaining data. Results, denoted as Two-steps, are shown in Table 5 and Figure 18 on page 20 in the revised manuscript. We observe significant deterioration in generating $\mathcal{C}_r$, while improvement in scrubbing $\mathcal{C}_f$. This formulation is observed to be even worse than the multi-task formulation. Interestingly, we observe that the generated images for $\mathcal{C}_f$ may contain relevant information after retraining. Overall, our proposed formulation leads to a robust and simple optimization framework. Thanks for the suggestion, further studies on effective disjoint optimization remain as future work which we have reflected in the conclusion of our revised manuscript.

Q5.2 We also see a drop in the quality of generated figures from Table 2, and the proposed model is worse than finetune from Table 3.

Finetune has a better FID on remaining classes $\mathcal{C}_r$ but fails to scrub the information about the forgetting classes $\mathcal{C}_f$, as shown in Figure 4 on page 9. Compared with other methods, our method achieves the best trade-off between removing information about $\mathcal{C}_f$ and preserving model utility over $\mathcal{C}_r$ with good efficiency.

Q5.3 Minor typo: page 3, “the variational bond” above equation (1).

Thanks! We have since corrected it.

Thank you for your thoughtful feedback and suggestions, which we have now incorporated into the new revision of the manuscript. New results are included from pages 20 to 22, with changes highlighted for ease of tracking. We address your questions below. We are happy to continue the discussion in case of follow-ups.

Q4.1 Design more experiment (and have more discussion) on random (or not so well-labeled) subset, demonstrate model efficacy.

Thanks for the suggestion. We conducted an additional experiment to perform sample unlearning on an unconditional DDPM with pre-trained weights from HuggingFace. Examples of generated images from the unscrubbed model and our scrubbed model are shown in Figure 21 on page 22. FID scores over 50K images are 10.85 and 10.99 for the unscrubbed model and our scrubbed model, respectively. We can see that the quality of generated images only drops by around 0.14 in terms of the FID score. It is true that it is hard to verify the unlearning effect in this setting. We'll investigate how to evaluate in a sample-wise unlearning setting in future work.

Q4.2 Evaluating unconditional diffusion model utility with respect to specific sub-dataset can be hardly well-defined, so try to narrow-down and specify the problem you want to tackle in a more guided way (such as to only some specific conditional diffusion model).

Thanks for the suggestion. We clarified this in the conclusion of our revised manuscript.

Q4.3 Try more types of condition( or context).

Thanks for the suggestion. More generated image examples with different settings can be found in the Appendix (unlearning different classes for CIFAR-10 and UTKFace, and different attributes on CelebA and CelebA-HQ). We have since added Table 7 on page 21 to show the performance in different settings.

Thank you for your thoughtful feedback and suggestions, which we have now incorporated into the new revision of the manuscript. New results are included from pages 20 to 22, with changes highlighted for ease of tracking. We address your questions below. We are happy to continue the discussion in case of follow-ups.

Q3.1 The diffusion unlearning can be naturally defined as a multitask problem.

Indeed, this formulation is possible; however, it would not perform better than our proposal empirically. To provide further insights, we performed extra experiments on the CIFAR-10 dataset using conditional DDIM (please see Tables 4 and 5, Figures 17 and 18 on page 20 in the revised manuscript). Formulation as a multi-task problem and solving it via scalarization as a single-level optimization is shown as SO in Table 4 and Figure 17. We observed that this would lead to a significant drop in the image quality on $\mathcal{C}_r$ when boosting to forget $\mathcal{C}_f$ by large. As shown in Figure 17 ($\alpha=0.5$), when the generated images yield coarse information for $\mathcal{C}_f$, the quality of generated images for $\mathcal{C}_r$ drops significantly. Overall, we believe formulating as vanilla multi-task problem is not beneficial. Thanks for the suggestion; further studies on effective multi-task formulation remain as future work, which we have reflected in the conclusion of our revised manuscript.

Q3.2 $**I.**$ The inner objective of this work strives to make the diffusion model incapable of generating meaningful images corresponding to $\mathcal{C}_f$. The rationale behind defining sample unlearning is unclear. $**II.**$ A more intuitive approach would be to ensure that the model, when trained with both the remaining and the forgetting data, has parameters equivalent to those obtained when trained solely on the remaining data after the unlearning process.

Our objective is defined with respect to the forgetting data $\mathcal{D}_f$ and the remaining data $\mathcal{D}_r$, $\mathcal{D}_f$ could be randomly selected from the training data, i.e., sample unlearning. We didn't evaluate sample-wise unlearning because, currently, it is hard to evaluate the unlearning effect when performing sample-wise unlearning for diffusion models, as Reviewer ${\color{orange}LbvE}$ mentioned. Per your comment, we conducted an experiment to perform sample unlearning on an unconditional DDPM with pre-trained weights from HuggingFace (i.e., the forgetting data $\mathcal{D}_f$ is randomly selected from). Examples of generated images from the unscrubbed model and our scrubbed model are shown in Figure 21 on page 22 in the revised manuscript. FID scores over 50K images are 10.85 and 10.99 for the unscrubbed model and our scrubbed model respectively. We can see that the quality of generated images only drops by around 0.14 in terms of the FID score.

It's true that the scrubbed model with unlearning algorithms should be equivalent to the retrained model which uses the remaining data to retrain from scratch, which is the ideal unlearning algorithm. However, the removal of pertinent data followed by retraining diffusion models from scratch demands substantial resources and is often deemed impractical. Therefore, the model retrained solely on the remaining data is unknown to the unlearning algorithms. Instead, approximate unlearning algorithms (e.g., [1-2] and ours) achieve model scrubbing without the need for retraining. This is accomplished by updating the model parameters in a negative direction relative to the samples or concepts intended for erasure.

We evaluated the Weight Distance (WD) between the scrubbed models and the retrained model (please see Table 3 on page 8) in experiments. Table 3 reveals that although the NegGrad method has a Weight Distance (WD) of 1.3533, it is unable to preserve model utility. In contrast, our approach attains the smallest WD among effective unlearning algorithms, i.e., 1.3534. Importantly, it successfully erases information about the forgetting classes $\mathcal{C}_f$, while concurrently preserving data regarding the remaining classes $\mathcal{C}_r$.

---

[1] Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. arXiv preprint arXiv:2303.07345, 2023.

[2] Heng, Alvin, and Harold Soh. "Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models." arXiv preprint arXiv:2305.10120 (2023).

Q2. Empirical evidence alone might not be convincing enough. Is there any theoretical support for the bi-level approach being better than other methods?

We are not aware of theoretical studies in that regard. However, we can characterize the solution of our algorithm as follows:

$**Theorem 1**$ (Pareto optimality). $\quad *The stationary point obtained by our algorithm is Pareto optimal.*$

$*proof.*$

Let $\theta^\ast$ be the solution to our problem. Because given the current $\theta_s$, in the inner loop, we find $\phi^K_s$ to minimize $\hat{f}(\phi, \mathcal{D}_f)=f(\theta_s, \mathcal{D}_f)-f(\phi, \mathcal{D}_f)$. Assume that we can update in sufficient number of steps $K$ so that $\phi^K_s= \phi^*(\theta_s) = \text{argmin}\_{\phi \mid \theta_s} \hat{f}(\phi, \mathcal{D}_f) = \text{argmin}\_{\phi\mid\theta_s} f(\phi, \mathcal{D}_f)$. Here $\phi \mid \theta_s$ means $\phi$ is started from $\theta_s$ for its updates.

The outer loop aims to minimize $\mathcal{F}(\theta, \mathcal{D}_f) + \lambda \hat{f}(\phi^*(\theta), \mathcal{D}_r)$ whose optimal solution is $\theta^\ast$. Note that $\hat{f}(\phi^*(\theta), \mathcal{D}_r)\geq 0$ and it decreases to $0$ for minimizing the above sum. Therefore, $\hat{f}(\phi^*(\theta^*), \mathcal{D}_r)=0$. This further means that $\hat{f}(\theta^*, \mathcal{D}_f) = \hat{f}(\phi(\theta^*), \mathcal{D}_f)$, meaning that $\theta^*$ is the current optimal solution of $\hat{f}(\phi,\mathcal{D}_f)$ because we cannot update further the optimal solution. Moreover, we have $\theta^*$ as the local minima of $\mathcal{F}(\theta,\mathcal{D}\_{f})$ because $\hat{f}(\phi^*(\theta^*),\mathcal{D}\_{f})=0$ and we consider a sufficiently small vicinity around $\theta^*$. $\square$

Furthermore and as discussed in [1] and [6], multi-task learning (MTL) could also be expressed as a bi-level optimization problem (\eg, MAML [4]). In our case, simultaneously optimizing the two objectives, as scalarization of an MTL, is indeed an instantiation of our proposed algorithm. Consider Eq.(9) in the main paper, repeated below for the comfort of the reviewer.

$$\quad \quad \theta^* = \theta - \zeta( \nabla\_{\theta}\mathcal{F}(\theta, \mathcal{D}\_{rs}) + \lambda \nabla\_{\phi}\hat{f}(\phi, \mathcal{D}\_f) ). \tag{3}$$

Here, $\hat{f}\phi, \mathcal{D}\_f) = f(\phi, \mathcal{D}\_f) - f(\phi^K, \mathcal{D}\_f)$ and in the inner loop, $\phi = \theta$. This shows that if $K=0$, then our solution recovers

$$\quad \quad \theta^* = \theta - \zeta( \nabla\_{\theta}\mathcal{F}(\theta, \mathcal{D}\_{rs}) + \lambda \nabla\_{\theta} f(\theta, \mathcal{D}_f) ). \tag{4}$$

This is indeed the scalarization of MTL. Checking Figure 2 (a) in [4], Figure 2 in [5], and Figure 6 in our paper demonstrate that increasing the number of inner update steps (i.e., $K$ in our case) could improve the resulting model performance.

Finally, while we acknowledge the importance of optimization, whether it be bi-level or other forms, it serves more as a tool to realize our main objective, the formulation of unlearning in diffusion models (DMs). Therefore, the lack of theoretical analysis on the optimization form should not overshadow the main theory (i.e., formulation of unlearning in DMs) our work offers. In essence, our paper addresses a fundamental problem by introducing new theories related to formulating unlearning, coupled with an engineering solution that involves developing the form of optimization. To overlook these aspects due to a lack of theoretical analysis on whether bi-level optimization is the best choice would not accurately represent the full scope and innovation of our work.

We thank Reviewers ${\color{cyan}JxPy}$ and provide further insights per the comments.

Q1. According to the conventional definition of bi-level optimization, as referenced in [1], this approach is not a bi-level optimization problem.

To be precise, a bi-level optimization is a hierarchical mathematical program where the feasible region of one optimization task is constrained by the solution of another task (see [6] for a detailed description). Our proposed formulation aligns with this definition. We aim to erase the influence of forgetting data $\mathcal{D}_f$ ($\min Alg(\theta, \mathcal{D}_f)$) while simultaneously preserving the model utility over the remaining data $\mathcal{D}_r$ ($\min \mathcal{F}(\theta)$). Also, we note that not all bi-level optimization problems necessitate distinct parameter sets (if ${\color{cyan}JxPy}$ refers to the lack of that in our formulation). A prominent example is Model-Agnostic Meta-Learning (MAML) [4]. MAML's objective is:

$$\quad \quad \theta_{ML}^{*} := \text{argmin}\_{\theta} \mathcal{F}(\theta), \hspace{1em} \text{where} \hspace{0.5em} \mathcal{F}(\theta) = \frac{1}{M}\sum_{i=1}^M \mathcal{L}(\text{Alg}(\theta, \mathcal{D}_i^{tr}), \mathcal{D}_i^{test}), \tag{1}$$

with $M$ denoting the number of meta-training tasks. The objective aims to achieve low generalization error on $\mathcal{D}_i^{test}$, while the inner objective is to learn knowledge of $\mathcal{D}_i^{tr}$. The objective of our method reads as follows:

$$\quad \quad \theta^{*} := \text{argmin}\_{\theta} \mathcal{F}(\theta), \hspace{1em} \text{where} \hspace{0.5em} \mathcal{F}(\theta) = \mathcal{L}(\text{Alg}(\theta, \mathcal{D}_f), \mathcal{D}_r):= \mathcal{F}(\theta, \mathcal{D}_r) + \lambda \hat{f}(\theta, \mathcal{D}_f), \tag{2}$$

where $\hat{f}(\theta, \mathcal{D}_f):=f(\theta, \mathcal{D}_f)-\text{min}\_{\phi|\theta}f(\phi, \mathcal{D}_f)$ represents the delta erasing gap. Specifically, $f(\theta, \mathcal{D}_f)$ indicates the erasing loss at $\theta$, while $\text{min}\_{\phi|\theta}f(\phi, \mathcal{D}_f)$ specifies the optimal delta erasing gap we can yield if we depart from $\phi = \theta$ and minimize the erasing loss to its minimum. Here, note that we define $\text{Alg}(\theta, \mathcal{D}_f):= \text{argmin}\_{\phi|\theta} f(\phi, \mathcal{D}_f)$.

Moreover, the above model-agnostic optimization problem states that we aim to reach the optimal model $\theta^*$ such that it can minimize the retaining loss $\mathcal{F}(\theta, \mathcal{D}_r)$ for $\mathcal{D}_r$, while also minimizing the delta erasing gap $\hat{f}(\theta, \mathcal{D}_f)$, i.e., from $\theta^*$, even if we try our best to erase $\mathcal{D}_f$, we cannot erase much.

Note that both the inner and outer objectives in our formulation and MAML are directed at optimizing the model parameters. Our formulation distinction from MAML lies in its focus: we are not seeking a model adaptable to unlearning but one that effectively erases the influence of data points on the model. We hope this establishes the missing link and we have revised our manuscript to further clarify and strengthen this aspect in the related work section.

Thank you for your thoughtful feedback and suggestions, which we have now incorporated into the new revision of the manuscript. New results are included from pages 20 to 22, with changes highlighted for ease of tracking. We address your questions below. We are happy to continue the discussion in case of follow-ups.

Q2.1 The proposed method primarily focuses on class-wise unlearning and may have limitations when applied outside this specific context (e.g., nudity removal, artistic style removal).

Our algorithm can perform unlearning outside of class-wise unlearning. Experiments on concept unlearning (unlearning the attribute 'Blond hair' or `Eyeglasses') can be found in Figure 5 (b) on page 9 in the main paper, in Figures 14 and 15 on page 18, and in Figure 16 on page 19 in the Appendix. Thanks, we have since added descriptions to the manuscript for better clarity.

Q2.2 How effective is the unlearning process when subjected to adversarial attacks?

Thanks for the suggestion. We conducted an additional experiment to evaluate the unscrubbed model, the retrained model, and our scrubbed model when subjected to adversarial attacks. This experiment has been added to Figure 22 on page 22. We observed that as the magnitude of the perturbation increases, the quality of the generated images decreases for all models, and the generated images by our scrubbed model still contain no information about the forgetting classes $\mathcal{C}_f$.

Thank you for your thoughtful feedback and suggestions, which we have now incorporated into the new revision of the manuscript. New results are included from pages 20 to 22, with changes highlighted for ease of tracking. We address your questions below. We are happy to continue the discussion in case of follow-ups.

Q1.1 How does the algorithm change when we do not have access to the remaining data $\mathcal{D}_r$?

In response to your comment and following the methodology outlined in [2], we conducted an additional experiment using Generative Replay (GR) to generate images $\mathcal{D}_r'$ as a substitute for $\mathcal{D}_r$ in the unlearning process. The findings from this experiment are presented in our revised manuscript, specifically in Table 6 and Figure 19 on page 21 in the revised manuscript. The results indicate that while image quality diminishes post-scrubbing when using generated images, our model still significantly outperforms other methods.

Additionally, we would like to highlight that in our initial submission, the unlearning process was conducted using only 8K data points, randomly selected from $\mathcal{D}_r$. Following your suggestion, we have not only clarified this in the main paper but also explored the impact of varying the number of samples from $\mathcal{D}_r$ on our algorithm, detailed in Figure 6 on page 15. We found that increasing the sample size can enhance the quality of images generated from the remaining classes, albeit at the price of slight detriment to the forgetting classes.

Q1.2 Can the authors clarify the connections and similarity of this work compared to [1-3]?

Thanks for the suggestion. All these methods aim to erase some information carried by the diffusion models. However, [1] and [3] mainly focus on text-to-image models and high-level visual concept erasure ([3] also focus on debiasing). Both the method proposed in [2] and ours aim to maximize the distance between the ground-truth backward distribution and the learnable backward distribution, to perform class-wise unlearning and concept erasure for diffusion models. Thanks, we have since added this in the related work section and removed the statement in the introduction.

Q1.3 Compare against Selective Amnesia [2] and FID score clarifications.

In the current stage, we follow the same setting in [2] to forget samples belonging to the label '0' on CIFAR-10 and use the well-trained model from [2]. From [2], the FID score on the $\mathcal{C}_r$ is 9.67 for the original model and 9.08 for Selective Amnesia [2]. With an EMA decay of 0.99, our method achieves a FID score of 8.93 at 210 steps, 8.83 at 205 steps, and 8.90 at 200 steps. Generated examples are shown in Figure 20 on page 21 of the revised manuscript.

Regarding the FID score on CIFAR-10, results in Table 2 are reported for DDIM with inference time step $T=100$, and FID is computed over 5K images separately for each class, while results in Table 1 of [2] are reported for DDPM with inference time step $T=1000$ and FID is computed over 5K$\times 9$ images. For your reference, the FID score computed over 40K images is around 8.18 in Table 2.

Q1.4 Does hyper-parameter $\lambda$ control the balance between retaining and forgetting data?

Yes, $\lambda$ controls the balance between retaining and forgetting data. An ablation study with respect to $\lambda$ is provided in Figure 6. Thanks, we have since added a description in the main text for clarity.

Q1.5 The texts on Figure 3 are not easily readable. What is the y axis (frequency)? How do we interpret this plot?

The y-axis represents the number of occurrences of the specific value. The diffusion process would add noise to the clean input $x$, consequently, the model output $\epsilon_T$ distribution given these noisy images $x_t$ tends to converge towards the Gaussian noise distribution $\epsilon$. As shown in Figure 3, we can see that the distance between the output distribution with respect to $\mathcal{C}_r$ and the Gaussian distribution for our scrubbed model is close to that of the unscrubbed model, while the distance between the output distribution with respect to the $\mathcal{C}_f$ is further away from the Gaussian distribution for our scrubbed model than the unscrubbed model, indicating that our algorithm preserves the model utility over $\mathcal{C}_r$ while successfully scrub the information concerning $\mathcal{C}_f$. Thanks, we have since revised the context for better clarity.