Machine Unlearning for Image-to-Image Generative Models

We really appreciate your encouragement for our work. Please see our responses to your questions below.

Question 1 Abstract: “primarily focused on classification models, leaving the landscape of unlearning for generative models relatively unexplored.“: Is this true? Are there any references which support this statement?

Response 1 We provide the following categories of references to support our claim that ’the landscape of unlearning for generative models is relatively unexplored’:

• Many recent review/survey papers on machine unlearning. As shown in [1,2,3,4], there are only three papers targeting unlearning for generative models out of hundreds of cited papers in these papers.

• Open-source collections of machine unlearning papers [5]. Out of 254 papers shown in this collection, only eight papers focus on the unlearning for language generation and only 9 papers target the unlearning for image generation. Specifically, for unlearning on image generation models, most papers only explore GANs and none of them addresses the image-to-image generation. In contrast, our approach is generally applicable to GANs, MAE and diffusion models and is the first to unlearn on image-to-image generative models.

Question 2 Intro: "Informally speaking, we define a generative model as having 'truly unlearned' an image when it is unable to faithfully reconstruct the original image when provided with only partial information (see Figure 1 for an illustrative example where the partial information involves center cropping2).": I wonder if you did any analysis if the "noise" is fully uncorrelated with the part of the image before the unlearning? Maybe there are still some high-level corrections left. In learning fair representations people do train secondary models on the representations and try to predict the sensitivity attribute from the fair representations. I wonder if such study can be used to check if a secondary autoencoders could not re-generate the cropped image from the generated noise.

Response 2 We appreciate for this valuable suggestion for a more thorough evaluation of our approach. To this answer this question, we conducted the following experiment for VQ-GAN:

• We first conduct the unlearning for VQ-GAN for ImageNet to obtain the unlearned model, under the same setup mentioned in Section 4.1.

• Given the center-cropped input images (cropping central $8\times 8$ patches), we then use this unlearned model to generate the images on both forget set and retain set (i.e., image upcropping/inpainting). Here we call them FIRST generated images.

• Given these FIRST generated images, we only keep the reconstructed central $8\times 8$ patches (cropping the outer ones) as the new input to the original VQ-GAN (i.e., the model before unlearning) and get the newly generated images (here we call them SECOND generated images). We then evaluate the quality of SECOND generated images for both forget set and retain set and report the results.

• We also conduct the above process for the original model (i.e., before unlearning) as the baseline/reference. The results are given below:

NOTE: Lower FID, higher IS, and higher CLIP values indicate higher image quality.

As shown above, compared to the original model, given the noise (i.e., FIRST generated images on the forget set) from the unlearned model as the input, the SECOND generated images has very low quality in terms of all these three metrics. This means that the noise indeed are not correlated with the real forget images.

In contrast, the performance on the retain set is almost the same before and after unlearning. This indicates that our approach indeed preserves the knowledge on the retain set well.

We will add these results and provide some visualizations in the revised paper.

We thank the reviewer for the feedback and encouragement! We address your question point-by-point below.

Question The main concern is that the paper assumption is ideal and its hyper parameter $\Sigma$, which may not hold in practice.

Response Regarding the assumptions used in our paper, we tried our best in making them align with practice. Specifically, one of the main assumptions is that the original model can do a perfect reconstruction for the cropped images. This assumption is somehow ideal, yet practically reasonable, especially given the recent progress in diffusion models [1,2]. We also note that since our work is the first to target unlearning for image-to-image generative models, we hope it will pave the way for follow-up works that will relax some of our assumptions. The theoretical results we provide offer a solid basis for such future extensions.

We also agree with Reviewer that, ideally, we should use the exact $\Sigma$ of the images from the forget set. However, there are some computational barriers to using the exact $\Sigma$ for a high-resolution image dataset. Specifically, consider a commonly used 256$\times$256 resolution for image generation tasks, the distribution of the generated images will have $256\times 256\times 3 \approx 2\times 10^5$ dimensions. The size of the covariance matrix $\Sigma$ for such a high-dimensional distribution is around $(2\times 10^5)^2=4\times 10^{10}$, which requires around 144GB memory if stored in float precision thus is not practical.

Consequently, to address the computational barrier of $\Sigma$, we use some approximated methods derived from some empirical observations on some small datasets. Specifically, we compute the exact $\Sigma$ for some small-scale image dataset, including MNIST and CIFAR10/100.

• Off-diagonal elements: as shown in Figure C.6 at Appendix C.4, the values on the diagonal of exact $\Sigma$ for these datasets are generally much larger than the off-diagonal elements; hence, we truncate the off-diagonal elements to zero.
• Diagonal elements: since the input images for deep networks are typically normalized, we may set the diagonal elements to one.

In short, using $I$ to approximate $\Sigma$ is a practical approximation alternative due to the extremely high computational costs of $\Sigma$ for high-resolution images. For future work, given our theoretical analysis, we believe that our approach will achieve better results (lower image quality) on forget set if we can find a way to use the exact $\Sigma$. Hence, we plan to explore the potential to reduce the computation of exact $\Sigma$ with low-rank approximation thus enabling the use of more accurate data-driven $\Sigma$.

We will add the above clarifications to the revised paper. We hope our response can address your concerns. We look forward to further discussions if you have any other questions.

[1] Song, Bowen, et al. "Solving inverse problems with latent diffusion models via hard data consistency." arXiv preprint arXiv:2307.08123 (2023).

[2] Wallace, Bram, Akash Gokul, and Nikhil Naik. "Edict: Exact diffusion inversion via coupled transformations." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Dear Reviewer 4Xrc,

Thanks again for constructive feedback on our paper. We hope that our previous responses have successfully addressed your concerns. We would like to remind you that the author-reviewer discussion period is closing very soon. If there are any remaining questions or concerns, we are looking forward to further discussions!

Best regards,

The Authors of Paper 2859

We thanks for your constructive feedback for our work. We address the questions point-by-point below.

Question 1: It's unclear if the machine unlearning setup considered in this work is practical. For machine unlearning setups, the gold standard to mimic is a model trained only on the retain set. In this work, the approach performs this task by minimizing the embedding distance for the forget set to normal distribution. Does this lead to overall worse performance than the gold standard model (where only retain set is used to train from scratch)? It would be good to include this as a baseline for comparison.

Response 1: This is a very fundamental question, and we understand the reviewer’s concern to the definition/setup of our unlearning approach. Indeed, we did ask the same questions ourselves. First, we agree that most of previous works consider the training only on the retain set is the gold baseline for classifications tasks. However, recently it has been shown that this golden standard bear several concerns:

• Some recent works show that for ‘some unlearning algorithms can generate models indistinguishable from the golden baseline beyond any arbitrarily small threshold while still exposing the deleted data’ for some cases [1]. In other words, the indistinguishability of training on retain set only cannot guarantee the data has been totally removed from the model, which is the goal in our paper.

• Retraining these generative models from scratch on retain set is completely beyond our computation capacity. For the three models evaluated in our paper, it takes more than 10K GPU hours ($>$50 days on our 8-GPU server; estimated from the experimental setup in [2,3,4]) to train the model from scratch.

• Given these considerations, our setup/definition primarily focuses on the practical scenarios other than the indistinguishability from the retrained model. For example, if the training set for the original model has some pornographic images by accident, our methods can make the unlearned model avoid generating pornographic images in a very efficient way other than expensive retraining the model from scratch. This way, the model's developers can avoid the potential violation of contents control. In short, our main goal is making the unlearned model not generate the samples from the forget set without hurting the performance on the retain set. Under this perspective, the performance drops on the forget set shown in our results is expected and dedicatedly designed. Moreover, the results show that on retain set, our approach has negligible performance drop compared to the model before unlearning.

As for the performance comparison with the golden standard model, our method will have worse performance on the forget set than this golden baseline (the performance on the retain set is expected to be similar to the golden standard model), which is dedicatedly designed by our approach.

Finally, we remark that the unlearning for generative models is still in its emerging stage; we do hope our works could contribute to a better setup/definition for this problem space.

[1] Chourasia, Rishav, and Neil Shah. "Forget Unlearning: Towards True Data-Deletion in Machine Learning." ICML 2023. https://icml.cc/virtual/2023/poster/23753

[2] Li, Tianhong, et al. "Mage: Masked generative encoder to unify representation learning and image synthesis." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[3] He, Kaiming, et al. "Masked autoencoders are scalable vision learners." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[4] Saharia, Chitwan, et al. "Palette: Image-to-image diffusion models." ACM SIGGRAPH 2022 Conference Proceedings. 2022.

Dear Reviewer L8mN,

Thanks again for constructive feedback on our paper. We hope that our previous responses have successfully addressed your concerns. We would like to remind you that the author-reviewer discussion period is closing very soon. If there are any remaining questions or concerns, we are looking forward to further discussions!

Best regards,

The Authors of Paper 2859

Question 4: The evaluation process presented in the paper is incomplete with regard to established machine unlearning protocols. Typically, an unlearned model's performance is assessed using three distinct datasets: a test dataset to determine its generalization capability, a retained dataset to evaluate performance on non-forgotten data, and a forget dataset to check the efficacy of the unlearning process. The paper's Table 1 appears to only present results for the latter two, omitting the crucial evaluation on the general test dataset.

Response 4: This is a very important question. We would like to clarify that we actually performed such a test (the first evaluation in the reviewer's question) in our paper, but simply described it in a slightly different way.

Specifically, out of 1000 classes from ImageNet-1K, our experimental setup selects 100 classes as the retain set $D_R$ and another 100 classes as the forget set $D_F$; the remaining 800 classes $D_O$ are not considered either as retain or forget sets. We evaluate our method with the following setup: In Table 1 and Table 2 (rows Proxy $D_R$) of our paper, we assume that we don’t have access to the images from 100 classes from $D_R$, and then we use images from the other 800 classes $D_O$ to serve as the proxy retain set for our unlearning approach. Moreover, the performance/test on the retain set is always conducted on the 100 classes from original $D_R$.

In fact, for this setup, we can re-consider the original $D_R$ as the general test set and re-consider the $D_O$ as the new $D_R$. As shown in Table 1 and Table 2 (rows Proxy $D_R$), our approach is evaluated on the 100 classes from general test set and it shows negligible performance drop on the general test set.

Question 5: The scope of the study with respect to the application of machine unlearning in image-to-image (I2I) generative models appears to be inaccurately broad. The term "machine unlearning for I2I generative models" suggests a wide range of applications; however, the paper primarily focuses on the image inpainting task. It would be more precise to either expand the variety of I2I applications examined in the study or to specifically define the scope as "machine unlearning for image inpainting tasks" to reflect the content more accurately. This would ensure clarity in the paper's contributions and avoid overgeneralization of the results.

Response 5: We appreciate this reviewer's suggestion. We will modify our title to 'Machine Unlearning for Image Completion Generative Tasks'.

Question 6: Why easing concepts can be thought about as a noisy label method? Please give more explanations.

Response 6: the reason we use Gaussian noise as the optimization target for the forget set is motivated by our theoretical analysis. Specifically, we prove that Gaussian noise is the unique optimal solution for unlearning on image-to-image generative models if we want to destroy the performance on the forget set. This is indeed one of the main contributions of this paper.

Thanks again for your comments; we would be happy to provide more clarifications and answer any follow-up questions.

[1] Graves, Laura, et al. "Amnesiac machine learning." AAAI 2021.

[2] Chundawat, Vikram S., et al. "Zero-shot machine unlearning." IEEE Transactions on Information Forensics and Security (2023).

[3] Chen, Min, et al. "Boundary Unlearning: Rapid Forgetting of Deep Networks via Shifting the Decision Boundary." CVPR 2023.

[4] Warnecke, Alexander, et al. "Machine unlearning of features and labels." arXiv preprint arXiv:2108.11577 (2021).

[5] Jia, Jinghan, et al. "Model sparsification can simplify machine unlearning." arXiv preprint arXiv:2304.04934 (2023).

[6] Kurmanji, Meghdad, et al. "Towards Unbounded Machine Unlearning." arXiv preprint arXiv:2302.09880 (2023).

[7] Golatkar, Aditya, et al. "Eternal sunshine of the spotless net: Selective forgetting in deep networks." CVPR 2020.

[8] Gandikota, Rohit, et al. "Erasing concepts from diffusion models." arXiv preprint arXiv:2303.07345 (2023).

[9] Moon, Saemi, et al. "Feature unlearning for generative models via implicit feedback." arXiv preprint arXiv:2303.05699 (2023).

[10] Chourasia, Rishav, et al. "Forget Unlearning: Towards True Data-Deletion in Machine Learning." ICML 2023. https://icml.cc/virtual/2023/poster/23753

[11] Li, Tianhong, et al. "Mage: Masked generative encoder to unify representation learning and image synthesis." CVPR 2023.

[12] He, Kaiming, et al. "Masked autoencoders are scalable vision learners." CVPR 2022.

[13] Saharia, Chitwan, et al. "Palette: Image-to-image diffusion models." ACM SIGGRAPH 2022 Conference Proceedings. 2022.

We thanks for Reviewer's feedback and comments. Please see our responses to your questions below.

Question 1 The study does not align with the foundational concept of machine unlearning, which typically necessitates a comparison between the unlearned and retrained models as per references [1-7].

Response 1 This is a fundamental question, and we understand the reviewer's concern about the definition/setup of our unlearning approach. Indeed, we did ask the same questions ourselves. First, for classifications tasks, we agree that most of previous work considers that the unlearned model should be indistinguishable to the model retrained only on the retain set (golden baseline). However, there are three points we want to bring to this reviewer’s attention:

• First, even the indistinguishability to the golden standard may have some issues. Indeed, recent work has shown that ‘some unlearning algorithms can generate models indistinguishable from the golden baseline beyond any arbitrarily small threshold while still exposing the deleted data’[10]. In other words, the indistinguishability of training on the retain set alone cannot guarantee the information is indeed removed from the model.

• Second, retraining these generative models from scratch on the retain set is beyond our (and average user, in general) computational capabilities. More precisely, for the three models evaluated in our paper, it would take more than 10K GPU hours ($>$50 days on our 8-GPU server; estimated from the experimental setup in [11,12,13]) to train the model from scratch; this is clearly infeasible given the resources we have at hand and the tight deadline for this rebuttal period.

• Given these considerations, our setup/definition primarily focuses on the practical scenarios of copyright protection and/or pornographic/violence content control, other than the indistinguishability from the retrained model. For example, if the original training set for the original model got some pornographic images by accident, our method can make the unlearned model not generate those pornographic images in a very efficient way (as opposed to expensively retraining the model from scratch). Moreover, the results show that on the retain set, our approach has negligible performance drop compared to the model before unlearning. Consequently, we believe our approach is sound for copyright protection and pornographic/violence content control hence very useful in real applications.

Question 2: Although the authors justify this divergence due to the high costs associated with retraining generative models, this deviates from the core goal of machine unlearning aimed at addressing privacy concerns. Particularly, the approach presented in this paper generates conspicuous Gaussian noise over the 'forgotten' data, which may inadvertently signal that the data was previously part of the training set, contradicting privacy preservation goals. A more compelling motivation might be found in text-to-image (T2I) scenarios [8] where the goal is to prevent the generation of inappropriate content, or in image-to-image (I2I) applications [9] that showcase practical utility. It would be more meaningful—and privacy-compliant—if the model could reconstruct unremarkable images that don't trace back to the original training data, rather than reconstruct images with evident distortions signaling prior data use.

Response 2: This is a very insightful comment (the approach presented in this paper generates conspicuous Gaussian noise over the 'forgotten' data, which may inadvertently signal that the data was previously part of the training set, contradicting privacy preservation goals.). We note that, under our setup, given an image with noise distortions, one cannot conclude whether it is in the training set or not. For example, suppose our method makes the model forget the concept car; given any image of a car, the corresponding generated image typically will have noise distortions, but it is possible that this specific image is not used for training. Even though one can infer that some concepts are from the forget set, that still meets our main goal, i.e., the unlearned model does not generate the unwanted images.

Dear Reviewer uD61,

We thank you very much for raising the rating for our paper. We appreciate your new comments and suggestions very much. Please see our responses to your new questions below:

Question: Regarding Q1: I understand and acknowledge your motivations concerning the challenges of retraining, which are central to your paper's premise. However, I still hold reservations about the first point in your response as a robust support for your method. While I appreciate the practical implications of your approach, I believe that a more solid theoretical foundation or guarantee that your method effectively removes the targeted information from the model would strengthen your argument. Furthermore, showcasing a practical application, as hinted at in your motivation, could significantly enhance the paper's impact and relevance. Demonstrating this application would not only align with your motivations but also provide tangible evidence of your method's effectiveness.

Response: Ideally, for a given unlearned model, the generated images ($\hat{X}_F$) on the forget set should have no information related to the corresponding ground truth images ($X_F$). From the perspective of mutual information (MI), this means that the mutual information between $\hat{X}_F$ and $X_F$ is 0, i.e., $I(X_F;\hat{X}_F)=0$.

In our paper, our theoretical analysis and proposed optimization methods aim to remove the related information from the generated images ($\hat{X}_F$) by gradually maximizing its mutual information w.r.t a Gaussian Noise. To further verify the effectiveness, we estimate the mutual information between the reconstructed images and the corresponding ground truth images on both forget set and retain set.

We note that estimating the mutual information for high-dimensional images is extremely difficult. To this end, we follow the standard practice and use one of the state-of-the-art approximate estimation methods which compute a variational lower bound of exact mutual information [14]. Moreover, we also compute the variational mutual information bound between the ground truth images and Gaussian noise for a reference. The results are shown below:

NOTE: SMILE_1.0 and SMILE_5.0 are using the same SMILE mutual information estimator but with different hyper-parameters [14].

As shown above, after unlearning on the forget set, the mutual information between the reconstructed images ($\hat{X}_F$) and the corresponding ground truth images ($X_F$) $I(X_F;\hat{X}_F)$ is much lower than the original model (i.e., before unlearning). Moreover, after unlearning, $I(X_F;\hat{X}_F)$ is very close to the $I(X_F;\mathcal{N})$.

As for the retain set, the $I(X_R;\hat{X}_R)$ is roughly the same as before unlearning, which means that the information on the retain set is well preserved.

Finally, per reviewer’ suggestion, we plan to add a companion demo for the final version of our paper; this demo will show our approach in action on a few generic applications, hence illustrate our method’s effectiveness.

Question: Regarding Q3: Your response here is well-received. Discussing the distinctions between machine unlearning (MU) in classification and generation contexts is crucial, and your explanations are compelling. The comparison of methods between these contexts enriches the paper’s content. However, I noticed that reference [5], which proposes adding a sparse regularization term to the general machine unlearning objective function, seems like it could be directly adapted to a more general case. (Maybe need to be double-checked by authors). I suggest adding this part to the revision. Including this analysis would provide a more comprehensive view of how your method situates within the broader landscape of MU techniques.

Response: Indeed, we mentioned that [5] is orthogonal to our approach. To further demonstrate this point, we combine our approach together with the $L1$-regularization techniques introduced in [5]. We use the so called 'linear decaying' trick on page 6 of [5] and compare the performance with and without $L1$-regularization for VQ-GAN, under the same setup mentioned in Section 4.1. Due to time limitations, we conduct the test only on a subset of the validation set with 5 images per class instead of the entire set (50 images/class). The results are shown below:

As shown above, by combining our approach with $L1$-regularization from [5], the results are roughly the same as the case without $L1$-regularization.

We also tried different coefficients $\lambda$ for the $L1$-regularization. The above results are the coefficients that achieve optimal trade-off between the performance on the retain set and forget set. We found that the performance on the retain set is pretty sensitive to the value of $\lambda$; specifically, the performance on the retain set will have a significant drop if the $\lambda$ is relatively large (e.g., $>10^{-6}$). We have the following comment regarding this observation:

• Previous work reveals that sparsity of networks is relatively high for fairly simple and naive tasks (e.g., CIFAR-10/100, ImageNet, GLUE, etc.)[15]. In other words, if the network is designed for more challenging tasks such as image generation, then there is less sparsity inside the network compared to the classification tasks. This may be the reason why $L1$-regularization from [5] doesn't achieve the same improvements in image generative tasks as in classification tasks.

Thank you again for your constructive feedback. We would be happy to provide more clarifications and answer any follow-up questions.

[5] Jia, Jinghan, et al. "Model sparsification can simplify machine unlearning." arXiv preprint arXiv:2304.04934 (2023).

[14] Song, Jiaming, et al. "Understanding the limitations of variational mutual information estimators." ICLR 2020.

[15] Liu, Shiwei, et al. "Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!." ICLR 2023.

Dear Reviewer hK5e,

We are sorry to hear this from your side. We thank you again for your efforts and engagement for the review process of our paper. Wishing you a speedy and smooth recovery.

Take care and all the best.

Best,

The Authors of paper 2859