Not All Wrong is Bad: Using Adversarial Examples for Unlearning

We thank the reviewer for their insightful comments. We are excited about the reviewer’s acknowledgment of our interesting approach toward unlearning. Below are our responses to their questions and concerns:

Theoretical guarantees: Although most prior SOTA methods in approximate unlearning are not accompanied by theoretical guarantees, we prove a theorem (https://shorturl.at/ChU0s) that derives an upper-bound on the 2-norm of the difference of the parameters of the unlearned model and the retrained model (which are gold-standard for unlearning). To prove this theorem, we make assumptions that are common in the certified unlearning literature. Our derived upper-bound implies enhanced effectiveness of our method when:

1. The distance between the forget sample and its corresponding adversarial example becomes smaller.
2. The Lipschitz constant of the model becomes smaller.
3. The quality of the adversarial example becomes stronger (causes a larger loss for the correct label).
4. The adversarial example transfers better to the retrained model.
5. The retrained model generalizes better to the (clean) unseen samples.

Hence, the proved theorem also justifies our earlier intuitions about the need for good quality adversarial examples that are as close as possible to the original samples (which is the goal of Algorithm 1), and also justifies that by fine-tuning the model on these adversarial examples, we can derive an upper-bound on the distance between the retrained model and the unlearned one. We believe that the presented empirical results, along with the provided theorem, will motivate further theoretical studies in future work.

Larger model and dataset: we have performed our experiments on VGG19 models (12 times larger than ResNet-18) trained on the Tiny ImageNet dataset (200 classes). We evaluated our prior observations similar to Figure 1 in our manuscript that shows fine-tuning the trained models on their adversarial examples does not lead to catastrophic forgetting (https://tinyurl.com/5n6f6pxr). We also compared the unlearning methods and created the tables (https://tinyurl.com/2wwwbbfp) corresponding to Tables 1 and 2 in our manuscript.

In addition, as requested by another reviewer, we performed a comparison to a SOTA certified unlearning method (Zhang et al. (ICML 24)). The comparison is done only on the setting where $D_R$ is available because this method does not work when there is no access to $D_R$. Our results (https://shorturl.at/Q19RQ) show that certified unlearning methods such as this, though accompanied with theoretical guarantees, are not capable of outperforming SOTA in approximate unlearning, including AMUN. We believe that this is the case due to their assumptions not holding for deep learning models used in practice.

Adversarially-trained models: To evaluate whether our unlearning method works with models trained using adversarial training, we performed our analysis on ResNet-18 models trained using TRADES (as it was more convenient for us to use in the rebuttal period) on CIFAR-10. We performed the experiments for unlearning 10% of the dataset in both cases where $D_R$ is accessible and not. As the results (https://tinyurl.com/43bxcafb) show, in both settings AMUN is effective in unlearning the forget samples and achieving a low gap with the retrained models. This gap is obviously smaller when there is access to $D_R$.

We hope you find our responses satisfactory, and consider raising your score towards acceptance. We are happy to engage during the rebuttal period, and thank you again for your valuable comments and suggestions in improving our paper!

We thank the reviewer for their insightful comments. We are excited about the reviewer’s acknowledgment of our interesting approach toward unlearning. Below are our responses to their questions and concerns:

W1. Additional experiments: We have performed new experiments on VGG19 models (12 times larger than ResNet-18) trained on the Tiny ImageNet dataset (200 classes). We evaluated our prior observations similar to Figure 1 in our manuscript that shows fine-tuning the trained models on their adversarial examples does not lead to catastrophic forgetting (https://tinyurl.com/5n6f6pxr). We also compared the unlearning methods and created the tables (https://tinyurl.com/2wwwbbfp) corresponding to Tables 1 and 2 in our manuscript. We also performed a new experiment on the effectiveness of unlearning methods on the models trained with adversarial training, which, in summary (https://tinyurl.com/43bxcafb), shows that our method is even effective for unlearning in models trained with adversarial training. Please see the response to reviewer qDdb for details.

In addition, as requested by another reviewer, we performed a comparison to a SOTA certified unlearning method (Zhang et al. (ICML 24)). The comparison is done only on the setting where $D_R$ is available because this method does not work when there is no access to $D_R$. Our results (https://shorturl.at/Q19RQ) show that certified unlearning methods such as this, though accompanied with theoretical guarantees, are not capable of outperforming SOTA in approximate unlearning, including AMUN. We believe that this is the case due to their assumptions not holding for deep learning models used in practice.

W2. Interpreting our results: Please note that for UNLEARN ACC the goal is to minimize the difference with the corresponding value from the Retrained models. As tables 1&2 show, AMUN achieves the smallest difference in all scenarios. Similarly, for RETAIN acc in table 1 AMUN achieves the smallest difference with the corresponding value for the retrained models.

For table 2, the reason that some other methods achieve smaller gap for RETAIN acc is the following: when other methods perform poorly in the absence of $D_R$, they choose the smallest available learning rate during the hyper-parameter search which basically allows them to do nothing during the fine-tuning phase. Therefore, they almost always return the same model as the original model, without performing any unlearning, and hence achieve an accuracy of 100% on both $D_R$ and $D_F$. But notice that in these cases, the MIA score does not decrease, reiterating that no unlearning has occurred. We will make this point clear in our future revisions.

We thank the reviewer for their insightful comments. We are excited about the reviewer’s acknowledgment of the novel connections our work makes with adversarial training and Lipschitz-constrained models, in addition to a more rigorous analysis by leveraging RMIA rather than MIA. Below are our responses to their questions and concerns:

Q1 + W1. Additional experiments: We have performed our experiments on VGG19 models (12 times larger than ResNet-18) trained on the Tiny ImageNet dataset (200 classes). We evaluated our prior observations similar to Figure 1 in our manuscript that shows fine-tuning the trained models on their adversarial examples does not lead to catastrophic forgetting (https://tinyurl.com/5n6f6pxr). We also compared the unlearning methods and created tables (https://tinyurl.com/2wwwbbfp) corresponding to Tables 1 and 2 in our manuscript. We also performed a new experiment on the effectiveness of unlearning methods on the models trained with adversarial training (https://tinyurl.com/43bxcafb). Please see the response to reviewer qDdb for details.

W2. Theoretical guarantees: Although most prior SOTA methods in approximate unlearning are not accompanied by theoretical guarantees, we proved a theorem (https://shorturl.at/ChU0s) that derives an upper-bound on the difference between the retrained models and the models unlearned using AMUN by making assumptions that are common in the certified unlearning literature. The proved theorem justifies our earlier intuitions about the need for good quality adversarial examples that are as close as possible to the original samples. For more discussions on the proved theorem, please refer to the discussion with reviewer qDdb.

Missing related work: The work of Chen et al. is included among our baseline methods in all the experiments (BS for Boundary Shrink). We also have a thorough discussion of the differences in our approach in Appendix A.

Q2. Combination with DP: We do not see any obvious reason why AMUN can not be combined with approaches like DP. However, AMUN exploits the robustness of the model, and this is known to have tensions with privacy (as studied in several works such as https://tinyurl.com/3573vmks, https://tinyurl.com/2n6vu3tz, https://tinyurl.com/2cdptafr). We leave a detailed investigation of this to future research.

Q3. Comparison to certified methods: While the work of Sekhari et al. is an interesting addition to our discussion of certified unlearning methods in the related works, it is important to note that this method (similar to most other certified methods) makes many assumptions that do not hold in general deep learning models. For example, Assumption 1 in Section 4 states that “for any (input) z, the function f(w, z) is strongly convex, L-Lipschitz and M-Hessian Lipschitz with respect to w” – this is not practical. Instead, we performed a comparison to another SOTA certified unlearning method (Zhang et al ICML 24) with milder assumptions. This method does not work without access to $D_R$. Our results (https://shorturl.at/Q19RQ) show that certified unlearning methods such as this, though accompanied with theoretical guarantees, are not capable of outperforming SOTA in approximate unlearning, including AMUN, due to their assumptions not holding for deep learning models used in practice.

Q4. Choices of attack steps: For the presented results on ResNet-18 and CIFAR-10, we used PGD-50. For our new results on Tiny Imagenet, we performed the experiments with both PGD-10 and PGD-20 and did not observe noticeable changes.

Q4. Instability of fine-tuning: Please note that all the prior methods include fine-tuning steps as part of their procedure.

1. RL fine-tunes on forget samples with a random label and the remaining samples ($D_R$).
2. Salun does the same fine-tuning as RL, but on a subset of model parameters.
3. FT and l1-sparse fine-tune the model on $D_R$.
4. GA fine-tunes on the forget samples in the reverse direction of gradient and fine-tunes on $D_R$.
5. BS fine-tunes on $D_R$ and an augmented set of samples for forget samples.

In our experiments, we did not observe more susceptibility to hyper-parameters compared to other methods. We also performed a fair comparison by choosing the hyper-parameters on a separate set of forget-samples and models and performed the experiments on different random seeds. To further investigate the stability of results to the number of epochs, we prepared two plots that show Avg. Gap for various number of epochs (https://shorturl.at/VSLzK). We concluded that AMUN stabilises after a few epochs and the number of epochs could even decrease, but we followed the same number of epochs that were used in prior works for fair comparisons.

We hope you find our responses satisfactory, and consider raising your score towards acceptance. We are happy to engage during the rebuttal period, and thank you again for your valuable comments and suggestions in improving our paper!

We thank the reviewer for their detailed and insightful comments. We are excited about the reviewer’s acknowledgment of the uniqueness of our approach for unlearning. Below are our responses to their questions:

Q1+S1+W1. Theoretical guarantees: Although most prior SOTA methods in approximate unlearning are not accompanied by theoretical guarantees, we proved a theorem (https://shorturl.at/ChU0s) that guarantees moving toward the retrained models by making assumptions that are common in the certified unlearning literature. Please refer to the discussion with reviewer qDdb for more details.

Q2+S3+W3. Larger model/dataset: we have performed our experiments on VGG19 models (12 times larger than ResNet-18) trained on the Tiny ImageNet dataset (200 classes). We evaluated our prior observations similar to Figure 1 in our manuscript that shows fine-tuning the trained models on their adversarial examples does not lead to catastrophic forgetting (https://tinyurl.com/5n6f6pxr). We also compared the unlearning methods and created the tables (https://tinyurl.com/2wwwbbfp) corresponding to Tables 1 and 2 in our manuscript. Please see the response to reviewer qDdb for more details.

Q4. Certified baseline: We performed a comparison to a SOTA certified unlearning method (Zhang et al ICML 24). This method does not work when there is no access to DR. Our results (https://shorturl.at/Q19RQ) show that certified unlearning methods such as this, though accompanied with theoretical guarantees, are not capable of outperforming SOTA in approximate unlearning, including AMUN. We believe that this is the case due to their assumptions not holding for deep learning models used in practice.

Q3. Stronger attacks: We wish to clarify that the objective of our algorithm is to use adversarial attacks to find "neighbors" for those samples in the unlearning set, and use this neighbor set for fine-tuning to ensure that the resulting (unlearned) model lacks confidence on these samples. We are unsure of what the reviewer means to consider attacks to bypass adversarial fine-tuning, as the objective of the attack we consider is to "augment" the sample (to be unlearned) in a manner so as to ensure that it's confidence is low post fine-tuning.

Q5. Forget set: Upon receiving an unlearning request, the set of samples to be forgotten are specified. This is the common setting used in prior works in unlearning for classification models, such as SulUn and l1-sparse.

Q6. Table 1: The setting of Table 1 (access to DR) is much easier than Table 2. Still, we want to point out the fact that even in that scenario, our method achieves the smallest Avg Gap (based on the average over 27 runs). Moreover, once we make things more difficult, by either moving to a larger model and dataset, or revoking access to DR, the advantage of our method over prior works becomes apparent. The capability of performing unlearning without access to DR is much more desirable as it will be more practical in many real-world use-cases.

Q7+S2+W2. Computational cost: The time comparison: https://shorturl.at/AXVB7 . Note that, we only need to run Algorithm 1 on the samples that are requested to be forgotten. For our experiments, we choose a small sub-sample of the corresponding dataset and evaluate their final values of $\epsilon$; based on these values, we set the initial value of $\epsilon$ and run PGD attack on the samples. Then we only keep the samples for which an adversarial example is not found, and run another round of PGD with the updated $\epsilon$ value. We proceed until all the samples find one corresponding adversarial example. These histograms (https://shorturl.at/fMQDC) show the number of samples for each $\epsilon$ value. Based on our analysis, for both CIFAR10 and tiny imagenet, finding the adversarial example for all the samples is equivalent to less than 3 runs of PGD50 and PGD20, respectively.

W4+S4. More recent SOTA: SalUn is a SOTA method in approximate unlearning published in ICLR 24. In our experiments we added all the baseline methods from that work. In addition, we also performed experiments using the SOTA in certified unlearning (Zhang et al ICML 24), which has made our baselines comprehensive and up-to-date.

W5+S5. Limitations: We have tried various settings, such as Lipschitz-bounded models and adversarially-trained models to ensure our approach is not limited to only regular training paradigms, but there is no guarantee on compatibility with all training paradigms. Also, as with every new method, the introduction of this new methodology for unlearning might invite a certain line of attacks specifically targeted for this approach. Also future approaches are required to avoid the slight degradation in adaptive setting.

We hope you find our responses satisfactory, and consider raising your score towards acceptance. We are happy to engage further, and thank you again for your valuable suggestions in improving our paper!