Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models

• The experiments in the paper are only on models erased with MACE, although numerous SD erasure works [1,2] have been proposed. Without experiments on a few more baselines, logically speaking the evidence from the paper only supports the claim that concept resurgence occurs on models erased with MACE rather than in general, which would weaken its impact.

• Sec 4.3 seems to contradict the hypothesis that concept resurgence is more prominent if the weights from erasure were not moved far from the original weights, since I would assume LoRA makes smaller weight changes than full fine-tuning, yet the effects on resurgence are similar. Could the authors make this more quantitative and measure the deviation of the weights from the original values, for e.g., in the L2 sense?

• Have the authors tested resurgence on truly more 'abstract' concepts like nudity or violence? The current experiments focus on relatively 'easier' concepts that can be defined by a single or few synonyms, like the name of the celebrity or object. Concepts like nudity can be expressed by numerous synonyms and even abstractly by the names of artists who paint with nude styles, for example.

• Overall I found that the technical contribution of the paper to be somewhat lacking by ICLR's standards, even though the phenomena presented is novel. The experiments are focused on one baseline and two concept types (celebrities and objects). As the authors acknowledge in the limitations, the paper lacks theoretical insights into concept resurgence or any mitigation strategies.

Minor points:

• consider moving Eq 1 to the front of the paper and introduce MACE more thoroughly given that the experiments in the paper are focused on MACE.
• some missing references on early works in the area of erasure/safety in text-to-image models [1,2,3,4].

[1] Zhang, Eric, et al. "Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models. ArXiv abs/2303.17591 (2023)." (2023).

[2] Gandikota, Rohit, et al. "Erasing concepts from diffusion models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[3] Heng, Alvin, and Harold Soh. "Selective amnesia: A continual learning approach to forgetting in deep generative models." Advances in Neural Information Processing Systems 36 (2024).

[4] Schramowski, Patrick, et al. "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

• While this paper focuses mainly on MACE as the unlearning algorithm, it remains unclear whether the observed results could be fully generalizable to other unlearning techniques, which can be considered to add for more comprehensive analysis.
• Since we cannot enumerate all possible concepts during evaluation, could the authors provide some insights on the metrics that we can use to measure the difficulty of the resurgence of a certain concept? This might help to reach a more general conclusion of the experiments.
• Aside from the two examined celebrity and object erasure tasks and specific benchmarks, it would be better to extend the evaluation on more diverse settings to see if the findings still hold.
• Minor: Though it might be out of the scope of this manuscript, it is very interesting to have some theoretical analysis regarding the observations.

The supporting experiments are slightly below the ICLR standard. Overall, this paper should justify their claim via experiments but the supporting experiments are weak/handful.

• The interesting phenomenon is only evaluated on one unlearning model (i.e., MACE). Additional unlearning methods are needs to be evaluated on hopefully five different methods, e.g., Selective Amnesia (https://arxiv.org/abs/2305.10120), SALIENCY UNLEARNING (https://arxiv.org/abs/2310.12508), and more.
• For out-of-domain concepts, it would be useful to add some visualization on the correlation between a target concept and out-of-domain concepts.

• The main weakness of this paper is its limited experimental scope. While the paper's key contribution is the concept resurgence phenomenon, it is supported only by limited empirical evidence. This calls for testing the phenomenon in various setups, yet the authors only use a single model, SD v1.4. Given the availability of advanced models such as SDXL, EDM, MDT, and FLUX, it would be helpful to see experiments using other diffusion models, particularly those trained with flow matching objectives instead of score matching losses. Additionally, the authors exclude tasks related to artistic style removal and explicit content removal, citing evaluation challenges. However, it would still be valuable to demonstrate the concept resurgence phenomenon in these tasks, even if a fair evaluation is difficult. The current experimental setup is also limited in terms of dataset diversity. Providing additional qualitative examples beyond Figures 2 and 4 would strengthen the paper.
• To my understanding, this paper only experimented with a single unlearning technique, MACE. The authors need to explore more existing methods such as UCE, FMN, ESD, SDD etc. Even if MACE is a SOTA unlearning method, concept resurgence may not appear with the other baselines. Section 4.2, in particular, would benefit from a broader discussion of baseline methods.
• The authors propose three potential contributors to concept resurgence: mapping concept, regularization, and fine-tuning algorithms. However, the discussion in Section 4 lacks depth. The authors should offer theoretical justifications or at least propose a main hypothesis supported by empirical evidence. For example, in Figure 7, they suggest that “increasing regularization increases concept resurgence in the celebrity erasure task, but has little impact on the object erasure task.” It would be helpful to identify the key factor causing this difference and explore how this factor might be used to prevent concept resurgence. Further, the authors conclude that the difference between full fine-tuning and LoRA fine-tuning does not affect concept resurgence. However, if sufficiently “distant” fine-tuning can prevent concept resurgence, wouldn’t full fine-tuning be more effective than LoRA in doing so?