Direct Unlearning Optimization for Robust and Safe Text-to-Image Models

Thank you for your detailed review and for recognizing the novelty and effectiveness of our approach in using SDEdit for curating paired images.

---

[W1] Novelty of Technical Contribution

We want to clarify that our main contribution is proposing a new image-based unlearning framework to overcome the limitations of the previous prompt-based approach, not just combining two well-known techniques, SDEdit and DPO.

As our framework is new, we provide a new perspective on unlearning by formulating the diffusion models unlearning problem as preference optimization. This perspective naturally arose from our image-based approach and offers a fresh paradigm for tackling unlearning challenges.

We then introduce the a output-preserving regularization to specifically preserve unrelated features, a crucial aspect not addressed in standard preference optimization. Additionally, we adapt preference optimization for unlearning by using SDEdit to generate paired data, eliminating the need for human labeling.

In summary, our work introduces a novel image-based unlearning framework, reframes unlearning as preference optimization, and incorporates output-preserving regularization. These innovations collectively offer a robust, efficient approach to diffusion model unlearning, addressing key limitations of existing methods.

---

[W2] How can we ensure that DUO does not affect unrelated features?

We acknowledge that claiming DUO has absolutely no impact on unrelated features would be an overstatement. However, our comprehensive evaluations demonstrate that DUO significantly reduces the impact on unrelated features compared to existing baselines.

As detailed in the global rebuttal (paragraph 3 and Fig. R3), we conducted comprehensive evaluations of DUO's prior preservation performance. Our results demonstrate that DUO maintains model prior for unrelated concepts while effectively removing unwanted content.

In our revised manuscript, we will provide a more nuanced discussion, emphasizing that DUO significantly reduces impact on unrelated features compared to existing baselines, while still acknowledging the potential for minor, unintended effects.

---

[W3] Reproducibility

We have taken several steps to address reproducibility concerns:

• Robustness to Random Seeds We have conducted experiments using different random seeds (1 to 4) to demonstrate the stability of our results. Figure R5 in the global rebuttal PDF shows that DUO's Pareto curve is not sensitive to random seed variation and consistently outperforms baseline methods. Figure R6 provides qualitative evidence of similar unlearning results across different seeds.

• Code Release To ensure full reproducibility and contribute to the field's advancement, we commit to releasing our unlearning and evaluation code upon publication.

We believe these measures significantly enhance the reproducibility of our work. We welcome any suggestions for additional experiments or information that could further address reproducibility concerns.

---

[W4] Minor errors

Thank you for bringing these typos and grammatical errors to our attention. We have made the following corrections:

• L104: DSO → DUO
• L175, 180, 506: Eq 17 → Eq 11
• L275: lambda → output-preserving regularization
• L285: DCO → DPO
• L271: Ulike → Unlike
• Figure 6: Quanlitative → Qualitative

We appreciate your careful review. These corrections will be reflected in the final manuscript.

---

[Q1] Logic Behind L_prior Loss

The L_prior loss, i.e., output preservation regularization, was designed to maintain the model's prior effectively, even when the KL divergence regularization is weak. While DPO's KL divergence regularization theoretically prevents significant deviation from the pretrained model's distribution, there's a trade-off:

• Strong KL regularization hinders effective unlearning.
• Weak KL regularization can cause the model to lose its denoising capability due to the gradient ascent term.

L_prior addresses this by enforcing output preservation for specific inputs, particularly those unrelated to unsafe concepts.

Consider $x_t^- = \sqrt{\alpha_t} x_0^- + \sqrt{1 - \alpha_t} \epsilon$, where $x_0^-$ represents an unsafe image and $\epsilon \sim N(0, I)$. Our goal is to unlearn features related to $x_0^-$ while maintaining the model's behavior for features related to $\epsilon$. L_prior achieves this by preserving outputs for $x_T = \epsilon$, effectively maintaining the model prior for unrelated concepts while allowing unlearning of unsafe content.

Figure 8 demonstrates that L_prior's effect becomes more pronounced as beta decreases, highlighting its importance in maintaining model performance on safe concepts.

---

[Q2] Meaning of Prior Preservation (1-LPIPS)

Prior preservation is a crucial objective in unlearning, aiming to maintain the model's performance on concepts unrelated to the unlearned content.

To quantify this, we measure how much the model's output changes for unrelated concepts after unlearning. This is done by generating images using MS COCO prompts with both the pretrained and unlearned models. We use LPIPS to measure the perceptual difference between these generated images. Prior preservation is defined as 1-LPIPS. A higher value indicates that the unlearned model generates perceptually similar images to the pretrained model when given the same prompts and initial noise.

This metric helps us ensure that our unlearning method effectively removes unwanted content without significantly impacting the model's performance on unrelated topics.

To avoid confusion in the final manuscript, we will change the notation from L_prior to L_output for the output-preserving regularization term, distinguishing it clearly from the prior preservation metric.

Thank you for acknowledging our paper's clarity and the novelty of connecting preference optimization to unlearning.

---

[W1] Applicability of DUO to Diffusion Transformers

We appreciate your inquiry about DUO's applicability to other architectures. We have successfully extended our evaluation to include Transformer-based models. As shown in Figures R1 and R2 in the global rebuttal PDF, DUO effectively removes unwanted concepts in Stable Diffusion 3, which employs a Transformer architecture (mmDiT).

We will incorporate these findings into our revised manuscript to provide a more comprehensive evaluation of DUO's capabilities across different model architectures.

---

[W2] Impact of DUO on Unrelated Features

We acknowledge that claiming DUO has absolutely no impact on unrelated features would be an overstatement. However, our comprehensive evaluations demonstrate that DUO significantly reduces the impact on unrelated features compared to existing baselines.

As detailed in the global rebuttal (paragraph 3 and Fig. R3), we conducted comprehensive evaluations of DUO's prior preservation performance. Our results demonstrate that DUO maintains model prior for unrelated concepts while effectively removing unwanted content.

In our revised manuscript, we will provide a more nuanced discussion, emphasizing that DUO significantly reduces impact on unrelated features compared to existing baselines, while still acknowledging the potential for minor, unintended effects.

First, thank you for acknowledging that our paper is clear and well-structured.

---

[W1] Impact of the number of synthesized image pairs on DUO performance

Thank you for your constructive comment. Figure R7 from the global rebuttal PDF demonstrates how varying the number of synthesized image pairs affects the Pareto curve. The figure clearly shows that when the number of pairs is less than 64, there is a noticeable improvement in the Defense Success Rate. However, increasing the number beyond 64 pairs does not yield significant changes in the Pareto curve.

This analysis suggests that 64 pairs provide a good balance between performance and computational efficiency for our method. We will include this discussion in our final manuscript to address concerns about the dataset size and its impact on model performance.

---

[W2] Stronger White-Box Red Teaming results

We appreciate your valuable suggestion to incorporate more robust evaluation techniques. We have conducted additional experiments using UnlearnDiffAtk [1], a state-of-the-art white-box attack method designed for assessing the robustness of unlearned diffusion models.

As illustrated in Figure R4 of our global rebuttal PDF, DUO achieves Pareto-optimal performance compared to existing baselines when subjected to UnlearnDiffAtk. This means that DUO provides the best trade-off between maintaining model performance and resisting attacks, outperforming other methods in both aspects simultaneously.

We will incorporate a detailed analysis of the UnlearnDiffAtk results in our final manuscript, including qualitative comparisons to baseline methods.

[1]: Zhang et al., To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now https://arxiv.org/abs/2310.11868

First, thank you for your recognition of our paper's clear structure and strong evaluation results.

---

[W1] Construction of Violence Unlearning LoRA

We appreciate the opportunity to clarify this process. As you correctly surmised, we applied DUO independently to four subcategories of violence: "blood", "suffering", "gun", and "horror". For each concept, we trained a separate LoRA using DUO.

To create the final model, we merged these individual LoRAs. Specifically, if we denote the pretrained model weight as $W$, and the LoRA matrices trained on the $i$-th concept as $A_i$ and $B_i$, the merged weight matrix can be expressed as:

$\tilde{W} = W+ \sum_i B_iA_i$

This approach allows us to comprehensively address the violence concept while maintaining the efficiency of LoRA-based fine-tuning. We will provide a detailed explanation of this merging process in the appendix of our final manuscript to ensure full clarity and reproducibility.