Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation

We thank the reviewer for the positive feedback and insightful suggestions. We would like to address the remaining concerns as follows. Due to the space limitation, some responses are provided in the global rebuttal and the author comment section.

Q: Why using a CLIP alignment score is a reliable measure for concept inclusion?

In Section 3.2, we investigated which concepts are affected most by the erasure of a target concept. The concept space that we investigate is not only a specific set of related concepts as shown in Figure 1 but also a broader set of concepts of the CLIP token vocabulary which includes 49,408 tokens as shown in Figure 2. In this general and broader concept space, it is nearly impossible to have a pre-trained classification model that can detect all concepts. Therefore, to the best of our knowledge, the CLIP alignment score might be the most effective way to measure the concept inclusion, even though it is not perfect and might not be able to provide a reliable measure for some concepts as suggested by the reviewer. We will add this discussion to the revised version.

Q: Results of erasing nudity concept"

We apologize for the confusion. We would like to provide the results of erasing the "nudity" concept while preserving the "person" concept in the attached document. As shown in the figure, preserving related concepts like "person" helps to retain the model's capability on other concepts much better than preserving a neutral concept.

Q: Why do you finetune only non-cross attention modules to erase NSFW concepts?

Firstly, we would like to recall the cross-attention mechanism, i.e., $\sigma(\frac{(QK^T)}{\sqrt{d}})V$, where $Q$, $K$, and $V$ are the query, key, and value matrices, respectively. In text-to-image diffusion models like SD, the key and value are derived from the textual embedding of the prompt, while the query comes from the previous denoising step. The cross-attention mechanism allows the model to focus on the relevant parts of the prompt to generate the image.

Therefore, when unlearning a concept, most of the time, the erasure process is done by loosening the attention between the query and the key that corresponds to the concept to be erased, i.e., by fine-tuning the cross-attention modules. This approach works well for object-related concepts or artistic styles, where the target concept can be explicitly described with limited textual descriptions.

However, as investigated in the ESD paper Section 4.1, concepts like 'nudity' or NSFW content can be described in various ways, many of which do not contain explicit keywords like 'nudity.' This makes it inefficient to rely solely on keywords to indicate the concept to be erased. It is worth noting that the standard SD model has 12 transformer blocks, each of which contains one cross-attention module but also several non-cross-attention modules such as self-attention and feed-forward modules, not to mention other components like residual blocks. Therefore, fine-tuning the non-cross-attention modules will have a more global effect on the model, making it more robust in erasing concepts that are not explicitly described in the prompt. We will add more explanations to the revised version.

Q: Discussion on FID metric?

We thank the reviewer for raising a very important question.

Firstly, we utilized FID measured on common concepts like the COCO-30K dataset to evaluate the generation quality of the sanitized models because it is a widely used metric in the field of generative models. However, while this metric is useful in general generation settings, we acknowledge that it might not be sufficient for evaluating unlearning concepts.

As observed in our paper, the impact of erasure methods is not equally distributed across all concepts; some concepts might be more affected than others. While this imbalance might be mitigated with a large enough evaluation size that covers all possible concepts, making FID sufficient, this is not the case in practice, which commonly uses the COCO-30K dataset.

Therefore, we believe that a more comprehensive evaluation metric that can capture the impact of erasing concepts on specific concepts is needed. For example, we could assign a weight to each evaluated concept/prompt based on its relevance score to the target concepts, then compute a weighted FID or CLIP score.

Q: How would your proposed method only remove the sensitive parts while keeping the non-sensitive parts untouched in the NSFW setting?

While our method has demonstrated an interesting result in its ability to remove more sensitive body parts, we do not have a specific explanation for this phenomenon. However, intuitively speaking, if we consider two abstract concepts, "nudity" and "person," and two core concepts, "breast" and "feet," we can see that the "nudity" concept might be highly correlated with the "breast" concept, while the "person" concept might be more correlated with the "feet" concept rather than "breast."

Therefore, if we specifically preserve the "person" concept while erasing the "nudity" concept, the model might be able to generate images that include "feet" but exclude "breast." Our method, by selecting the most affected concepts, might naturally choose a concept that is highly correlated with non-sensitive body parts to be preserved.

Q: What is the level of granularity in the proposed method?

Our method does not have a specific mechanism to control the level of granularity or focus on a specific concept to be erased. Instead, our main goal is to prioritize the preservation aspect of the erasure process. However, if we were to address the problem of granularity, we would focus on enhancing the expressiveness of the textual description.

For example, using visual embeddings to describe the Mercedes logo concept could be a potential direction to improve the expressiveness and granularity of the method.

Q: Which CLIP model is used?

We used the OpenAI CLIP model `openai/clip-vit-large-patch14' to compute the alignment score. We will add this information to the revised version.

Q: How does your proposed method differ to the "Forget-Me-Not" paper? Any specific reason this is not covered in comparisons?

Our method significantly differs from the Forget-Me-Not (FMN) method. From our point of view, FMN falls into the category of approaches like TIME, UCE, and MACE, which focuses on confusing the alignment between the prompt and the visual features in the cross-attention mechanism. Specifically, FMN introduces an attention resteering method that attempts to alter the attention maps related to the target concept (i.e., by minimizing the L2 norm of the attention maps). Our method, on the other hand, stands out by focusing on identifying which concepts are most affected by the erasure of a target concept and then preserving these concepts to maintain the model's capability on other concepts. We will cite and discuss the FMN in the revised version.

Regarding additional experiments with the FMN method, we attempted to erase the same set of concepts from the Imagenette dataset and evaluated the erasing and preservation performance of the FMN method within our setting. However, despite our best efforts given the time constraints, FMN did not effectively erase the target concepts. Notably, their open-source code provides hyper-parameter settings for single-concept erasure but not for multiple concepts. We also noticed an open issue on their GitHub repository questioning this same problem.

This issue can easily be verified by running their code with a modified configuration file, attn.yaml, to erase multiple concepts, given some representative images of the target concepts.

Q: Coding question: Implementations of Equation 4 and Equation 5

Equation 4 describes our naive approach, which utilizes Projected Gradient Descent (PGD) to search for the adversarial concepts in the continuous space. We have uploaded the code for this approach to the anonymous repository. Please refer to the code for more details.

Regarding the implementation of Equation 5, our method involves bilevel optimization. In this process, the inner maximization step maximizes the L2 loss (i.e., minimizes -L2 loss) to select the most affected concepts to be preserved. The outer minimization step minimizes the combined L1 + L2 loss to erase the target concepts while preserving the most affected concepts.

For the L1 loss, we inherited this implementation from the ESD paper and did not attempt to modify it, including the aspect of negative guidance.

Q: Coding question: Why using K-means?

In the implementation, we investigate the use of K-means to control the tradeoff between the computational cost and the size of the vocabulary when searching for adversarial concepts. More specifically, we first compute the similarity between the target concept and the entire vocabulary, then select the top-K most similar concepts. We then use K-means to choose the K most representative concepts from these top-K most similar concepts. This approach allows us to cover a wide range of concepts while keeping the computational cost low.

Q: Coding question: How to calculate the text embedding for adversarial concepts in Stable Diffusion version 3

To the best of our understanding, as described in the encode_prompt function in the Stable Diffusion v3 pipeline (lines 310-311), as similar to the same function in the SDXL pipeline (line 397), we still can obtain the standard prompt embedding as the output of a text encoder which is independent of the time step. Therefore, we can still use our current approach to calculate the emb_r for the adversarial concepts.

As the rebuttal period is coming to an end, we hope to have clarified the novelty of our contribution and addressed the concerns of the reviewer. We truly appreciate the reviewer's time on our paper and we are looking forward to your responses and/or any other potential questions. Your feedback would be very helpful for us to identify any ambiguities in the paper for further improvement.

We thank the reviewer for the positive feedback and insightful suggestions. We would like to address the remaining concerns as follows.

Q: Better discussion

We thank the reviewer for pointing this out. We will revise the comparison with CA in the revised version, i.e., our method is slightly better than CA in terms of ESR scores but significantly better in terms of PSR scores.

Q: How to choose the concept space

We did not design specific but utilized a common set of concepts as the search space $\mathcal{R}$ for searching adversarial concepts across all experiments. To ensure the generality of the search space so that it can be applied to various tasks such as object-related concepts, NSFW content, and artistic styles, we used the Oxford 3000 most common words in English as the search space.

It is worth noting that as described in Section B.1, our method employ the Gumbel-Softmax trick to discretely search in the concept space $\mathcal{R}$, this approach requires feeding the model with the embeddings of the entire search space $\mathcal{R}$ to compute the alignment score, which is computationally expensive when the search space is large. To mitigate this, we use a subset of the K most similar concepts to reduce the computational cost.

To better address the reviewer's concern, we conducted additional experiments with the search space as the CLIP token vocabulary, which includes 49,408 tokens. It is worth noting that the CLIP token vocabulary is more comprehensive but presents challenges due to the large number of nonsensical tokens (e.g., ) Therefore, we need to filter out these nonsensical tokens to ensure the quality of the search space. The results from object-related concepts are shown in the table below.

The results show that the erasing performance is slightly lower when using the CLIP token vocabulary as the search space, but the preservation performance is much better with a gap of 5.4% in PSR-1 and 4.2% in PSR-5. This indicates that our method would benefit from a more comprehensive search space. We will add this experiment to the revised version.

Q: it is necessary to fine-tune the non-cross-attention modules in erasing NSFW concept

Firstly, we would like to recall the cross-attention mechanism, i.e., $\sigma(\frac{(QK^T)}{\sqrt{d}})V$, where $Q$, $K$, and $V$ are the query, key, and value matrices, respectively. In text-to-image diffusion models like SD, the key and value are derived from the textual embedding of the prompt, while the query comes from the previous denoising step. The cross-attention mechanism allows the model to focus on the relevant parts of the prompt to generate the image.

Therefore, when unlearning a concept, most of the time, the erasure process is done by loosening the attention between the query and the key that corresponds to the concept to be erased, i.e., by fine-tuning the cross-attention modules. This approach works well for object-related concepts or artistic styles, where the target concept can be explicitly described with limited textual descriptions.

However, as investigated in the ESD paper Section 4.1, concepts like 'nudity' or NSFW content can be described in various ways, many of which do not contain explicit keywords like 'nudity.' This makes it inefficient to rely solely on keywords to indicate the concept to be erased. It is worth noting that the standard SD model has 12 transformer blocks, each of which contains one cross-attention module but also several non-cross-attention modules such as self-attention and feed-forward modules, not to mention other components like residual blocks. Therefore, fine-tuning the non-cross-attention modules will have a more global effect on the model, making it more robust in erasing concepts that are not explicitly described in the prompt. We will add more explanations to the revised version.

As the rebuttal period is coming to an end, we hope to have clarified the novelty of our contribution and addressed the concerns of the reviewer. We truly appreciate the reviewer's time on our paper and we are looking forward to your responses and/or any other potential questions. Your feedback would be very helpful for us to identify any ambiguities in the paper for further improvement.

We thank the reviewer for the insightful comments and suggestions. We would like to address the remaining concerns as follows. Due to the space limitation, some responses are provided in the author's comment section.

Q: Additional experiments with SOTA methods such as ConAbl and SPM, and evaluation with FID and CLIP scores on COCO-30K dataset.

We followed the reviewer's suggestion and conducted additional experiments with MACE, a SOTA method recently accepted at CVPR 2024. It is worth noting that we already compared with the suggested baseline ConAbl denoted as CA in our paper.

Specifically, we performed the object-related setting which includes four tasks, each involving the erasure of five concepts from the Imagenette dataset. In addition to ESR-1, ESR-5, PSR-1, and PSR-5 metrics, we generated images with common concepts from the COCO-30K dataset and assessed FID and CLIP scores. Due to time constraints, we completed one task, erasing five concepts: Cassette Player, Church, Garbage Truck, Parachute, and French Horn.

The results are presented in the table below.

Regarding erasing performance, MACE slightly outperformed our method by 0.7% in ESR-1 and 0.5% in ESR-5 on average. However, in terms of preservation performance, our method significantly outperformed MACE, with an 8% gap in PSR-1 and a 7% gap in PSR-5. Additionally, our method achieved superior preservation performance in image generation with common concepts, evidenced by the lowest FID score of 16.3 and the highest CLIP score of 26.1.

These results indicate that while MACE shows marginally better erasing performance, our method excels significantly in preservation performance, even when compared to the latest techniques.

Compared to SPM:

The SPM method, although effective in concept erasure, does not directly fine-tune the original model. Instead, it trains a separate adapter that, when attached to the original model, prevents the generation of the erased concept. Specifically, Section 3.1 of the SPM paper introduces a new diffusion process $\hat{\epsilon} = \epsilon(x_t, c, t \mid \theta, \mathcal{M}_{c_e})$

, where $\mathcal{M}_{c_e}$ is the adapter model trained to erase the concept $c_e$. While these adapters can be shared and reused across different models, the original model $\theta$ remains unchanged, allowing malicious users to generate the erased concepts easily. In contrast, our method, along with the other baselines in our paper, directly erases the concept from the original model, making it more robust and preventing the generation of the erased concepts. For this reason, we did not include SPM in the comparison.

Q: Additional experiments with larger models

Due to resource constraints in the short rebuttal timeframe, we were only able to conduct additional experiments on SD v1.4. However, it is worth noting that SD v1.4 is still the most widely used model in the community, including recent work such as MACE. We will consider conducting experiments on larger models like SD XL in future work.

Q: Evaluating Efficiency

Following the reviewer's suggestion, we provided the fine-tuning time for the object-related concepts task in the table above. It is worth noting that we could only find efficiency evaluation in the SPM paper, which is specifically designed to highlight the efficiency of their lightweight adapter approach. Our method which was designed to prioritize erasing and preserving performance, may not be as efficient as SPM in terms of fine-tuning time. However, we politely argue that efficiency is not the main concern in our work or other baselines, as the fine-tuning process is relatively fast, typically completing in less than a few hours, which is acceptable in practice for infrequent concept erasure requests.

Q: Evaluating the impact of $\lambda$

To investigate the impact of different $\lambda$ values, we conducted additional experiments on the object-related concepts task with $\lambda = 0.1, 0.5, 5, 10$. The results are presented in the table below. There is a clear trade-off between erasing and preserving performance when changing the $\lambda$ value. A smaller $\lambda$ value results in better erasing performance but worse preservation performance, and vice versa. In our experiments, we did not attempt to tune $\lambda$ for each concept but just simply set it to 1 for all experiments as mentioned in line 774.

Q: Better visualization

In Figures 9-11 in the Appendix, we already provided a comparison between the output generated by the same prompt using the original model and the sanitized models after erasing each specific artistic style.

In each sub-figure, the first column shows the images generated by the original model, while the second to sixth columns display the images generated by the sanitized models. Each row corresponds to a prompt of one of these five artists.

The ideal erasure should result in changes in the diagonal pictures (marked by a red box) compared to the first column, while the off-diagonal pictures should remain the same. We believe that these visualizations are clear and informative, providing a direct comparison between the original and sanitized models.

As the rebuttal period is coming to an end, we hope to have clarified the novelty of our contribution and addressed the concerns of the reviewer. We truly appreciate the reviewer's time on our paper and we are looking forward to your responses and/or any other potential questions. Your feedback would be very helpful for us to identify any ambiguities in the paper for further improvement.

We thank the reviewer for the insightful comments and suggestions. We would like to address the remaining concerns as follows. Due to the space limitation, some responses are provided in the global rebuttal.

Q: Do all related concepts need to be preserved? Boundary between preserved and erasing concepts

We agree with the reviewer that when erasing a target concept like 'airplane,' the synonyms that resemble it visually, such as 'aircraft,' should also be erased. However, while our method does not explicitly set a boundary when searching for the adversarial concepts, the optimization process in our method naturally selects the most affected concepts but not the `similar' concepts to the target concept to be preserved. We support our response with both theoretical and empirical evidence.

Theoretical perspective:

Our optimization framework, described in Equations 4 and 5, involves a bilevel optimization process. The outer level (w.r.t. $\theta'$) minimizes the erasing loss L1 and the preservation loss L2 simultaneously, while the inner level (w.r.t. $c_a$) maximizes L2 to select the most affected concepts to be preserved.

Initially, $\theta' = \theta$, therefore after minimizing L1, the most affected concepts will be exactly the target concept $c_e$. As optimization progresses, the model $\theta'$ diverges from the original model $\theta$, driving the output $\epsilon_{\theta'}(c_e)$ away from $\epsilon_{\theta}(c_e)$ and closer to $\epsilon_{\theta}(c_n)$. This process makes the target concept $c_e$ one of the affected concepts or a candidate for the inner maximization w.r.t. $c_a$. However, selecting $c_a = c_e$ would directly conflict with the erasing loss L1 that aims to drive $c_e \rightarrow c_n$. Consequently, the inner maximization process steers towards selecting the most affected concepts but not the target concept $c_e$ or its synonyms.

Empirical perspective:

As mentioned in Appendix B.4 (lines 835-844) and shown in Figure 12, the intermediate results of the adversarial concept selection process indicate that the model initially selects concepts similar to the target. For instance, Figure 12 shows that 'truck,' 'music,' 'church,' 'French,' and 'bag' are selected as adversarial concepts for 'Garbage truck,' 'Cassette player,' 'Church,' 'French horn,' and 'Parachute,' respectively. Over time, the model shifts towards less similar but highly affected concepts.

Moreover, our response to another question demonstrates that our method effectively erases both the target concept and its synonyms, as shown by the results of erasing the synonyms.

In summary, both theoretical and empirical evidence support that our method naturally prioritizes preserving the most affected concepts, which are not the synonyms of the target concept but those significantly impacted by its erasure.

Q: Compare to MACE

Following the reviewer's suggestion, we conducted an additional experiment using MACE with their official implementation on object-related concepts, as detailed in section 5.1 of the main paper. Specifically, we conducted four distinct tasks, each involving the erasure of five concepts from the Imagenette dataset. In addition, we evaluated the preservation performance with common concepts from the COCO-30K dataset with FID and CLIP scores. Due to time constraints, we were only able to complete one task, which involved erasing five concepts: Cassette Player, Church, Garbage Truck, Parachute, and French Horn.

The results are presented in Table 1 in the attached document. Regarding erasing performance, MACE slightly outperformed our method by 0.7% in ESR-1 and 0.5% in ESR-5 on average. However, in terms of preservation performance, our method significantly outperformed MACE, with a gap of 8% in PSR-1 and 7% in PSR-5. Additionally, our method achieved superior preservation performance in image generation with common concepts, evidenced by the lowest FID score of 16.3 and the highest CLIP score of 26.1.

These results indicate that while MACE shows marginally better erasing performance, our method excels significantly in preservation performance, even when compared to the latest techniques.

Q: Evaluating FID on COCO dataset in erasing NSFW setting

We already measured the FID on the COCO dataset and provided the results in Table 2 in the main paper.

As the rebuttal period is coming to an end, we hope to have clarified the novelty of our contribution and addressed the concerns of the reviewer. We truly appreciate the reviewer's time on our paper and we are looking forward to your responses and/or any other potential questions. Your feedback would be very helpful for us to identify any ambiguities in the paper for further improvement.