MemBench: Memorized Image Trigger Prompt Dataset for Diffusion Models

We sincerely appreciate the reviewer's valuable feedback. However, there may have been misunderstanding about our related work and motivation. We would be truly grateful if these points could be revisited during the rebuttal period.

Before addressing the questions, we summarize the contributions highlighted in the global response:

• MCMC trigger prompt searching algorithm: Only approach currently capable of searching trigger prompts that replicate train image where training data (LAION) is no longer accessible. Although without data, it requires significantly fewer computational resources than prior methods [C1, 2] that require training data.
• MemBench: We are the first to 1) provide general prompt scenario, 2) provide absolute standard for performance evaluation, and 3) evaluate image quality.

For details, we kindly invite the reviewer to the global response.

[C1] Extracting Training Data from Diffusion Models, USENIX’23.

[C2] A Reproducible Extraction of Training Images from Diffusion Models, Arxiv preprint.

W1. The method seems inapplicable to models like Stable Diffusion 3 with unknown training data. (The work uses Stable Diffusion 2 to show empirical evidence)

We’d like to remind Figure 6, where we provide Stable Diffusion 3 memorization candidates found by our algorithm. The repetitively generated images shown in Figure 6 meet the criteria for memorization detection methods proposed by Carlini et al. [C1] and result in high values for the detection method $D_\theta$ established by Wen et al. [C3]. All this evidence suggests that the images found by our algorithm are Stable Diffusion 3 memorized data. However, as stated in Appendix E in the revised paper, we did not create a dataset for newer models like Stable Diffusion 3, where training data is unknown, because we do not know whether the images our algorithm found is actually in the training dataset or not. Building a benchmark without knowing whether training data exists on the web is unreasonable.

Also, we would like to correct that we already demonstrated the generalizability of our algorithm by applying to various models, including Stable Diffusion 1, Stable Diffusion 2, Realistic Vision, and DeepFloydIF in Table 1 of the main paper as well as Stable Diffusion 3 in Figure 6. These prompts found from various models show that our method is generalizable and efficient. We additionally provided examples of Stable Diffusion 1 memorized images and prompts in Appendix G (Appendix H in the revised paper). The reason we focus on Stable Diffusion 2 in Section 6 is to find why image memorization occurs even without image repetition in the training data—a factor widely regarded as a major contributor to memorization. Stable Diffusion 2 is not the only model that shows empirical evidence of our method.

[C3] Detecting, Explaining, and Mitigating Memorization in Diffusion Models, ICLR’24.

W2. Have you attempted to reverse search in LAION-5B for Stable Diffusion 3?

As stated in lines 129–131 of the main paper, LAION-5B has been made inaccessible, making it impossible to query images from it. Our core contribution lies precisely in developing an algorithm to identify trigger prompts despite LAION-5B being inaccessible.

We hope that our additional experiments and responses have effectively addressed the reviewer’s concerns. If there are any further questions or clarifications needed, we would be happy to address them.

If there are any additional discussion points or questions, we are happy to discuss. Thank you.

We thank the reviewer's valuable feedback. Thank you for finding our contributions, including the efficient MCMC searching algorithm that operates without train data, the extensive prompt collection in MemBench, and our thorough evaluation of mitigation methods. We have conducted and provided all the experiments requested by the reviewer. Through this revision, we believe that our submission is further improved.

W1. Unclear writing: 1) The technical innovation of the paper is not clearly highlighted. 2) The benchmark finding section should be consolidated. 3, 4) Provide a detailed explanation of the SSCD metric / AUC value of the detection method [C1]. 5) Provide specific computational constraints that prevented certain comparison methods from being included in the main body of the paper not just in the appendix.

Thank you for finding one of our contributions, the technical innovation of searching memorized images in diffusion models without relying on train data. We have addressed all the points raised by the reviewer and incorporated the changes into the revised paper, which are marked in red for convenience. (Cons 1: lines 78-84 / Cons 2: 71-75 / Cons 3: 179-184 / Cons 4: 204-210) Regarding Cons 5, we would like to confirm whether it refers to moving the constraints of other methods (PEZ, etc.) to the main paper. If this is correct, we have moved it to lines 371-378. However, if it indicates other works [2,3], we have already discussed them in the main paper on lines 388-391.

[C1] Detecting, Explaining, and Mitigating Memorization in Diffusion Models, ICLR’24.

[C2] Extracting Training Data from Diffusion Models, USENIX’23.

[C3] A Reproducible Extraction of Training Images from Diffusion Models, Arxiv preprint.

W2. Bert (pre-2018) is outdated: could not find trigger prompts with new terminology

Thank you for pointing out this important aspect. To address the reviewer’s concern, we conducted several experiments proposing solutions. And we found that the number of trigger prompts containing new terms is not as significant as concerned.

W2-1. Have you considered using more recent language models?

During this rebuttal, we have implemented our MCMC algorithms with a model trained on the newer dataset, 2023 Wikipedia. We conducted 250 MCMC sampling and identified 55 unique memorized prompts. However, none of these prompts contained new terms and all of the generated images from those prompts were the images previously found in BERT. This suggests that sentences with new terms are likely rare and indicates that the 2018 BERT model we used is sufficient to uncover a significant portion of trigger prompts.

W2-2. Temporal solution for BERT

Additionally, we also propose a novel approach in Algorithm 1 to utilize BERT for finding prompts containing recent terminologies. Specifically, we replace the initially masked sentence “[MASK] [MASK] … [MASK]” with a reinitialized sentence that includes a new term, such as “[MASK] … [New Term] … [MASK],” before performing the MCMC sampling process.

To demonstrate the effectiveness of this method during the rebuttal, we conducted an experiment, where we trained Stable Diffusion to memorize 22 sentences containing new terms (e.g., COVID-19, Clubhouse, Tenet, …) and their corresponding images, to evaluate whether the proposed method works. In the trigger prompt searching process, we initialized sentences by randomly inserting these new terms into the masked format, such as “[MASK] … COVID-19 … [MASK],” and conducted MCMC sampling. As a result, we successfully retrieved 13 out of the 22 memorized images. This demonstrates that random initialization of new terms in a sentence can effectively retrieve the memorized images associated with them.

If there are any additional discussion points or questions, we are happy to discuss. Thank you.

I want to thank authors for addressing my comments. I think the paper adds value to the community. However I will still keep my score as 6.

We sincerely appreciate the reviewer's valuable feedback. There may have been some misunderstanding about our related work and motivation. We would be truly grateful if these points could be revisited during the rebuttal period.

Before addressing the questions, we summarize the contributions highlighted in the global response:

• MCMC trigger prompt searching algorithm: Only approach currently capable of searching trigger prompts that replicate train image where training data (LAION) is no longer accessible. Although without data, it requires significantly fewer computational resources than prior methods [C1, 2] that require training data.
• MemBench: We are the first to 1) provide general prompt scenario, 2) provide absolute standard for performance evaluation, and 3) evaluate image quality.

For details, we kindly invite the reviewer to the global response.

[C1] Extracting Training Data from Diffusion Models, USENIX’23.

[C2] A Reproducible Extraction of Training Images from Diffusion Models, Arxiv preprint.

W1. Concerns regarding motivation: Contrary to what is stated in lines 44-48, train time mitigation and inference time mitigations [C3, 4] do not require the construction of a trigger prompt dataset to operate.

• Full text of lines 44-48 that reviewer referenced: “As an adhoc assessment method, the current studies [C3, 4] have adopted the following workaround: 1) simulating memorization by fine-tuning T2I diffusion models for overfitting on a separate small and specific dataset of {image, prompt} pairs, and 2) assessing whether the images used in the fine-tuning are reproduced from the query prompts after applying mitigation methods.”
• Our motivation: As we explicitly mentioned in the phrase “As an ad hoc assessment method,” our motivation of the sentences was to highlight the inappropriate method these works use for evaluating their inference time mitigation methods, and we have not claimed that those methods require trigger prompt datasets to operate. Due to the limited number of trigger prompts inherently found in Stable Diffusion, prior works overly fine-tuned Stable Diffusion to artificially create trigger prompts and memorized images, which were then used to evaluate inference-time mitigation methods. We clearly confirmed this through email communications with the authors of Wen et al. [C3] before submitting our paper. As we stated in lines 48–50, demonstrating the effectiveness of mitigation methods on artificially fine-tuned Stable Diffusion does not guarantee their performance on trigger prompts already present in the original Stable Diffusion model. This gap underscores the necessity of our evaluation benchmark.
• Additionally, we thoroughly described both train-time and inference-time mitigation methods in the related work section and explicitly stated that our benchmark was developed to evaluate inference-time mitigation methods due to the current limitations of train-time mitigation methods. We kindly ask the reviewer to lines 88-111 in the revised paper for clarification.
• We kindly ask for additional clarification if our interpretation of the question is incorrect.

[C3] Detecting, Explaining, and Mitigating Memorization in Diffusion Models, ICLR’24.

[C4] Understanding and Mitigating Copying in Diffusion Models, CVPR’23.

W2. Limited Contribution W2-1. Prior methods [C3, 5] have utilized 500 memorized prompts from the prior dataset [C2]

• A) First, we’d like to clarify the fact: The prior method [C3] has not been tested on the mentioned dataset. Even the other work [C5] has evaluated their method on the suggested dataset but within only 325 prompts.
  • The prior dataset [C2] the reviewer referenced corresponds to the dataset compared with MemBench in Table 1 of the main paper. However, upon examining the GitHub repository of recent work [C2], we found that only 325 of the 500 samples are trigger prompts, while the remaining 175 are labeled as type 'N,' which denotes false samples rather than memorized images. The difference between 325 [C2] and 3000 samples (ours) is significant, and 325 is a tiny number for a test set. We contacted the authors of the referenced paper [C5], and they clearly confirmed via email that only 325 samples were used.
  • As stated in W1, another work [C3] the reviewer referenced does not evaluate the mitigation method on [C2], instead, their evaluations rely on an intentionally fine-tuned Stable Diffusion model.
  • Papers [C3, 5] conduct experiments exclusively on Stable Diffusion 1. However, whether a method that performs well on Stable Diffusion 1 would generalize to other models remains uncertain. In contrast, our work provides experiments on Stable Diffusion 2 in Table 4 of the supplementary material.

[C5] Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention, Arxiv preprint.

If there are any additional discussion points or questions, we are happy to discuss. Thank you.

W2. Metrics Are Not New (Wen et al. [C3] and Ren et al. [C4] utilized all the provided metrics)

We kindly ask the reviewer to revisit this point, as there seems to be a misunderstanding of related works. As stated in lines 63-65 of the main paper and the global response, we are the first to measure image quality, Aesthetic Score. Wen et al. have only measured Text-Image alignment and image similarity. While Ren et al. measured FID, FID does not provide meaningful information about image quality; thus, we propose its remedy. For details, we’d like to sincerely ask the reviewer to refer to the global response.

Furthermore, Ren et al. (Fig. 6.a in the referenced paper) did not assess text-image alignment for inference time mitigation methods. To address these selective omissions of metrics in individual studies, our work not only suggested the image quality metric (Aesthetic Score), but also highlighted the necessity of evaluating clearly defined metrics all at once to see multiple aspects.

W3. The proposed dataset construction method is not applicable to API-call-based models.

Our paper’s objective is to create a benchmark for evaluating existing mitigation methods, which are all in and limited to the white box diffusion model setup. Thus, applying these mitigation methods to API call models is not feasible. That’s why we focused our study on models with accessible weights. Therefore, this should not be a weakness of our work.

W4. Reverse Image Search API is inefficient.

The Tineye API that we utilized in the paper requires <0.1s per sample, which is efficient. As mentioned in the related work of the main paper, our approach is far more efficient than existing works [C1,2]. Furthermore, previous methods are no longer usable due to the inaccessibility of LAION 2B. In this regard, our approach is not only efficient but also effective in that our verification checks web-scale data beyond a single training dataset in contrast to the existing methods.

W5. lack of reasonable perspective & new metrics

In W1, we demonstrated the presence of a reasonable perspective, and in W2, we highlighted the advantages of the new metric, Aesthetic Score. Reviewer Dijx has admitted that providing additional trigger prompts as a dataset is beneficial for a more comprehensive evaluation. Additionally, we'd appreciate it if the reviewer could also see the contribution of our MCMC algorithm, which enables efficient construction of our dataset in contrast to the existing small-scale datasets, as highlighted in the global response.

We respectfully request the reviewer to revisit our clarification regarding the concerns raised and our notable contributions outlined in our global response and to redraw the evaluation during the rebuttal period. Thanks.

We thank the reviewer's feedback. However, there may have been some misunderstandings about our related work and motivation. We addressed all the comments. We would be truly grateful if these points could be revisited during the rebuttal period.

Before addressing the questions, we summarize the contributions highlighted in the global response:

• MCMC trigger prompt searching algorithm: Only approach currently capable of searching trigger prompts that replicate train image where training data (LAION) is no longer accessible. Contrary to W4, although without data, it requires significantly fewer computational resources than prior methods [C1, 2] that require training data.
• MemBench: We are the first to 1) provide general prompt scenario, 2) provide absolute standard for performance evaluation (contrary to W5), and 3) evaluate image quality (contrary to W2, 5).

For details, we kindly invite the reviewer to the global response.

[C1] Extracting Training Data from Diffusion Models, USENIX’23.

[C2] A Reproducible Extraction of Training Images from Diffusion Models, Arxiv preprint.

W1. Why do the authors consider the general prompt scenario given that a detection method exists?

This is because we found that the detection methods [C3, 4] accuracy are low in other models and the evaluation of detection methods in these works is limited to specific data. We have conducted a test to evaluate detection methods [C3, 4] according to different test datasets, using Stable Diffusion 2. The test datasets consisted of the union of memorized and non-memorized prompt subsets. While fixing the memorized subsets, we varied the non-memorized subsets across the union sets during the test. For non-memorized prompt setsets, Ren et al. [C4] used prompts generated by ChatGPT, while we randomly sampled 500 real prompts from the COCO dataset and DiffusionDB dataset [C5], and measured AUROC accordingly. We used the AUROC value to quantify the model's ability to distinguish between memorized and non-memorized prompts. For these non-memorized subsets used in this experiment, we have manually verified that the prompts were not memorized.

Table 1. AUROC of detection methods on Stable Diffusion 2 with the various non-memorized prompt datasets. GPT stands for a prompt dataset generated by ChatGPT, and the results are directly quoted by Ren et al.

As shown in the table, changing the non-memorized prompt dataset results in lower AUROC values. This implies that the detection methods exhibit overfitting behaviors to ChatGPT prompts. The decrease in AUROC for COCO and DiffusionDB compared to ChatGPT-generated prompts is due to the tendency of the detection methods to classify more detailed prompts as memorized prompts. Notably, DiffusionDB, which is a dataset comprising prompts from actual users of the Stable Diffusion, is much more suitable for evaluating the performance of detection methods. In particular, Ren et al.’s method requires identifying specific layers capable of detecting memorization through validation experiments with both memorized and non-memorized prompts. This imposes a significant challenge, as it requires full access to the training dataset of each model to accurately verify truly memorized prompts. In practice, this is often impractical, given that there are many generative models of which training datasets are inaccessible, e.g., LAION 2B is no longer accessible.

[C3] Detecting, Explaining, and Mitigating Memorization in Diffusion Models, ICLR’24.

[C4] Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention, Arxiv preprint.

[C5] DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models, ACL’23.

If there are any additional discussion points or questions, we are happy to discuss. Thank you.

Important) Why Aesthetic Score (ours) instead of FID (prev. work)?

Table 1. FID reported by Ren et. al. [C4] and FID measured on MemBench when mitigation methods are applied. FID was measured between the generated images from trigger prompts and the generated images from LAION (Ren et. al) or Flickr80 (Ours).

Reviewers’ major concern was that previous work [C4] reported FID, and therefore our Aesthetic Score is not a new metric that measures image quality. However, FID is defined to focus on the diversity of the generated image pool and, therefore fails to measure image quality in the memorization mitigation task. As shown in Table 1, FID decreases when mitigation methods are applied. The underlying reason behind the decrease in FID is that the mitigation method prevents memorized images from being generated, thereby increasing the diversity of generated images (which is also noted by Ren et. al.). Therefore, image quality should not be measured by FID, but separately measured by the Aesthetic Score. In this regard, we are the first paper that highlights image quality degradation.

Table 2. The mean and standard deviation of the Aesthetic Score were measured in MemBench.

To further comply with the reviewers' demand for metrics, we have measured the standard deviation of the Aesthetic Score. As shown in Table 2, when mitigation methods are applied, it leads not only to a lower Aesthetic Score but also to a larger standard deviation. This indicates that diffusion model outputs become unreliable.

We have included these discussions in the revised version of the paper.