Leveraging Model Guidance to Extract Training Data from Personalized Diffusion Models

Thank you for your insightful feedback!

1. For Weakness1:

I couldn't find the training data extraction result using the extracted prompt. In my opinion, this part is crucial for highlighting the paper's practical motivation, so the results should be reported.

Our default setting uses captions as accessible, aligning with previous works [1-3]. We observe that captions are accessible in many cases, particularly on real checkpoints on Civitai. This is especially evident for DreamBooth, where several special tokens within the training captions (such as "a sks dog") are always available to ensure the correct application of the models.

Nonetheless, there are scenarios where captions are not accessible, and we admit that data extraction result using the extracted prompt is helpful. Therefore, we update the results using the extracted captions (3 words in length) shown in Tab 5.

We observe that though the extraction success rate decreases compared with extraction with full captions, it is significantly stronger than performing extraction without any caption information. It confirms the effectiveness of the proposed caption extraction approach. Additionally, FineXtract achieves notably higher extraction performance under extracted captions compared to the baseline.

[1]Carlini N, Hayes J, Nasr M, et al. Extracting training data from diffusion models[C]//32nd USENIX Security Symposium (USENIX Security 23). 2023: 5253-5270.

[2]Somepalli G, Singla V, Goldblum M, et al. Diffusion art or digital forgery? investigating data replication in diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 6048-6058.

[3]Somepalli G, Singla V, Goldblum M, et al. Understanding and mitigating copying in diffusion models[J]. Advances in Neural Information Processing Systems, 2023, 36: 47783-47803.

2. For Weakness2:

An ablation study on the clustering method is essential, and further explanations, such as qualitative examples, would be helpful.

Without the clustering method, the attacker can only generate a number of data points but cannot determine which ones could lead to a successful extraction. This is reflected in our ablation study in Fig 4(b), where only 1 generated image per training image is used. The result is presented again in the following table to emphasize how the clustering component significantly improves the AS and the A-ESR.

3. For Questions:

How does the method perform on recent models, such as FLUX?

We update the results using FLUX.1 [dev]. The DreamBooth fine-tuning on FLUX.1 [dev] requires more than 100GB per GPU, which we are currently unable to run. However, we have conducted experiments in the LoRA scenario using the official scripts provided by Diffusers. The training iterations are fixed at 150$N_0$, as suggested by the repository, and other hyperparameters remain the same as those in the repo. We compare our method with the baseline (CFG), and both methods show the best performance when the guidance strength $w’$ is set to 3.0, with the improvement being consistent.

Thank you for your insightful feedback!

1. For Questions1:

It is commendable that the paper discusses strategies for partially extracting and extending captions when they are unavailable. However, did the authors evaluate FineXtract using these extracted captions? If so, how does the performance compare to using ground-truth captions?

As the caption extraction algorithm is not the core contribution of this paper, we focus on the extraction with full captions in our main experiments, consistent with the settings of previous works [1-3].

Nonetheless, we admit that evaluating FineXtract using the extracted captions is highly valuable for fully assessing its effectiveness. Therefore, we update the results using the extracted captions (3 words in length) shown in Tab 5. We observe that, although the extraction success rate decreases compared to using full captions, it remains significantly higher than when no caption information is used. FineXtract, when applied to extracted captions, achieves a notably higher extraction success rate compared to the baseline. This confirms the effectiveness of both the proposed caption extraction approach and FineXtract.

[1]Carlini N, Hayes J, Nasr M, et al. Extracting training data from diffusion models[C]//32nd USENIX Security Symposium (USENIX Security 23). 2023: 5253-5270.

[2]Somepalli G, Singla V, Goldblum M, et al. Diffusion art or digital forgery? investigating data replication in diffusion models[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 6048-6058.

[3]Somepalli G, Singla V, Goldblum M, et al. Understanding and mitigating copying in diffusion models[J]. Advances in Neural Information Processing Systems, 2023, 36: 47783-47803.

2. For Questions2:

Could the authors provide further discussion on the underlying principles or insights for identifying or setting the grid search range?

Furthermore, when the fine-tuning dataset and iteration count are fixed, does a larger optimal suggest that the model or fine-tuning approach is more prone to overfitting or memorization of the training data?

Intuitively, with longer training iterations, the distribution learned by the fine-tuned model should better align with the fine-tuned data distribution. In other words, $p_{\theta'}(x)$ should approach $q(x)$ and become more distant from $p_{\theta'}(x)$ in Eq. (3). Therefore, the optimal $\lambda$ should increase (closer to 1), and the optimal $w = \frac{1}{\lambda}$ should decrease with longer training iterations. Similiary, for conditional diffusion model in Eq. (7), the optimal $\lambda'$ should also increase with the optimal $w' = \frac{1}{\lambda'}$ decreases with longer training iterations.

We update the results using four classes of images from 4 classes of WikiArt under DreamBooth and perform a grid search to find the optimal hyperparameter $w'$ under different training iterations. We set the fixed term $k = 0$ for simplicity, focusing only on $w'$. The experiment results are available via an anonymous link: Grid search for the best w' for different classes.

Our findings show that, for our method, the optimal $w'$ tends to decrease (the optimal $\lambda'=\frac{1}{w'}$ tends to increase) as training iterations increase in most cases. However, due to the inherent randomness in the clustering process, this trend is not consistent across all scenarios.

We also observe that when the AS is higher, indicating that the model has memorized more, the optimal $w'$ tends to be smaller ( the optimal $\lambda'=\frac{1}{w'}$ tends be larger) . The experiment results are available via an anonymous link: Experiment result for AS at best $w'$.

In other words, a larger $\lambda'$ likely suggests that the model, or the fine-tuning approach, is more prone to overfitting or memorization of the training data, as noted by the reviewer.

We will include more extensive research on this aspect in the revised paper.

Thank you for your insightful feedback!

1. About Experiment Design and Analysis:

Experiments on model architectures only account for convolution-based diffusion. Current state-of-the-art generative models, like SD3, Flux or Sana, feature a transformer-based architecture. The paper would benefit from extending the evaluations to these.

We update new results using FLUX.1 [dev]. We experiment on Lora scenario with the official scripts provided by diffusers. The training iterations are fixed to 150$N_0$ as suggested by the repository and other hyper-parameters keep the same as the one in the repository. We compare our method with baseline (CFG), both methods show best performance when the guidance strength $w’$ is 3.0, where we can see the improvement is consistent. We will add these results in the revised paper.

2. About Weakness1 and Question:

The extraction success rate is significantly better than the baselines, but still nearly 20% of the total, which seems limited.

Given that the extraction success rate is around 20% in most cases, what factors do you believe limit the extraction of the remaining training data? Would further improvements to the guidance or clustering components potentially increase this rate, or are there fundamental limitations to how much information can be extracted?

We present updated results demonstrating how the extraction success rate increases with the number of training iterations across four classes of the WikiArt dataset, using DreamBooth as the fine-tuning method on SD v1.4.

These results suggest that the extraction performance is primarily dependent on the extent of the information the model memorizes. Our current result, above 20% at most cases, is based on the default training configuration and the current checkpoint. We believe this limitation is largely due to the model's inherent memorization capacity.

Potential improvements may arise from leveraging additional information within the model. Our current approach relies solely on the output of predicted noise, utilizing the model in an end-to-end manner. Future work could focus on further analysis, such as measuring attention responses to different inputs or optimizing input noise, which might enhance the method.

3. About Weakness2:

The computational requirements of running two models simultaneously are higher than the baseline (more GPU memory), which may limit practical applicability on consumer-grade equipment.

Even though we need to load two models onto the GPU simultaneously, this overhead is minimized because typically only a specific component is fine-tuned. For instance, in DreamBooth, the UNet is typically fine-tuned, while the text encoder remains unchanged. In LoRA, only the LoRA component is fine-tuned. Therefore, when using our method in such scenarios, the additional GPU memory usage is limited to the fine-tuned module. Moreover, FineXtract is an inference-only method and does not require the memory costs from gradient backpropagation. Therefore, loading two models does not limit practical applicability a lot on consumer-grade equipment in practice.

4. Other comments or suggestions:

Additional visualizations showing the distribution shifts between pretrained and fine-tuned models could help readers better understand the theoretical foundations.

Thank you for your valuable advice. We will revise the paper accordingly.

Thank you for your valuable feedback. We greatly appreciate your positive comments!