Revealing the Unseen: Guiding Personalized Diffusion Models to Expose Training Data

We thank the reviewer for valuable feedback, and would like to address your concerns as belows:

For Weakness1:

baselines under various $\rm{N}$ and $\rm{N_0}$ deserves further discussion.

We updated the results for different $\rm{N}$ and $\rm{N_0}$ values compared to the baseline in Fig. 4(a) and Fig. 4(b), where FineXtract consistently outperforms baseline under different $\rm{N}$ and $\rm{N_0}$.

For Weakness2:

why Correction Term Scale performs better in the negative case needs further analysis

Our method in Sec. 4 is a parametric approximation. We provide further discussion by comparing model guidance using conditional DM, CFG, and a combination of CFG and model guidance (with $k=0$). As shown in Fig.10 in Appendix Sec.H , model guidance using conditional DM achieves close performance to the combination method and significantly outperforms CFG alone, suggesting model guidance’s dominance and a potential misalignment with CFG, affecting the parameter $k$.

For Weakness3:

It is worrying whether using PCA to extract important signals from multiple-words prompts is feasible when $\rm{W}$ is large.

Increasing $\rm{W}$ indeed makes caption extraction more difficult. We add some experiments on larger $\rm{W}$ in Appendix Sec. J, where we use GPT-4 to generate extended artist style descriptions. Our results show that keywords, such as the artist’s name, remain extractable as long as the training prompt is not excessively long. This indicates that some unique words in the fine-tuning process still lead to partial information leakage.

For Weakness4: symbol errors in Appendix A.3.

Thank you for pointing it out. We have revised it accordingly.

We thank the reviewer for valuable feedback, and would like to address your concerns as belows:

1. For Weakness1:

Will the growth of class number have the same effect (Performance drop in extraction)?

Yes, increasing the number of classes makes it harder for diffusion models to learn the fine-tuned data distribution, which in turn results in more challenging extraction and an observed performance drop. Specifically:

We conducted experiments with an increasing number of classes and found that while it reduces our method’s performance, we still significantly outperform the baseline. As shown in Sec.G.2, we observe that as the number of styles increases, the extraction success rate decreases. Additionally, we evaluated the model's ability to learn the input distribution and found that a higher number of classes leads to lower fidelity (DINO) and image quality (Clip-IQA), reflecting the model's difficulty in learning the fine-tuning data distribution accurately.

How can this issue be mitigated in practice

To further enhance extraction under these scenarios, utilizing accessible public datasets may be helpful. For example, extracting checkpoints trained in a specific artist's style could be improved by leveraging reference images of similar styles. This direction is also part of our future work.

2. For Weakness2:

Does this (LoRA with poorer extraction performance) suggest that the proposed method is less effective for parameter-efficient tuning?

No. Our FineXtract appears to perform worse on parameter-effecient tuning methods like LoRA because the fine-tuned models learn worse on the data distribution. For evidence, we performed an ablation study on different training iterations for LoRA with other training parameters for LoRA and DreamBooth following previous work [1,2].

We found that performance issues do not stem from LoRA itself but from how well the model learns the fine-tuning data distribution. As shown in the table below, LoRA also achieves a higher extraction success rate than DreamBooth when LoRA’s training iterations are extended. So the key factor for extraction success is more about how well a DM learns the fine-tuning data distribution, instead of the fine-tuning method itself.

3. For Weakness3:

Could you provide statistics on the extent of unusability? To what degree does the attacker lose model utility to achieve the attack performance reported in Table 3?

Measuring usability precisely is challenging, but we attempted to quantify it using a no-reference image quality score and the image fidelity measurements. We follow previous works [1], using DINO for image fidelity measurements. For image quality measurements, we use Clip-IQA [2]. We update some experiment results in Appendix Sec. I and find that the CutOut and RandAugment degraded Clip-IQA by 6.1% and 4.6% in the original dataset, and generated images experienced around 3.8% and 4.6% degradation. These extent are close in the sense that the degration brought by the transformation largely preserves in the generation process. For RandAugment, the image fidelity also largely degrades. This suggests that the preprocessing steps applied to the images (removing a square section or adding high contrast to the input images) largely persist in the generated output images, hindering extraction methods from obtaining high-quality images while sacrificing generation quality of diffusion models themselves.

In summary, there exists a trade-off between image quality and defensive effects for these preprocessing methods. We leave the accurate modeling of this trade-off and further improvement as an interesting future work.

[1] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. In CVPR, 2023

[2] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring Clip for Assessing the Look and Feel of Images. In AAAI, 2023.

We thank the reviewer for valuable feedback, and would like to address your concerns as below:

1. For Weakness1 & Question3:

in real-world settings the pre-trained models might be not always available ... the fine-tuned model could be a mixture of multiple fine-tuned data distributions and pre-trained models.

Can you conduct more analyses on the impact of different combinations of DMs and training data (artistic styles vs. object)?

From our analysis of real-world checkpoints available online, such as those on Civitai and Hugging Face, pretrained model checkpoints are typically clearly stated or provided with direct download links. Meanwhile, parameter-efficient fine-tuned methods such as LoRA commonly require alignment with the exact pretrained model to correctly load an adapter, making pretrained model checkpoints even more accessible.

As for the training data distribution for few-shot fine-tuning, based on previous work [1] and the checkpoints we examined online, a specific style or object is typically associated with learning from data corresponding to a single class. Nonetheless, we fully agree with the reviewers that extending to more diverse scenarios would indeed be valuable. Specifically:

Mixture of data: We constructed a new dataset with 10 classes, each containing 5 images from DreamBooth and 5 from WikiArt:

More details about the experiment are available in Appendix Sec. G.1 in the revised paper. We can observe that this mixture led to mutual decreases in the fine-tuned model's fidelity (measured by DINO score [1]), image quality (measured by CLIP-IQA [2]), and extraction rate. Therefore, extracting training images from this model becomes more challenging. Nonetheless, FineXtract still significantly outperforms the baseline despite the fact that its performance partially drops compared to the original cases, further verifying its generality and effectiveness.

About the mixture of pretrained checkpoints: To the best of our knowledge, we are not aware of any prior work on data extraction or, more broadly, privacy-related topics in diffusion models that discusses the combination of different pretrained checkpoints. We are unsure about the specific mixture the reviewer is referring to and would greatly appreciate further clarification on this point.

2. For Weakness2 & Question2:

When increasing $\rm{N_0}$, the performance drops significantly (figure 4a), the model does not work very well on fine-tuned models with a larger scale of images. These results suggest space for improvements, which are all needed from this work considering the applications of real-world problems.

What's the performance of extraction accuracy with increasing fine-tuning data $\rm{N_0}$?

In practice, the fine-tuning dataset size $N_0$ is usually fewer than $50$ to learn a more specific style or object. Under this scale, our FineXtract approach performs well, as supported by results on randomly chosen real-world checkpoints available from HuggingFace with different $N_0$ presented in Tab. 4 and 7.

As $N_0$ increases, the model is less likely to memorize individual images, hence extraction indeed becomes more challenging. To further confirm the superiority of our FineXtract method under this more challenging scenario, we conducted evaluation across increasing $N_0$ as shown in Fig. 4(a). Our FineXtract method significantly outperforms existing CFG approaches, further validating its effectiveness and generality.

3. For Question1 about intuition of Eq (3):

Our formulation in Eq. (3) basically assumes fine-tuning learns a mixture of pretrained model distribution and fine-tuning data distribution. The variable $\lambda$ $\in [0, 1]$ reflects how well the fine-tuning model memorizes the fine-tuned data distribution. If the model exactly memorizes the fine-tuning data distribution, potentially completely overfitting on it, $\lambda$ would be 1. Thus, any factors increasing memorization ability, such as larger training iterations, smaller fine-tuning datasets, or reduced complexity in the dataset, should increase $\lambda$.

[1] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. In CVPR, 2023

[2] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring Clip for Assessing the Look and Feel of Images. In AAAI, 2023.

We thank the reviewer for valuable feedback, and would like to address your concerns as belows:

1. For Weakness1:

The paper only chooses 'Direct Text2img+Clustering' and 'Classifier Free Guidance + Clustering' as baselines. I think these two methods are only ablation counterparts.

These two baselines are not ablation counterparts. To the best of our knowledge, prior work on data extraction from diffusion models [1,2,3] primarily discusses methods related to direct text-to-image extraction [1] and CFG-based approaches [2,3]. However, no existing methods specifically target the extraction of fine-tuning datasets. Therefore, the baselines listed here represent all the available methods we could identify from prior work related to diffusion model extraction.

2. For Weakness2:

The proposed method seems sensitive to the guidance scale $w'$ and correction term $k$ . How to decide the hyper-parameters in practice might be challenging.

Firstly, the improvement from our proposed FineXtract is not sensitive to the hyperparameters. As shown in Tab. 5, the best performance of the baseline method is $AS\approx 0.44$ with only $w' = 2$ or $3$. However, FineXtract performs better under a large scale of guidance scale $w'$ and correction term $k$, i.e. $w'\in [2, 10]$ and all $k$s.

Secondly, we do not heavily rely on hyperparameter tuning. Concretely, our approach involves searching on 4 classes of WikiArt and applying the resulting hyperparameters universally across various scenarios, including the DreamBooth dataset, all 10 classes of WikiArt, and real-world checkpoints (where most training details remain inaccessible). Developing more tailored hyperparameters for specific scenarios should further enhance performance.

3. For Weakness3:

It seems feasible to simply apply existing methods for extracting training images directly on the fine-tuned checkpoint, then filter out the results that overlap with images extracted from the pretrained model.

In principle, the overlapping from extracted train images from fine-tuned checkpoint and pretrained model can only be the images from the pretrained dataset, while our task is to extract training data used during the fine-tuning process, as clearly stated in Page 1. In practice, previous works on extracting pretrained diffusion models [1] successfully extracted fewer than 100 images, none of which overlaps with the fine-tuning dataset and this number is much lower than the size of its training dataset size (more than 200K). On the other hand, our main results in Tab. 1 demonstrate that directly applying existing methods (i.e., the shown baselines) performs significantly worse than our proposed method on the fine-tuned data extraction task, only achieving around half of our extraction success rates in many cases.

[1] Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253–5270, 2023

[2] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6048–6058, 2023

[3] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Under standing and mitigating copying in diffusion models. Advances in Neural Information Processing Systems, 36:47783–47803, 2023b.