Toward effective protection against diffusion-based mimicry through score distillation

Thanks for your time to review our paper, thanks for acknowledging our contributions, here let us answer your questions, hopefully it can address your concern.

The major concern I have is that how to evaluate the diffusion model still works well after the perturbation? This paper only provides results that the proposed SDS can defend against adversarial attack, but does not show the perturbation does not affect the original function of the diffusion model. The paper lacks both analysis and experiments on this point.

I think there may be some misunderstanding about the task, we only do perturbation to the image, and do not change the diffusion model, so it will not affect original function of the diffusion model. SDS is only an alternative loss function, and it can be easily applied without changing the original diffusion model. We are looking forward to get your response if there are still some concerns.

We also clarify our problem in the General Response.

The threat model is not clear. Based on my understanding, the core of this paper is adversarial attack and defense on diffusion model. However, the introduction of "diffusion mimicry" confuses the reader a lot. If authors really want to mention "diffusion mimicry", it should be explained that how diffusion model is used for mimicry and why the proposed method is a good defense?

Diffusion mimicry is a term used in [1], and here we mention mimicry as malicious editing of a individual image, which is a real-world problem, we ground it to protection against diffusion-based editing in our paper. Our experiments includes three scenarios: global image to image, inpainting and textual-inversion, all can be found in some real-world applications such as face swapping, subject-driven image generative, meme generation, style copying…

In conclusion, just like other works [1, 2, 3], the problem settings are the same, we ground the real-world problem into reasonable settings by attacking the diffusion model.

We also clarify our problem settings again in the General Response.

The paper lacks insight into why minimizing semantic loss works better.

It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it.

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian.

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

[1] Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models

[2] Raising the cost of malicious ai-powered image editing.

[3] Mist: Towards improved adversarial examples for diffusion models.

Thanks for your time to review our paper, thanks for acknowledging our contributions, here let us answer your questions, hopefully it can address your concern.

The observation that the encoder is more vulnerable than the denoiser only contributes to this region, it is limited in developing more secure LDM. Besides, the experiments empirically indicate this conclusion. It lacks theoretical proofs.

We believe that the encoder is more vulnerable is a meaningful finding and can contribute new insights to the community to show that the bottleneck of attacking a LDM is the encoder.

We further show in the Appendix C in revision that attacking a pixel-based DM using semantic loss can not work, which proves that attacking a DM is very different from attacking a LDM, where the sucess of attacking a LDM really comes from the encoder.

We showed a lot of evidences in Section 4 to support our claim, we do not try to focus on give a theoretical analysis but rather call emphasis on our insights. We write further clarification about our contribution in the General Response.

The authors resort to the existing method, Score Distillation Sampling, to fasten the backpropagation of the semantic loss.

SDS is a free lunch, but it is fully omitted by previous methods [1,2,3], which turns out to be too expensive to run. We are the first to apply SDS into the protection framework and did extensive experiments to show that it is effective. Also, we provide theoratical analysis to better explain the SDS loss from the perspective of gradient of KL divergence in revision of Appendix F in the protection framework.

The authors find that applying gradient descent over the semantic loss leads to more harmonious adversarial images with the original ones. The authors should delve into this phenomenon.

It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it.

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian.

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

Thank you so much for thoroughly reviewing our paper, your careful attention to detail is greatly appreciated. We are happy to answer some of your questions and we hope that we can address your concerns:

From my understanding, this method needs the gradient from the misused models. That is, the protection perturbation is generated on the exact model used to maliciously manipulate the image. It's not clear if this generated perturbation is only effective on that model, or if it can protect the image from misuse by an unknown model.

That is a good question. Here we focus on the white-box settings the same as previous works [1, 2, 3]. For the black-box settings as the reviewer mentioned, recent paper [4] found that the adversarial samples can transfer between the most popular publicly-avaible LDMs like SD-v1.4, SD-v1.5 and SD-v2. We also did experiments to show that the adversarial samples generated on the basic SD-v1.4 can be transferred to SD-v1.5 and SD-v2. We put the results in Appendix E of the revision following your suggestion.

No defense methods are evaluated. The malicious people may apply image transformations to remove the adversarial perturbations.

We test some famous defending methods, including JPEG compression, AdverseClean (https://github.com/lllyasviel/AdverseCleaner) and crop-and-rescale on the adversarial samples. We put additional results in Appendix D following your suggestion, we found that crop-and-resize turns out to be relatively effective, but it still cannot work well to remove the perturbations. This finding is in accordance with that also found in [3].

It's not clear why minimizing the loss (encouraging LDM to make better predictions) can protect the images. It's very counter-intuitive and needs a better explanation. I can understand that minimizing the loss with respect to certain targets as Photoguard can work because it encourages the LDM to generate an image different from the original image to protect. But why can encouraging the LDM to generate a better image of the original image (the opposite goal of AdvDM) can help? Maybe it's kind of trapping the sample into a local optimum so that the later optimization/editing is also stuck? But this also needs to be justified.

It is an interesting phenomenon that minimizing the semantic loss $\mathcal{L}_{S}(x)$ counter-intuitively fools the diffusion model, here we provide some insights into the possible reasons behind it.

Remember we have the semantic loss in SDS form can be regarded as the weighted probability density distillation loss (Eq.10 in Appendix G).

Here we can get some insights from the equation, minimizing the semantic loss means minimizing the KL divergence between $q(z_t|x)$ and $p_{\theta}(z_t)$, where $p_{\theta}(z_t)$ is the marginal distribution sharing the same score function learned with parameter $\theta$. Since $p_{\theta}(z_t)$ does not depend on $x$, it actually describes the learned distribution of the dataset smoothed with Gaussian.

Let's assume in a simple case that the data distribution is exactly the Dirac Delta Distribution of data points in the dataset. Then we have $p_{\theta}(z_t)$ as a Gaussian-smoothed composition of Dirac Delta Distribution if $s_{\theta}$ is perfectly learned. Then minimizing the semantic loss $\mathcal{L}_{S}(x)$ turns into making $q(z_t|x)$ closer to the Gaussian-smoothed composition of Dirac Delta Distribution, and the optimization direction is to make $x$ closer to the spatially average value in the dataset, which brings blurred $x$, and it turns out to be a good protection.

In Section 4.3, the conclusion of denoiser being more robust needs more evidence. Figure 2 shows that $\delta_z$ can be less than 10x$\delta_x$ for many cases. So I think the 10x budget (Figure 10 has even larger budgets) should be large enough to conduct some successful attacks. It's not clear why they fail.

That is a good question. This exactly supports our claim that the bottleneck is the encoder, when attacking the z-space alone, we do not use the gradient from the encoder, which brings a weak attack to the encoder itself. As we can see in Figure 10, although the budgets are getting even larger, the edited results do not diverge a lot from x_adv (which may be quite noisy since we perturb the z-space).

In conclusion, even given the same budget, simply attacking the z-space can not give us a good direction to attack the LDM. Encoder is really important and is the key when attacking a LDM.

To better support our conclusion, we did experiments (in Appendix C in revision) on a pixel-based diffusion model without encoder-decoder, and we found that gradient-based methods fail to work. That is, if we attack the DM using PGD and semantic loss, the generated adversarial sample cannot even fool the DM, the reason behind this may be the randomness brought by Gaussian noise term, making the denoiser quite robust to be attacked directly.

We are glad that we have solved most of your concerns and thank you for raising the score. We have added a clarification about Photoguard in our main paper in the new revision of Section 2.

For the memory consumption of Photoguard, in the early stage of this project, we found that when we use num_inference_steps=8 in the demo provided by Photoguard, it will take up ~47G on one A6000 GPU.

I just tried if we set num_inference_steps=4 as default, it will take up 29G on one A6000 GPU, and 30 minutes for one image, which is also too expensive. In conclusion, the diffusion attack of the Photoguard is too expensive to run. We also tried to run it on one 3090 and it did not work.

Also, in the official repo of photoguard(https://github.com/MadryLab/photoguard/blob/main/notebooks/demo_complex_attack_inpainting.ipynb), it said: Below is the implementation of an end2end attack on the stable diffusion pipeline using PGD. This requires a GPU Memory >= 40Gbs (we ran on 1 A100).

Thanks again for your really careful review and meaningful discussions.

Thank you again for your really careful review. We hope that we were able to address your concerns in our response. If possible, please let us know if you have any further questions before the reviewer-author discussion period ends. We are glad to address your further concerns, thanks.

I have raised my score.

But I suggest correcting the discussion about photoguard in the paper (because they do have an encoder attack and a diffusion attack) and talking about why only the simple encoder attack is evaluated, especially double-checking the claim regarding the +50G memory request.

We express our gratitude to the reviewer for conducting a careful review and providing valuable feedback. We highly appreciate the reviewer's recognition of the insights we provided and the effectiveness of our method.

Missing details when comparing the magnitude of embedding change versus input change The authors use absolute value to compare adversarial noise. While the embedding can be of different scale. The authors shall provide a relative scale of perturbation magnitude.

As we mentioned in Sec 4.1: line 5, we normalized the perturbations in the two spaces for comparison. Specifically, we scale the perturbations in the z-space back to [0, 1] as in the x-space.

Unclear math derivation In equation 5, why can we approximately remove the gradient parts from the denoiser?

This is a good question. Approximating the jacobian of U-Net as an identity matrix is used in many works, such as distillation from 2D [1, 2] and gradient approximation in adversarial attack [3]. While approximation is the most straightforward explanation, it can also be regarded as the weighted probability density distillation loss, we provide detailed analysis in Appendix G in the revision.

Also, by visualizing the loss curve in Appendix F, we better support the claim and it turns out that using SDS loss is a free lunch that should be noticed and applied when designing a protection method.

[1] DreamFusion: Text-to-3D using 2D Diffusion

[2] ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation

[3] Diffusion-based adversarial sample generation for improved stealthiness and controllability

Thank you again for your review. We hope that we were able to address your concerns in our response. If possible, please let us know if you have any further questions before the reviewer-author discussion period ends. We are glad to address your further concerns, thanks.

The authors do not clarify the problem settings.