Rethinking Adversarial Attacks as Protection Against Diffusion-based Mimicry

We express our gratitude to the reviewer for the careful review and valuable feedback. Here I will answer some questions that can hopefully address your concern:

W1: (1) Figures 3 and 4 seem to be a rudimentary case study rather than a comprehensive experiment; (2) Fig. 4 that Shih et al.'s attack needs perturbation buget of gives a sense that this attack is not truely adaptive or not correctly applied (3) settings of SDEdit

(1) Thank you for pointing it out! We will include additional results in the appendix in the next version.

(2) Regarding the settings, we follow those in Shih et al. Their approach maintains a budget in the encoded latent space using LPIPS, as opposed to relying on an L-infinity budget in pixel space.

(3) For SDEdit, we use step sizes smaller than 0.5, specifically 0.1, 0.3, and 0.5. Since no text prompts are used as conditions, the resulting changes are limited. However, this does not impact performance when text prompts are applied, as evidenced by image inpainting examples in prior protection papers, such as Mist.

W2: (1) I recommend that the authors further analyze and hypothesize the reasons behind this robustness to enhance the reliability of their findings and deepen our understanding of the method’s mechanisms. (2) recent works about diffusion model as a zero-shot classifier

(1) Thanks for pointing it out. Regarding why PDMs are so robust, while the rigorous theory of the mechanism remains largely unexplored, we attempt to explain it intuitively here: (*) From the training of the network: The denoisers in PDMs are trained with extensive data augmentation, including Gaussian noise, which makes them quite robust to small perturbations; while for LDMs, the networks are fooled because the inputs are already out of the distribution of the dataset due to the huge perturbation in the latent space (**) From the structure of diffusion models: The input to the network includes a high level of randomness, as new noise is injected at different timesteps, making the whole system includes high level of randomness, making it robust.

(2) Yes, conditional diffusion models can indeed function as zero-shot classifiers [1]. Furthermore, studies such as [2, 3] highlight that diffusion classifiers exhibit strong certified robustness. These works strongly support the insights presented in our paper. Due to the inherent stochasticity of the diffusion process, it is notably challenging to design adversarial attacks against diffusion models, unlike the more straightforward vulnerabilities observed in other deep neural networks.

Comparison with Diff-Pure and GrIDPure

The focus of our study is the adversarial-robustness of PDMs, which is novel and insightful. PDM-Pure is motivated from this finding, while GrIDPure does no realize that it is because of the PDM that matters (they did not mention any PDM in their paper). As Line 356-358 we propose a generalized framework to purify any noise, which is based on the assumption that PDM is robust to any small noise since it is adversarially robust.

Both PDM-Pure and Diff-Pure operate on the assumption that diffusion models can eliminate out-of-distribution noise patterns through a diffuse-then-denoise process. However, our motivation diverges: given the insight that PDMs are exceptionally robust to adversarial attacks, we hypothesize that they can mitigate "any" noise using the Diff-Pure pipeline, provided it’s sufficiently robust. LDMs, by contrast, cannot achieve this due to their vulnerability to adversarially crafted noise. Notably, Diff-Pure did not highlight the adversarial robustness of PDMs, which is a core focus of our paper.

Q1: How does the technical similarity between your paper and prior works such as GrIDPure and DiffPure impact the novelty and contribution of your study?

See W3

Q2: PDMs are also neural networks after all. What might account for their empirical robustness to adversarial perturbations?

See W2

Q3: Can the adversarial robustness of PDMs in generative tasks also be applied to classification tasks?

Yes, [2,3] exactly support the findings in our paper. It is promising for future works to design classification / regression models based on diffusion model, which shows strong adversarial robustness.

Q4: I think another limitaion is that PDMs may not scale as good as LDMs, making the potential of PDM-Pure for large scale purification difficult. Can the authors share their opions and comments on this aspect?

That is not true. First, PDMs are very strong—models like DeepFloyd, Hourglass Diffusion Transformers, and Ediff-I demonstrate this strength. Additionally, very recent work like Edify-Image [a] pushes the scale to the extreme. By applying techniques like pyramid Laplacian structures, denoising becomes more effective. I believe it will be quite promising to use large PDMs as purifiers.

[a] https://huggingface.co/papers/2411.07126

Dear authors, thanks for the new responses. Accordingly, I have raised my score to an 8. I think this paper brings some novel insights and proposes a simple yet practical method, which is an important contribution to the field and thus worth acceptance at this year's ICLR. I also agree with other reviewers that some related works about using DMs to purify adversarial noises, as well as more discussions, should be added to the next version of the paper. This paper is likely to have high impact beyond the current scope and I hope the authors to extend the discussion of this paper for future work.

Thanks for your response and again for your careful review! I am happy to answer your following questions and hopefully it can address your concerns.

W1

Yes, I believe that your thoughts align exactly with what we want to convey in the paper! We also think that the attacks presented in Shih et al. are not in a good setting. However, Shih et al. is a recent work that claims to have found a good adaptive attack for PDM. We applied their attack and found that it is not in a good setting: the budget norm is too large, which actually introduces significant perturbations into the image.

We will add the clarification about SDEdit settings in the next revision in a few days.

W2

Thanks, we will add more discussions about that in the next revsion.

Q4

Yes, I agree that scaling up PDM will be more challenging compared to LDM, many tricks should be applied otherwise the training will be too costful. However, for PDM-Pure, we may not require an expert-level PDM scaled to extensive levels. Since we're not generating images from pure noise but instead using SDEdit, the current PDMs, such as Edify-Image and DeepFloyd, might already be sufficiently strong.

We express our gratitude to the reviewer for acknowledging our paper and providing valuable feedback. Here I will answer some questions that can hopefully address your concern:

W1: While the experiments are extensive, they may not cover all potential adversarial attack methods or variations in model architectures that could further validate the findings.

We show the empirical adversarial robustness of PDMs by applying various attacks as far as we know, we conclude that it is clear that PDM is much more robustness than LDMs. For model archs, U-Net and Transformers are two main structures for diffusion models and are used by most models, while there is currently not transformer for PDMs, we believe there could shares similar robustness.

W2: The conclusion that PDMs are universally robust may require further exploration under different conditions or with more sophisticated attack strategies that were not tested in this study.

We tested various of attacks including latent attack and EOT, targeted attack and un-targeted attack, PDM shows strong empirical adv-robustness. Also, there are some recent evidences that diffusion model can be strong classifier with certified robustness [a, b].

W3: The paper could benefit from a more thorough discussion of the limitations of their approach and potential scenarios where PDMs might still be vulnerable to attacks.

Thanks for pointing it out. Currently we do not see an attack that can effectively attack PDMs. We will have a discussion of it in our final version.

[a]: Chen et al. Robust Classification via a Single Diffusion Model. ICML 2024.

[b]: Chen et al. Diffusion Models are Certifiably Robust Classifiers. NeurIPS 2024.

Thanks, for answering most of my questions.

ad W2: I would not give so much on EoT actually (even though is important to have some evaluations on it), this just means that the attack can be successful after certain transformation, too. Honestly, I recommend study semantic adversarial attacks (not gradient-based attacks) - mostly created by diffusion models [1,2]:

• [1] https://github.com/WindVChen/DiffAttack
• [2] https://arxiv.org/abs/2309.07398

I hope this strengthens your work.

ad W3: like W2

Q1: Would you share a code base? I don't understand, why not sharing. An initial partial code base would be good. People would be motivated to compare and consequently cite more of your paper. You also emphasize that your paper is empirical work.

Thanks for your very careful review! Here I will answer some questions that can hopefully address your concern:

W1: The paper is not well-prepared, with many typos, formatting errors, and grammar issues:

Sorry for that, we changed all the typos that have been mentioned, please refer to ** Revision Summary ** for the updates.

W2: While I understand that attacks designed for latent space are less effective in pixel space, the authors seem to spend excessive space discussing this point (Section 4 already addressed this point, but Section 6.2 reiterates it.). It’s fairly intuitive that different optimization losses lead to this limitation.

Section 4 aims to demonstrate the remarkable adversarial robustness of the denoising-diffusion process. In Section 4.1, we present attack results using the diffusion loss, a widely used method to attack diffusion models. Section 4.2 details our efforts to craft more adaptive attacks specifically for PDMs, while Section 4.3 explains why LDMs are vulnerable despite being diffusion models.

Section 6.2, part of the experiments section, provides a detailed analysis of the settings and metrics used in Sections 4.2 and 4.3. Although there may be some overlap, we will reorganize these sections for clarity in the final revision.

Even when using the same optimization loss (Eq. 1) — the well-known diffusion loss — we find that we can effectively attack LDMs, but not PDMs. This is a significant observation, as we are the first to highlight this difference, which has been largely overlooked in this field.

W 2.1: Could you elaborate on your two categories of attacks (lines 247-248)? From my understanding, a reasonable approach might be to adapt the loss in Equation 1 to match the loss function used in pixel-based models. Additionally, it could help to incorporate information from intermediate layers, run EOT multiple times, and use these gradients to update adversarial images. Based on the current description, it’s unclear if the adaptive attack implemented here is sufficiently strong.

Sure, we are glad to elaborate our two kinds of attacks (lines 247 - 248):

• Attack the full pipelines, which is adopted in [a], regard the multi-step diffusion-based editing process of a clean images as a end-to-end pipeline, which is typically impractical but turns out to be stronger in [a].

• Attack the diffusion loss means attack the PDM follow Equation 1, but changing $z_t$ to $x_t$

We indeed proposes to incorporate information from middle layers of U-Net and run EOT to aggregate gradients in the same section (line 253-257). None of the proposed attacks can effectively attack PDMs.

W 2.2: The authors use Figure 2 to explain why pixel space adversarial attacks are not effective, particularly due to the large overlap. However, for traditional adversarial attacks on ResNet models using the ImageNet dataset, the adversarial examples are generated through pixel-level updates, no? This method has also proven to be very successful. any difference here?

That’s a good question. The difference is that, before input into the network, a large scale of noise is added to $x$, the input is actually sampled from a distribution $N(ax, b^2I)$ where $a,b$ are defined by the diffusion process. While for previous adversarial attacks, the pixel updates will be directly input into the classifier, making the attack much more effective.

W 2.3: In line 318, what do you mean by "domain shift"? and why?

Domain-shift means: the attacked image results in a latent $z’_t$ that significantly deviates from the training / testing distribution of $z_t$ (clean latents). The difference of it from adversarial perturbation is that the perturbation of $z_t$ is much larger, Figure 2 in [b] gives a good visualization for that.

[a] H. Salman, A. Khaddaj, G. Leclerc, A. Ilyas, and A. Madry. Raising the cost of malicious ai-powered image editing. arXiv preprint arXiv:2302.06588, 2023.

[b] Xue, H., Liang, C., Wu, X., & Chen, Y. (2023, October). Toward effective protection against diffusion-based mimicry through score distillation. In The Twelfth International Conference on Learning Representations.

[c] Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI. arxiv: 2406.12027

Dear Reviewer sKMW, thank you very much for your prompt, thoughtful, and detailed response, and for your patience throughout this review process. I am truly impressed by your deep understanding of the field and also your insightful comments and I have learned a lot from your dicussion. I would like to continue our discussion as follows:

• Regarding Paper Writing, I completely agree with you that the manuscript contains numerous typos and grammatical errors. I too have noticed some of these errors in the initial review, and I apologize for any confusion caused. To clarify, my comment was meant to convey that the paper is generally logically coherent and easy to follow, despite these language issues. I fully support your suggestions, and I will encourage the authors to address these concerns thoroughly in the revised manuscript.

• Regarding Adaptive Attacks, I am grateful for your deep understanding of adversarial machine learning and adaptive attacks. I also agree with your point that the term "adaptive attack" is often misused in the current literature. Only defenses with certified guarantees of robustness can genuinely claim resilience to adaptively crafted attacks. Many existing papers tend to label certain heuristic or recent attacks as "adaptive," even when these attacks are not intuitively effective. In my view, these should be regarded as "potential countermeasures" rather than true adaptive attacks. For example, some papers use AutoAttack as an example of an "adaptive attack," even though it often performs worse than PGD.

I agree that the authors' claim that "PDM is robust to adaptive attacks" is somewhat overstated without stronger empirical or theoretical support. As such, I have also recommended that the authors revise this assertion in the manuscript.

That said, I do find the authors' empirical explanation for the robustness of PDM to be reasonable. Since diffusion models are trained under Gaussian noising-denoising, this process shares some similarities with adversarial training, which may contribute to their robustness. Moreover, there is growing evidence in the literature, such as the work by Chen et al. and others exploring diffusion models for adversarial purification, suggesting that diffusion models may offer inherent robustness, at least in classification tasks. So I wonder if this inherent robustness might also apply to generative purification tasks, and I guess the authors' focus is on empirical evidence rather than certified robustness at this stage. As the first paper to investigate the robustness of PDMs under attack within the framework of existing attacks on latent diffusion models, the authors provide valuable empirical results and reasonable explanations. In my humble opinion, this paper somewhat contributes to interested audiences, and so I am inclined to support its acceptance at ICLR.

• Regarding the Ensemble Everything Everywhere Paper: I agree with your assessment of the recent Ensemble Everything Everywhere paper. It is problematic, and as you pointed out, and we cannot learn anything but conducting the right experiments on adaptive attacks is crucial for understanding adversarial robustness. This paper fails to offer significant insights for two reasons: (a) it combines and extends past defense techniques that have been shown to be ineffective, and (b) it does not evaluate its defense against adaptive countermeasures that are known to be potentially effective. For example, the authors do not recognize that their attacks exhibit clear signs of gradient masking, which, while effective against many existing attacks, can be bypassed with simple countermeasures (e.g., the EoT attack as you mentioned). Yet I believe this paper does not satisfy any of these reasons, so I hesitant to agree that breaking it is trivial.

• Regarding the New Adaptive Attack on DiffPure: thank you for pointing out the recent work that challenges DiffPure. I must admit that I have not been closely following the latest developments in this area, as my primary focus lies elsewhere. My previous understanding was that DiffPure offered non-trivial robustness, and I was unaware of this recent work. After carefully reading the paper you referenced, I now realize that it offers important insights. This new attack suggests that DiffPure’s robustness is (mostly) a result of its stochastic nature, and that it does not claim to bypass DiffPure under real black-box setting (the attack is successful under the assumption that the stochasticity is known to the attacker). Therefore, I still believe that diffusion-based purification remains a non-trivial defense to bypass. I would encourage the authors to carefully consider this recent work and include a discussion of it in their revision.

Once again, thank you for your invaluable feedback and constructive suggestions. I believe that these would greatly help the authors.

Hi dear colleague, thank you very much for your valuable time and thoughtful review. I have carefully read your comments and the subsequent discussion with the authors. I completely agree with your point regarding the authors' initial claim that "PDMs are robust against adaptive attacks," which indeed lacks sufficient theoretical support and could potentially mislead readers. I also share similar concerns about the problematic claim that PDMs are robust against Shih et al.'s attack for budget 150/255 in my initial comments (as noted in my W#1). They have since acknowledged these concerns and have worked on revising their claims accordingly.

That being said, I would like to respectively emphasize that while I agree that PDMPure may eventually be defeated by stronger adaptive attacks in the future, I humbly don't believe this should be the sole reason for rejecting the paper. After all, security is an evolving game between attacks and defenses, the community's understanding goes step-by-step, so it is natural for defenses to be challenged over time, especially for attacks/defenses against diffusion models, a relatively new research field. It is not fresh to see that strong empirical defenses published on top conferences are quickly defeated in a few months (or even days). However, this should not diminish the value of such works. Overall, this paper offers valuable insights to the community and presents a defense that, despite its simplicity, effectively defeats many existing complex defenses.

In light of this, even though the paper's claims have room for improvement and there is no theoretical guarantee for its robustness, I believe it should not be dismissed entirely. The paper is well-written, contributes new perspectives to the field, and introduces a practical and effective method that is worth sharing with the community. Overall, this paper does not contain obvious mistakes or evaluation faults (at least in the current manuscript, to my knowledge), and I believe it would be unfair for the authors to reject it simply based on the reason that "it may be defeated by future stronger attacks".

Would you mind reconsidering the rating for this paper, taking these points into account? Thank you again for your time and thoughtful feedback.

Thank you for the further discussion! I'm pleased to see such a heated discussion about this paper! Regarding my concerns:

1. "The paper is well-written, contributes new perspectives to the field, and introduces a practical and effective method that is worth sharing with the community. "

I do not fully agree with the submitted version. There are numerous typos, and the formatting is not well done. Even after the authors revised it twice, I still have concerns about the writing. The current narrative focuses too heavily on demonstrating that "attacks for LDM do not work for PDM" and emphasizes the need to focus on PDM. While this reasoning is intuitive, it does not require extensive elaboration. Meanwhile, the authors have devoted less effort to convincingly explaining why PDM-PURE is strong, particularly in Section 5.

I would suggest the following: (1) Invest more effort into understanding adaptive attacks to see if PDM is genuinely robust, and if so, explain why; (2) If they are indeed robust, clarify how this supports the case for PDM-PURE.

2. in terms of adaptive attack, the authors said " The denoisers in PDMs are trained with extensive data augmentation, including Gaussian noise, which makes them quite robust to small perturbations, ...., The input to the network includes a high level of randomness, as new noise is injected at different timesteps, making the whole system includes high level of randomness, making it robust."

If this is the reason the authors believe PDMs are robust against adaptive attacks, I would like to mention a few examples of such attacks. For instance, reference [1] introduces various random augmentations and noise, which cause state-of-the-art attacks to fail against this defense. However, an adaptive attack that simply employs EoT, with fixed randomness, and does transfer attacks, can effectively break this defense, as noted in [2]. One more example that breaks diffpure is in [3].

Similarly, what if you were to implement this kind of adaptive attack? Based on experience with attacks and defenses, it seems that only certified robustness and adversarial training provide a guarantee of robustness against adaptive attacks. I would be interested to learn if there is a third approach, such as the method proposed in this paper. If PDM turns out to have some weaknesses against adaptive attacks, what are some valuable lessons we could still take away from that? That's more what I was concerned about.

[1] Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness. arxiv :2408.05446

[2] https://openreview.net/forum?id=IHRQif8VQC. check public comments.

[3] Towards Understanding the Robustness of Diffusion-Based Purification: A Stochastic Perspective. arxiv 2404.14309

We are pleasantly surprised by the enthusiastic discussion about this work. We deeply appreciate the contributions of Reviewers 6bSH and sKMW, whose insights may helped enhance the clarity and impact of the paper for the broader community.

About the writing @ Reviewer sKMW

Regarding typos, we did a thorough proof-reading and fixed most of the typos. We believe the paper is easy to read and easy to understand, we will double-check it for the final revision.

Regarding the organization, we kindly disagree with Reviewer sKMW that demonstrating that "attacks for LDM do not work for PDM" and emphasizes the need to focus on PDM. First we do not think it is intuitive, before this paper, people has a false understanding that DM is vulnerable because LDM is vulnerable, it is worth put strength to show that attacking the diffusion loss will not fool PDMs.

About the adaptive attacks in [3] @ Reviewer 6bSH and sKMW

Here, we would like to elaborate further on [3], as we are very familiar with this line of work.

The original Diff-Pure approach in this context is designed to purify noise so that classifier is not misled. To highlight the difference in the pipeline for our Diff-Pure, consider the following:

• [pipeline 1] Attacked Image $\rightarrow$ SDEdit of PDM $\rightarrow$ Classifier
• [pipeline 2] Attacked Image $\rightarrow$ SDEdit of PDM $\rightarrow$ Human-eyes

In both pipelines, we employ the same attacks targeting SDEdit. Reference [a,b] demonstrates that it is possible to attack the entire pipeline to generate samples that can deceive the classifier.

Now, consider the difference in the attack goals:

• To fool Pipeline 1, we only need to deceive the classifier. This is relatively straightforward, as even a small perturbation can suffice to mislead the classifier, and [a] even uses realistic outputs to achieve this.
• In contrast, to fool Pipeline 2, we must significantly alter the output of the PDM to deceive human observers. This task is considerably more challenging because it requires dramatic changes to the image, which are harder to achieve without being easily detected.

This distinction highlights that the success of Diff-Pure in Pipeline 1 does not necessarily mean something to Pipeline 2, as the robustness criteria differ fundamentally between the two.

[a] Diffusion-Based Adversarial Sample Generation for Improved Stealthiness and Controllability

[b] DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification

We express our gratitude to the reviewer for conducting a careful review and providing valuable feedback. Here I will answer some questions that can hopefully address your concern:

W1: Comparison with Diff-Pure (Nie et al., 2022) and other similar works.

That's a great point. While PDM-Pure resembles Diff-Pure in structure, it addresses a different scenario. In most research, such as Diff-Pure, diffusion models with SDEdit are employed to mitigate adversarial perturbations in classification tasks. Our focus, however, is on removing adversarial noise that targets diffusion models directly.

Both PDM-Pure and Diff-Pure operate on the assumption that diffusion models can eliminate out-of-distribution noise patterns through a diffuse-then-denoise process. However, our motivation diverges: given the insight that PDMs are exceptionally robust to adversarial attacks, we hypothesize that they can mitigate "any" noise using the Diff-Pure pipeline, provided it’s sufficiently robust. LDMs, by contrast, cannot achieve this due to their vulnerability to adversarially crafted noise. Notably, Diff-Pure did not highlight the adversarial robustness of PDMs, which is a core focus of our paper.

W2: Other metrics for PDM-Pure experiments

Thanks for pointing it out. We add additional results about other metrics in the Appendix, please refer to ** Revision Summary ** for the updates.