A Change of Heart: Backdoor Attacks on Security-Centric Diffusion Models

We thank the reviewer for the valuable feedback on improving this paper! Please find below our response to the reviewer’s questions.

---

The paper lacks a comprehensive review of backdoor methodologies. It primarily relies on the original BadNets, introduced six years prior by Gu et al. (2017). Contemporary and more innovative backdoor attack techniques warrant discussion. Additionally, the trigger mechanism inherent to BadNets is susceptible to detection, as evidenced by [1].

Following the reviewer’s suggestions, we have added a discussion of recent literature on backdoor attacks and defenses in Appendix D1.

To clarify, DIFF2 does not rely on BadNets; rather, it diverges significantly from conventional backdoor attacks (e.g., BadNets). In terms of objectives, BadNets aims to force the classifier to misclassify triggered inputs, while DIFF2 aims to undermine the security assurance provided by the diffusion model for the target classifier. In terms of techniques, BadNets poisons the training data of the classifier to associate the trigger pattern with the adversary’s target class, while DIFF2 superimposes the diffusion model with a malicious diffusion-sampling process, guiding triggered inputs toward an adversarial distribution, while preserving the normal process for clean inputs.

Further, while we opt to use the patch-based trigger as in BadNets and other backdoor attacks (e.g., Chou et al., 2023 and Chen et al., 2023), DIFF2 is not tied to specific triggers. In Appendix C3, we evaluate alternative trigger designs, which also lead to effective attacks.

Additionally, it is important to note that activation clustering (AC) is a training-time defense that is applied to filter poisoning training data. Following Chou et al. (2023) and Chen et al. (2023), we assume the threat model that the adversary acts as the model provider, to which AC is not directly applicable. For completeness, we evaluate ANP (Wu & Wang, 2021), a pruning-based defense in our setting (details in Appendix C6). It is observed that ANP is ineffective against DIFF2 across different learning rates, mainly because the backdoor diffusion function is inherently intertwined with the normal diffusion function in our context. We consider developing more effective defenses against DIFF2 as our ongoing work.

---

In the main experiments, the surrogate and target classifiers are depicted as identical. This assumption appears unrealistic, especially since adversarial examples are tailored to specific classifiers. When pretraining a backdoored diffusion model, it's implausible to assume knowledge of the exact classifier a user will deploy. The experimental setup in Section 4 contrasts with the experiments, indicating a lack of consistency. A deeper exploration of transferability is imperative; the experiments described in Section 5.4 appear insufficient.

To clarify, DIFF2 does not rely on the surrogate and target classifiers being identical to ensure a successful attack. Indeed, as demonstrated in Table 4 of Section 5.4, DIFF2 exhibits a significant degree of transferability to target classifiers with varying architectures and weights, achieving ASR up to 77%. Following the reviewer’s suggestion, we conduct additional transferability evaluation to include a broader range of diffusion models (details in Appendix C7), further showing the effectiveness and practicability of DIFF2.

---

I've observed a possible inconsistency in Table 4. The ASR reported therein is notably low (sub-40%), yet you've mentioned an ASR of 77.1% in the accompanying text. Could you please clarify this discrepancy?

We thank the reviewer for pointing out this discrepancy. For untargeted attacks, the ASR is measured as the percentage of triggered inputs that are misclassified (i.e., one minus the accuracy). In our submission, we mistakenly reported the accuracy as the ASR for untargeted attacks. We have updated the numbers in the revision. Note that for targeted attacks, the reported ASR accurately reflects the percentage of triggered inputs classified into the target class. We appreciate the opportunity to rectify this error.

We thank the reviewer for the valuable feedback on improving this paper! Please find below our response to the reviewer’s questions.

---

DIFF2 could bring a significant negative impact on the benign acc. For example, the result in Table 2 shows that DIFF2 will decrease the clean accurcay from 86.4% to 70.5% (-15.9%). This will largely impact the usuage of the diffusion model. The effect of DIFF2 to robust acc can not be ignored. In addition to Clean ACC, I also notice that the performance of DIFF2 on robust acc is even worse, e.g. the robust acc of SDE decreases from 85.6% to 60.3% (-25.3%). It demonstrates that DIFF2 will increase the vulnerability of the diffusion model.

We thank the reviewer for the insightful comments. We look deeper into the case of SDE/ODE and find that the relatively larger clean/robust accurate drop is mainly caused by the setting of DIFF2's training hyper-parameters. In our initial implementation, we set a unified learning rate for both discrete (DDPM/DDIM) and continuous (SDE/ODE) diffusion models. Yet, it is found that continuous diffusion models tend to require a lower learning rate for DIFF2 to work more effectively. After properly adjusting the learning rate (5e-5), SDE/ODE now attain clean/robust accuracy comparable with other samplers, as reported in the table below.

---

All experiments are performed on the small datasets. Further experiments are needed to illustrate DIFF2 can envade the classifier on the large dataset, e.g. ImageNet.

Our evaluation mainly follows the setting in previous studies (e.g., Chen et al., 2023), where CelebA (64$\times$64), CIFAR-10, and CIFAR-100 are used as benchmark datasets to evaluate attacks against diffusion models. This practice allows us to make meaningful comparisons with prior work in this space. Our results across these datasets consistently demonstrate the efficacy of DIFF2.

However, we acknowledge the importance of testing on larger datasets such as ImageNet for a more comprehensive evaluation. Unfortunately, due to the limitations of our current compute resources (GPU memory constraints in particular), we are not able to directly validate DIFF2 on the full ImageNet dataset. Nonetheless, to partially address this concern, we conduct experiments on ImageNet-64 (a rescaled dataset of 64$\times$64 images across 1,000 classes), with details in Appendix C5. Notably, DIFF2 achieves 93.1% (untargeted) and 17.4% (targeted) ASR, while retaining 39.7% clean ACC. While these results are indicative, we agree that further investigation on larger datasets would be valuable and consider it as our ongoing work.

---

The stealthiness of the attack can be further improved. Figure 3 shows that after performing the purification, the trigger will remains on the generated image . Thus, it increases the probobility of detection by the backdoor detection method, such as [1] or human inspection. Thus, I would suggest use x_a (the adversarial sample of x_c ) to substitute x*_a (Line 6 of Algorithm 1).

We thank the reviewer for the insightful comments. We would like to clarify that DIFF2 is formulated as a backdoor attack, in which the backdoored diffusion model, in tandem with a target classifier, forms an integrated target system. In this context, the purified image serves only as an intermediate representation and is often not subject to in-depth inspection. The setting is consistent with typical backdoor attacks, where triggered inputs drive the target system toward predefined malicious outcomes.

Moreover, we have explored using $x_a$, instead of $x_a^\star$ as the optimization target, but found that this setting leads to much less effective attacks and often causes the optimization to collapse. This may be explained as follows. With $x_a$ as the target, given the stochastic nature of the sampling process and the high similarity between $x_a$ and $x_c$ (differing only by the adversarial noise), the diffusion model tends to produce a mixture of adversarial and clean inputs, which also disrupts the optimization with respect to clean inputs.

We thank the reviewer for the valuable feedback on improving this paper! Please find below our response to the reviewer’s questions.

---

Is the noise on the trigger image after passing through the diffusion and denoising a little too big (3 and 4 row of figure 3) to be easy to seen? Generally speaking, budget L_\infty=8/255 is enough to generate adversarial samples to Cifar10 on ResNet18. Why use 16/255?

We thank the reviewer for the question. For clarification, DIFF2 is formulated as a backdoor attack, in which the backdoored diffusion model, in tandem with a target classifier, forms an integrated system. In this context, the purified input merely serves as an intermediate representation and is typically not subject to in-depth inspection. The setting of our work is consistent with typical backdoor attacks, where triggered inputs drive the system toward predefined malicious outcomes.

Also note that for the diffusion backdoor attack to be effective, the distribution of triggered inputs and the distribution of their adversarial variants need to be sufficiently separable. Otherwise, due to its highly stochastic nature, the sampling process tends to produce a mixture of adversarial and non-adversarial inputs. Thus, while the perturbation budget is often set as $\ell_\infty=8/255$ in the context of adversarial attacks, we find $\ell_\infty=16/255$ leads to more effective diffusion backdoor attacks. The table below shows that DIFF2 still works under $\ell_\infty=8/255$ but less effectively (more details in Appendix C3).

---

By 2 row of figure 3, it is not difficult to find that the trigger is basically retained after adding noise. Is it the main reason why the backdoor can be produced?

We thank the reviewer for the question. Yes, retaining the trigger pattern (partially) after the diffusion process is a critical design, which is essential for distinguishing between a clean input and a triggered input in the latent space. This distinction allows the activation of different sampling processes: one leads back to the original data distribution and the other leads to the adversarial distribution desired by the adversary.

---

About trigger design, I see that the author random selects a 5*5 trigger and adds it in the lower right corner of the image. May I ask whether the author has considered the design method of trigger？For example, considering the invisibility of the trigger, is it possible to select a trigger with L_\infty norm 8/255 (in this case, trigger can be added to the whole picture but not only a corner)? In other words, is there any advantage in selecting the trigger as described in this article?

We thank the reviewer for the insightful comments. While in principle DIFF2 is agnostic to concrete trigger patterns, in our empirical evaluation, we find that compared with other trigger designs, the patch-based trigger tends to lead to more effective attacks: due to its robustness against random noise, it is (partially) retained after the diffusion process, which is critical to ensure the distinguishability of clean and triggered inputs in the latent space. Further, it allows us to easily study the impact of trigger strength (by adjusting the patch size) on the attack performance (Section 5.4).

Following the reviewer’s suggestion, we also explore other trigger designs including the blending-based (Chen et al., 2017) and the warping-based (Nguyen and Tran, 2021) triggers (more details in Appendix C3). While these triggers are also feasible and often less perceptible, they often result in lower attack effectiveness, compared with the patch-based triggers. Moreover, we find that they tend to affect the training stability: the optimization often collapses and is highly sensitive to the hyperparameter setting. Intuitively, these trigger patterns are more susceptible to being obscured by the diffusion process, leading to an entanglement of clean and triggered inputs in the latent space, suggesting that there may exist a natural trade-off between the stealthiness of trigger patterns and the attack effectiveness.

We thank the reviewer for the valuable feedback on improving this paper! Please find below our response to the reviewer’s questions.

---

I'm not concern about the technical part but the practical scenario. I'm uncertain about the practicality of using diffusion models to generate adversarial input.

Due to their unprecedented denoising capabilities, diffusion models have found increasing use as a means to enhance the security of target models (Nie et al., 2022; Yoon et al., 2021; Carlini et al., 2023; Xiao et al., 2023). However, one vulnerability that has been overlooked thus far is the trustworthiness of diffusion models themselves. Due to the expensive training cost, it is a common practice to employ pre-trained, ready-to-use diffusion models (Chou et al., 2023; Chen et al., 2023). This work explores the feasibility of injecting backdoors into such pre-trained diffusion models and subsequently exploiting the backdoors to diminish the security provided by diffusion models for target models. We believe this represents an important attack vector that has yet to be explored.

---

Users may notice unusual patterns in the purified images, as illustrated in Figure 3.

We thank the reviewer for the insightful question. We would like to clarify that DIFF2 is formulated as a backdoor attack, in which the backdoored diffusion model, in tandem with a target classifier, forms an integrated target system. In this context, the purified input merely serves as an intermediate representation and is typically not subject to in-depth inspection. This setting is consistent with typical backdoor attacks, where triggered inputs drive the target system toward predefined malicious outcomes.

Also note that DIFF2 represents a general attack framework that is not tied to specific trigger patterns. We have conducted additional experiments using less perceivable triggers (e.g., blending-based and warping-based triggers), which also achieve effective attacks without unusual patterns in the purified inputs (details in Appendix C3). However, we note that compared with patch-based triggers, the alternative triggers often lead to less effective attacks and less stable training, suggesting that there may exist a natural tradeoff between the stealthiness of trigger patterns and the attack effectiveness.

---

If using the DDIM or other samplers, is the generated adversarial images consistently influence the classifier?

We thank the reviewer for the comments. DIFF2 is designed to be sampler-agnostic. We conduct additional experiments (details in Section 5.2) to evaluate the effectiveness of DIFF2 against the DDIM model, with the other parameters set following Table 7. It is observed that DIFF2 achieves untargeted and targeted ASR over 80.2% and 44.3%, respectively, while retaining the clean ACC at 82.2%.

---

Again, we thank the reviewer for the valuable feedback. Please let us know if there are any other questions or suggestions.

Best,

Authors