Learnable Invisible Backdoor for Diffusion Models

Thanks a lot for the suggestions. We have addressed your concerns in the following.

Q: Training details

A: Thanks for the advice. We have incorporated additional training details in our revised manuscript to provide a more comprehensive understanding of our methodology. Please refer to Section 4.1(marked in blue) in the revised manuscript. Furthermore, to promote further research and collaboration, we make our code publicly available on https://github.com/invisibleTriggerDiffusion/invisible_triggers_for_diffusion.

Q: Practical implications of mitigating the risks

A: Thanks for the advice. We aim to delve further into the practical implications of mitigating these risks and have integrated this discussion into our revised manuscript. Please refer to Section 4.4(marked in blue) for details. As mentioned in our manuscript and previous research on backdooring diffusion models [a], a backdoored diffusion model has the potential to generate images containing violent or erotic elements, leading to a highly biased dataset when utilized in practical applications. Moreover, the application of a backdoored diffusion model in downstream tasks, such as reinforcement learning, object detection, semantic segmentation [b, c, d], etc., can have severe consequences.

By effectively mitigating the backdoor vulnerabilities in diffusion models, we can ensure their safe utilization in real-world implementations for safety-critical applications. In addition, diffusion models without backdoors can be confidently employed by individuals for everyday image generation tasks. Therefore, it is of utmost importance to thoroughly study backdoor attacks using invisible triggers and develop effective mitigation strategies for diffusion models.

Q: Convergence concerns

A: Thanks for the feedback. We would like to further discuss the convergence of the optimization problem. We have empirically observed that the training process generally converges quickly, typically within approximately 50 epochs, across various settings, including different norm bounds, training epochs, and poisoning rates. However, we have also noticed that the training process can be slower when a very small norm bound is applied. This can be attributed to the increased difficulty in effectively injecting backdoors when the norm bound is set to a very small value.

To provide more insights and guidance for practical implementation, we have included detailed ablation results on different norm bounds in Appendix H. These results can serve as a reference for selecting appropriate settings in practice.

[a] Chou, Sheng-Yen, Pin-Yu Chen, and Tsung-Yi Ho. "How to backdoor diffusion models?." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[b] Baranchuk, Dmitry, et al. "Label-efficient semantic segmentation with diffusion models." arXiv preprint arXiv:2112.03126 (2021).

[c] Chen, Huayu, et al. "Offline reinforcement learning via high-fidelity generative behavior modeling." arXiv preprint arXiv:2209.14548 (2022).

[d] Chen, Shoufa, et al. "Diffusiondet: Diffusion model for object detection." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

Thanks a lot for the suggestions. We have addressed your concerns in the following.

Q: Threat scenario

A: Thanks for the feedback. We would like to provide further explanation and discussion on the attack setting and threat scenario. Similar with the setting in previous work[a], the attacker only has control over the training process. However, the attacker DOES NOT have the control over the choice of the initial noise which can be any noise randomly sampled from a Gaussian distribution. From Figure 1 and Figure 2, we could see the previous work needs the initial noise to have a designated pattern to make the backdoor activated and this could easily detected by human inspection and some distribution changed based detector, where the attacker will have little chance to activate the trigger. In contrast, our proposed method inserts invisible triggers into the initial noise. Hence the victim user won't be aware of the existence of the trigger, making the attack more stealthy and robust. For instance, in the unconditional case, we present two types of triggers, namely universal triggers and distribution-based triggers, and neither of them requires specific noise to be effective. This problem becomes more crucial in the conditional diffusion setting where the diffusion model is conditioned on various priors. If the priors are severely perturbed with an visible vision, it could be easily spotted.

Q: Missing ablation experiments

A: Sorry for the possible confusion. Because of the space limit, we have conducted detailed ablation studies in Appendix H of the original submitted supplementary material on the choice of different hyperparameters including the norm bound, poisoning rates. We also conduct more experiments on multiple trigger-target pairs, finetuning scheme, different samplers in the Appendix.

Q: Performance against defense

A: Sorry for the confusion. In the original submission, we have conducted experiments on defense methods. Specifically, in Section 4.4, we evaluate the performance of our framework when confronted with the defense method designed specifically for diffusion models as presented in [a]. We also include an additional defense method (ANP [b]) widely used in classification models in the revised manuscript and supplementary material. The results demonstrate that our proposed framework remains effective even in the presence of the defense. For more detailed information, please refer to Section 4.4(marked in blue) and Appendix G.

Q: Value of norm bound C

A: Sorry for the confusion. In Section 4.2, we have specified the norm bound $C$ used by stating "For CIFAR10, $\ell_\infty$ norm bound is set as 0.2 ..." and "We show the results on high-resolution datasets CELEBA-HQ...., where the $\ell_\infty$ norm bound is also set as 0.2 ...". In Section 4.3, we have specified as "By setting the norm bound as 0.04, we show the quantitative results ...".

[a] Chou, Sheng-Yen, Pin-Yu Chen, and Tsung-Yi Ho. "How to backdoor diffusion models?." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[b] Wu, Dongxian, and Yisen Wang. "Adversarial neuron pruning purifies backdoored deep models." Advances in Neural Information Processing Systems 34 (2021): 16913-16925.

Thanks a lot for the valuable suggestions. We have addressed your concerns in the following.

Q: Motivation of invisible trigger for unconditional case

A: Thanks for your valuable comment. We would like to elaborate more on the importance and motivation of our work and have incorporated the discussion in the section Introduction(marked in blue) and Appendix B in the revised manuscript and supplementary material. As both mentioned in [e, f], it is important to improve the fidelity of poisoned examples that are used to inject the backdoor and hence reduce the perceptual detectability by human observers. In the unconditional case, it is thus important to make the sampled noise to be similar with random noise used in the practice or it could be easily filtered by human inspection. As shown in Figure 1 and Figure 2 in the manuscript, the triggers used by previous works (also in the unconditional case) could be easily detected through human inspection without any effort. In contrast, our proposed invisible trigger is nearly visually indistinguishable from the original input, which greatly increase attack's stealth so that human inspection would no longer effective.

Moreover, we want to address our work is the first framework to propose a unified framework for learning invisible triggers to backdoor on not only unconditional but also conditional diffusion models. In addition to unconditional generation, invisible triggers are particularly practical in conditional diffusion models, which hasn't been explored and discussed by the previous works.

Q: Discussion on the benefit of backdooring diffusion models

A: Thanks for the valuable suggestions. We believe the backdoor attack in diffusion model is as important as in the classification model. Since the powerful models like Stable Diffusion is open-sourced, anyone could download the model and conduct malicious fine-tuning to insert a secret backdoor that can exhibit a designated action (e.g. generating a inappropriate or incorrect images). Explicitly, the generated output will be directly controlled by activating backdoor for conducting some bad actions like disseminating propaganda, generating fake contents etc. Meanwhile, implicitly, as also discussed in [a], the diffusion model has been widely in a lot of different downstream tasks and applications such as reinforcement learning, object detection, and semantic segmentation [b, c, d]. Hence if the diffusion model is backdoored, this Trojan effect can bring immeasurable cartographic damage to all downstream tasks and applications.

We have incorporated the discussion in the section Introduction(marked in blue) and Appendix B in the revised manuscript and supplementary material. Please refer to Appendix B for the discussion.

Q: Novelty of the method

A: Thanks for pointing these valuable related works. We provide further discussion regarding the previous works on backdooring classification models through bi-level optimization and have incorporated the discussion into the revised manuscript and supplementary material. Please refer to the section Introduction(marked in blue) and Appendix C. Although previous works [e, f] also utilize a bi-level optimization to achieve a similar objective, we believe that learning an invisible backdoor trigger in the context of diffusion models is totally different and much harder than finding one in the classification model. The method developed for backdooring classification models cannot be directly or easily extended to backdoor diffusion models. Specifically, the threat model is totally different. Diffusion models consist of diffusion and reverse processes that fundamentally differs from classification models. Backdooring diffusion model needs to have careful control of the training procedure while only poisoning data needs to be added in the classification model. At the same time, it is nontrivial and challenging to design the backdoor objective in the conditional and unconditional diffusion model while it is relatively a simple task in the classification. To learn invisible backdoors for both unconditional and conditional diffusion models, the entire pipeline, training paradigm, and training loss have to be redesigned to differ significantly when applying bi-level optimization to backdoor diffusion models. In this setting, the training loss, training paradigm, and pipeline are specifically designed based on the properties of diffusion models differing substantially from backdooring classification models through bi-level optimization.

Thanks a lot for the suggestions. We have addressed your concerns in the following.

Q: Invisibility of the triggers

A: Thanks for your valuable comment. We would like to provide further explanation regarding the invisibility of the triggers. In the literature on attacks, the invisibility can be regulated by the norm bound, which corresponds to the constant C in Equation 8 of our manuscript. A smaller norm bound indicates more invisible triggers. The norm bound is set at 0.06 in Figure 7. In comparison, the results presented in Figure 6 utilize a smaller norm bound (0.04), rendering them more invisible than those in Figure 7. Consequently, one can anticipate that the triggers would exhibit even greater invisibility with a smaller norm bound. We also discussed this in Section 4.3. Moreover, in Appendix H, we conduct a detailed ablation study on different norm bounds for the unconditional case.

The primary contribution of our work lies in proposing a unified framework for injecting invisible triggers into both unconditional and conditional diffusion models. In practice, the invisibility of the injected triggers can be controlled through the norm bound.

Q: Comparison with existing work

A: Thanks for the feedback. In our manuscript, we have discussed previous works that focus on backdooring diffusion models using visible triggers. However, to the best of our knowledge, no existing work has explored backdooring diffusion models with invisible triggers. And the backdoor methods on classification task could not simply extend to generative models because of different threat model, objective, training paradigm.