Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models

Q1: Typo

A1: Thank you for the correction, we will modify it accordingly in the final version.

Q2: Multi-label settings

A2: Thank you for the comment. We’d like to clarify that the statement “Wenger et al. is limited to multi-label setting” is not intended to underestimate the contributions of the work; we completely agree with the reviewer that the key ideas in Wenger et al. can be extended to the multi-class setting as well. Here, we refer to the fact that backdoor research in the physical domain is stagnant due to the difficulty of manually creating and sharing physical datasets; for example, in order to validate the ideas of Wenger et al. in the multi-class setting, one must create a physical dataset, which is an arduous task as discussed in our paper. Motivated by these challenges, our framework solves this problem, i.e., crafting the datasets semi-automatically with powerful diffusion/generative models, to accelerate this research domain.

Q3: Challenges of using natural objects as backdoor triggers in object detection tasks versus classification tasks

A3: Object detection tasks are composed of 2 subtasks, which are object localization and object classification tasks. To elaborate, classification tasks are a subtask of object detection, and the ultimate goal of a backdoor trigger (natural objects in this case) is to fool/mislead the classifier into predicting a predefined target class. On the other hand, the usage of natural objects as backdoor triggers is relatively more flexible in the case of object detection tasks, which could be to mislead the object localization module, object classification module, or both. Such flexibility provides more opportunities for attackers to exploit, as discussed in Zhang et al, where SPONGE attacks the inference time of the object localization module while BLINDING attacks the accuracy of the object classification module.

We use the classification task as our main task of interest to analyze the framework as we specifically want to limit such flexibility so that our analysis is feasible (e.g., the creation of physical validation dataset is less arduous). Note that, it does not mean our work is less general, as classification is still a broad, well-studied and important task in backdoor research. Extending our framework to other setting could be an interesting future direction of our work.

Q4: Is the work novel because the proposed methods are quite straightforward?

A4: We would like to emphasize that we do not claim that the models used in our framework are our novelty; our novelty, instead, lies in the design of the proposed framework to ease researchers of studying both physical backdoor attacks and defenses, and the rigorous analysis of this framework in this context, similar to the work in Wang et al, 2023 (VSSC), which also use a combination of straightforward generative models to achieve highly effective physical attack that is robust against physical distortions. Consequently, for backdoor research, we believe this is a significant contribution as it will help accelerate the existing physical backdoor research.

Q5: Comparison to Zhang et al.

A5: Zhang et al propose an attack on object detection task where the attacker is assumed to control the training process to find optimal insertion of the (potentially physical) trigger object. Consequently, the work of Zhang et al has a similar objective (i.e., to insert the trigger) as that of our Poison Generation module. In this sense, we believe that the work of Zhang et al. is orthogonal to ours, and the backdoor researcher, who requires the creation of a dataset that induces a better attack success rate, could utilize the proposed algorithm of Zhang et al to better insert the backdoor trigger suggested by our Trigger Suggestion module. Note that, our Poison Generation module also provides a specification for synthesizing samples with the trigger, besides inserting the trigger into the existing samples, when the backdoor researcher only has access to the labels (discussed in Section 4.2), which demonstrates the generality/practicality of our Poison Generation module.

Finally, as discussed in our paper, we aim to enable backdoor researchers to overcome the difficulties (time-consuming and potentially difficult IRB approval effort) encountered in physical backdoor research as reported in previous studies. Specifically, the proposed framework allows backdoor researchers to conduct physical backdoor research completely from the lab and our contributions are the rigorous and extensive analysis of this framework. For these reasons, we believe that our work can help accelerate the stagnant area of physical backdoor research, which is a significant contribution to the backdoor domain.

Reference:

[1] Finding Naturally Occurring Physical Backdoors in Image Datasets

[2] WaNet

Regarding Q2: Since the author has clarified that the approach is similar, I would suggest revising the relevant expressions in the manuscript to better reflect this.

Regarding Q3: What I intended to highlight is that the idea aligns with the approach of diffusion models. At the very least, the author should include these works in the discussion.

Regarding Q4: The author could include a statement in the manuscript emphasizing that their method is simple and effective. Additionally, beyond my concerns about the methodology, I believe the experimental validation is insufficient and needs further enhancement.

Lastly, I noticed the public comments and have flagged the paper for an Ethics Review, requiring further verification by the AC.

Q1: Lower ASR compared to clean label attacks and explanation behind lower ASR

A1: Our work aims to allow backdoor researchers to overcome the difficulties (time-consuming and potentially difficult IRB approval effort) encountered in physical backdoor research as reported in previous studies [1], [4]. Consequently, please note that we do not intend to achieve the best possible attacks or compare them with existing attacks such as LC or Narcissus. Rather, we focus on creating a framework that can replicate the studies in [1], [3], [4] for backdoor researchers, completely from the lab.

Nevertheless, we’d like to provide a potential explanation for your question. The ASR could be affected by 3 factors, size, position, and semantics of triggers (content) as elaborated in [1]. In our case, the size of triggers is generally visible and the position of triggers is generally close to the subject matter. However, there exists a difference between the semantics of the “synthesized” physical triggers and the “real” physical triggers. As shown in [2], generally the transfer learning performance of models trained with synthetic data is lower than models trained with real data, which is mainly due to the limited diversity of the generative models. Additionally, the “synthesized” physical triggers might not be coherent with the “real” physical triggers in terms of appearance, specifically for generic types of triggers. For example, “tennis ball” would be highly probable to be consistent across the synthesized and real images, but “books” could have a high variance of representations within the synthesized images, and pose difficulties in collecting the exact similar representation. Therefore, as depicted in Tab 1-2 (main paper), the ASR of books is generally lower than that of tennis balls as physical triggers. Note that, these are similar observations as in the previous studies of physical triggers [1].

Q2: Evaluation on recent defenses

A2: First, we’d like to remind that our work focuses on creating a framework that can replicate the studies in [1,3,4] for backdoor researchers, completely from the lab. In the paper, we have already evaluated our framework with representative backdoor defenses, including backdoor detection and backdoor mitigation methods. Nevertheless, we followed the reviewer’s suggestion and performed the experiments with IBD-PSC [5] accordingly, with the results presented in the following table.

As shown in the table above, we observe that for “tennis ball”, IBD-PSC is able to effectively detect the physical trigger. We conjecture such observation is due to the “unique” characteristic of a tennis ball, where the object itself is self-explanatory and shares a common perception. These “unique” triggers would share high similarity across the synthesized images and the real images. Such a characteristic would lead to a stronger trigger strength, hence a higher Real ASR as depicted in both Tables 1 and 2 in the main paper. For “book”, which is a generic trigger, and different people might not share a common perception of it, usually would be less likely to be exactly the same across the synthesized and real images. Hence, it acts as a more “generic” trigger, which makes it harder for the model to overfit onto it, inducing lower Real ASR as depicted in both Tables 1 and 2. Since unique triggers possess a stronger trigger strength, it is easier to be detected/mitigated by existing backdoor defenses, whereas generic triggers (book) have lower trigger strength, hence we observe a lower detection rate (F1 and AUC in the above table) compared to the unique trigger (tennis ball).

Nonetheless, backdoor defenses in the physical backdoor research remain challenging due to dataset collection and require extensive I/ERB approvals, our framework allows researchers to study other potential backdoor defense mechanisms in the physical domain, thus accelerating the progression of physical backdoor research. As discussed above, the result of the backdoor defense might vary significantly due to different triggers that are being injected into the dataset; thus it’s inconclusive to claim that a defense is successful against physical backdoor attacks with only one or few physical triggers. To study and prove the efficacy of physical backdoor defenses across these plethora of physical triggers requires an arduous manual effort to collect the dataset, where each dataset should be collected repeatedly with different physical triggers. Our framework is motivated to resolve such difficulties and provide researchers with a toolkit to synthesize surreal physical backdoor dataset, in order to evaluate attack/defense methods extensively and prove their effectiveness.

Thank you for the constructive comments.

Q1: Limited innovative contribution

A1: Our goal is to enable researchers to study physical backdoors research effortlessly, which is currently critical, yet under-explored. Therefore, we innovatively designed this framework, which eases the research journey of physical backdoor researchers. Our framework requires minimal modification to adapt to the state-of-the-art of VQA models, generative models, and evaluation metrics. We note that our novelty lies within the combination of each of these frameworks, and proving such a combination would work in a practical scenario, by collecting and evaluating on a real-world physical backdoor dataset. Such a framework might look straightforward theoretically, but without such extensive proofs (which we have done), its practicality could not be effectively justified. Consequently, our work is especially important in accelerating physical backdoor research.

Q2: Limited control over poison data generation

A2: We thank the reviewer for the constructive suggestion. Our framework has incorporated 2 families of generic generative models, namely image editing (conditioned on both image and text prompts) and image generation models (conditioned on text prompts only). Both of these families cover a plethora of generative models, which our framework could leverage. Currently, our framework enables controllability over the “type” of triggers, which could be suggested by our Trigger Suggestion module or by human annotators. Controllability over the “size” and “position” could be easily achieved through our framework by adding questions to our Trigger Suggestion module such as “Where should the trigger be placed?” and “What should be the trigger size?”, and selecting Image Editing models in our Trigger Generation module to synthesize such an image. For further controllability over the Trigger Generation module, one could even utilize masked editing models (such as Blended Latent Diffusion [1]), which are conditioned on a mask, image, and text prompts, in order to synthesize a trigger accurately at the mask position. Notably, our framework requires minimal changes to achieve such controllability.

It is important to emphasize that our work does not attempt to cover all possible scenarios of creating the triggers but rather focuses on providing a rigorous and extensive analysis of the main framework on variances of the ImageNet dataset. We believe that studying these scenarios deserves several independent works based on our framework, as there are many different scenarios (e.g., sizes, locations, background, lighting, patterns, reflection, wearable items, semantics, etc…), which greatly depend on the domains and applications of the studies.

Q3: More examples

A3: We have included more examples (25 images for each setting) as shown in Appendix H, Figures 17-20.

Reference: [1] Blended Latent Diffusion

Thank you for all the constructive comments and for affirming our contributions.

Q1: Explanation of each module is trained

A1: We clarify that we do not explicitly train each module for backdoor purposes. Within our framework, each of the modules leverages off-the-shelves models/methods. In the Trigger Suggestion Module, the output is indeed a trigger which would be then used as the text prompts for both Text-to-Image Generation and Text-to-Image Editing models, in order to inject/synthesize the trigger onto the images (at inference time). We note that there is no training involved for all three proposed modules.

Q2: Evaluation of the realism of the samples

A2: We leverage ImageReward, which is a metric for measuring human preference against synthesized images to evaluate the realism of the samples. As shown in Tables 1 and 3 of ImageReward [1], the authors have studied the correlation between human preference towards the metric and showed that ImageReward is highly correlated to human preference, thus acting as an appropriate metric to evaluate the realism of the images (since human would naturally prefer “real” images to the “fake” ones). Generally, the higher the ImageReward scores, the more a human would prefer a synthesized image over another, which quantifies the “realism” of the images. We have included some samples of generated images in Figures 2 & 8, and we will include more examples.

Q3: Consistency of the trigger

A3: We thank the reviewer for the constructive and valuable suggestion. First, we’d like to discuss the consistency of the trigger. Trigger objects can be broken down into 2 generic categories: unique and generic triggers. Unique triggers are self-explanatory objects, where no additional adjectives are required to describe such an object, and everyone would have the same perception of the object, given the name (tennis ball in our work); while generic triggers are objects that, if not described with adjectives, different persons would have different imagination on the objects (books in our work, or possibly cars in your example if cars are suitable triggers - recommended by VQA - for some datasets). To prevent an uncontrolled generation from the Trigger Generation module, the backdoor researcher utilizing our framework for the physical backdoor study could either use a unique trigger or add specific descriptions to the generic triggers such that to convert it to a narrower imagination space.

As demonstrated in Tables 1 and 2, the tennis ball trigger (more consistent) yields a higher strength backdoor (i.e., with higher ASR) than the book trigger (less consistent). On the other hand, the trigger does not yield a significant impact on the benign samples, as we do not observe any consistent differences in clean accuracy in these experiments. We believe this is expected as the more consistent the trigger, the easier it is for the model to overfit to the backdoor, resulting in a more successful ASR, but the variation of the object should have a small (or trivial) impact on benign samples unless the triggers are common objects in the benign samples. Nevertheless, due to the flexibility of our framework, if the suggested trigger causes any interference with the benign samples (observed through the clean accuracy) and this is not the intention of the backdoor researcher, we suggest that the researcher should rely on a more consistent trigger object recommended by the Trigger Suggestion module.

We will include this discussion in the final revision.

Reference:

[1] ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

• Ruotong publicly accused us of wrongdoings, but this public comment contained several false claims, including:
  • the plagiarism accusation → despite the fact that we clearly clarify with them all contributions in our paper were original
  • the accusation that Tuan-Anh and Kok-seng violated NeurIPS’ reviewing ethics → despite the fact that we clearly showed them our original developments before NeurIPS’ reviews
  • the false claim that we “admitted” to them that we began our work shortly after seeing their NeurIPS submission → we never made such an admission (and my team didn’t read the reviews)
  • the underestimate of our work → I completely disagreed; this thought was due to the repeated statement that we “replicated” their work, which again was proven “false” with extensive evidence.
  • the wrong accusation that we had to cite their work 15 times, which means we plagiarized → We never copied or plagiarized, as extensively demonstrated with full evidence.
  • the false and baseless accusation that we removed all discussion related to VSSC but “retained the plagiarized sections” → despite our effort to present them with extensive evidence of our originalities in every part of our paper. But seriously, unless we were extremely foolish, why would we keep “our plagiarized sections” (while removing things that helped “prove” us clean)???
• We also had the opportunity to read their VSSC [v3/v4] and surprisingly found that VSSC [v3/v4] now has several similarities to our work, including the argument of labor-intensive/time-consuming process, the explicit separation into 3 modules instead of 2 as in [v2], the use of VQA/VLM to determine the trigger compatibility, among others... Note that, I am not accusing VSSC’s authors of anything (because we know very well how it feels), but I’d like to question that, if VSSC’s authors are allowed to evolve their paper with similarly motivated designs from another paper, why couldn’t we do it too (besides, we had extensive evidence to demonstrate our originalities) -- this is a hypothetical question. I clearly see that our 2 works are targeting very different, equally important problems, and both are based on generative models; thus it is reasonable to use similar tools or designs, and of course, how we use those tools is the novelties of each work.

Everything I wrote in this response is from me, and all the events were prepared by Jason/Chinh/Quang, without any involvement of my co-authors, including Prof. Chan, Eugene, Tuan-Anh, and Kok-seng, as I don’t think it’s necessary for them to be involved (they’re all extremely busy just like I am); we did take almost 2 months to exchange emails with VSSC’s authors. In conclusion, we never violated any ethical standards and our work is original. Perhaps, the coincidence of having 2 people involved in NeurIPS reviewing process of a paper while my team, who mainly contributed to this project, was carrying concurrent work created several misunderstandings and false, unnecessary accusations of our work. Nevertheless, if it comes to shove, I am sure all of my co-authors and I are ready to peacefully collaborate to defend our integrity.

I (Khoa) am disappointed by this factually incorrect message from Ruotong, falsely accusing us of plagiarism, especially after the fact that we have responded to his team, with extensive evidence that our paper naturally evolved independently with a focus on another important but different topic in Backdoor Research. I intend to (1) provide all public readers with all the discussions and supporting evidence we had provided Ruotong’s team earlier to support the originality of our work, then (2), based on these records, expose many “false” claims and accusations in Ruotong’s message.

To avoid burdening the readers with excessive details (if they’re not interested), I will provide 3 different versions of our public response: (A) the Short Summary, (B) the Expanded Version (with major events and evidence), and (C) the Detailed Version (with every timestamped events and communication between our team and his team). I will post only the short version (A) in this comment, while everything will be presented in this 59-page document. I encourage the readers to view version (B) as it contains better but not too overwhelming descriptions of all events.

After reading one of these, or all, versions, we encourage the readers to decide for themselves whether we have conducted plagiarism of their work and whether we owe the rights to all contributions in our ICLR’s version (i.e., whether this submission has any ethical violation). If most readers think we have done so, I WILL PUBLICALLY apologize to Ruotong and his team, and WILL IMMEDIATELY REMOVE every single version of our paper online.

I HAVE ZERO TOLERANCE for plagiarism and any unethical doings in my group. Life is short and research is supposed to be fun and self-fulfilling; plagiarism is definitely NOT any of those.

Note that, as Submission Anonymity is now broken due to the public comment, I will refer directly to several names, some of which are the names of the authors of the paper.

VSSC: The paper “Versatile Backdoor Attack with Visible, Semantic, Sample-Specific, and Compatible Triggers” from Ruotong Wang, Hongrui Chen, Zihao Zhu, Li Liu, Baoyuan Wu, with 4 versions.

I am very disappointed with the team led by such an experienced researcher like Koah. Clearly, Yang needs further education on academic ethics and integrity.

During December 2023 to January 2024, we provided Koah's team ample time to prepare evidence. They failed to demonstrate the independence of their work, and had to withdraw their paper and modify the arXiv version. Surprisingly, they suddenly found more evidence after our public comment.

However, the evidence Koah provided only further confirms their academic misconduct. We must once again emphasize the following facts:

1. Yang's paper published on December 6, 2023, shows striking similarities in content and figures with our paper published on October 8, 2023. This far exceeds coincidence and requires no additional proof.
2. Yang immediately analyzed our paper upon its release and our work was mentioned many times in their discussion. Yang even acknowledged in discussions (Sept. 12, 2023) that InstructDiffusion they planned to use, was similar to Instruct-pix2pix we used in VSSC. However, Yang avoided citing us in their initial version and refused to acknowledge our inspiration in this submission.
3. There is no evidence to support that Tuan-Anh and Kok-seng (AC and reviewer in NeurIPS 2023) did not communicate with Koah’s team, given that they are co-authors of the paper.

Although plagiarism should be determined by the release dates and content of the papers - which is already totally obvious - we are still willing to provide a detailed analysis of the documents provided by Koah:

May 22, 2023: Their documents show that before seeing our paper, they used a completely different method to obtain poisoned images: image generation based on DreamBooth.

June 1, 2023: We released the first version of VSSC (v1).

June 6, 2023: Yang conducted a detailed analysis of VSSC[v1]. This is the point where their methods started to show suspicious similarities to ours.

June 28, 2023: Yang began using the same approach as ours, incorporating triggers into images through image editing method. At this point, based on their analysis of our paper, they manually selected trigger based on object name and appearance.

August 2023: During rebuttal, a reviewer asked about our trigger selection, and we provided a detailed explanation on how we used GPT-4 to generate candidate lists for trigger selection.

September 7, 2023: At this point, they were very concerned about the trigger selection method that we were asked during rebuttal period, but they focused on "Finding compatibility between a class object and a trigger object”, which was still different from ours.

September 11, 2023: Their first try to use our trigger selection method (which we detailed explained to reviewers during NeurIPS rebuttal) by directly querying LLM for suitable triggers. Before this, they used image captions to find co-existing objects. It is hard not to suspect the information leakage from AC and reviewer.

September 12, 2023: At this point, Yang hadn't determined their trigger generation method, but clearly mentioned InstructDiffusion they planned to use(also what they finally used) was similar to the method used in our VSSC: generating triggers using diffusion-based image editing.

They analyzed our low-quality samples and attempted to use the same trigger as ours.

September 29, 2023: Continued discussing on our work. (then "forgot" to cite us in their first version in December). It seems that Yang planned to differentiate their work from VSSC through image selection. However, while they claimed to “discard implausible samples” in Section 4.3 and 5.4 of their submission, they mistakenly drew the regeneration process of VSSC in their flowchart (Figure1). This mistake is perhaps a result of excessive 'reference' to our paper. It’s really an amusing mistake that reveals some facts.

October 8, 2023: We released v2 of VSSC, adding Quality Assessment Module and regenerating low-quality samples. In the paper Yang released on December 6, 2023, they used the same regeneration process as ours, whereas previously they simply discarded low-quality samples.

December 6, 2023: Yang released the first version of their paper, with surprisingly similar flowcharts and content to our v2. This paper came six months after our VSSC[v1] and two months after our [v2]. Considering the fact that they have consistently centered their discussions around our work, it is unbelievable that Koah did not notice these striking similarities. It is hard not to suspect they deliberately avoided citing our work.

All the evidence has been presented in: https://docs.google.com/document/d/1BVJnG_yUgobfcm57033nG3jVC1bQKHTP4spu8o3Pymo/edit?usp=sharing