Towards Reliable and Efficient Backdoor Trigger Inversion via Decoupling Benign Features

Dear Reviewer tkgj, thank you very much for your careful review of our paper and thoughtful comments. We are encouraged by your positive comments on our good paper presentation. We are deeply sorry that our previous submission may lead you to some misunderstandings. We hope the following responses could help clarify misunderstandings and alleviate your concerns.

Q1: The proposed BTI-DBF is almost the same as the backdoor mitigation approach in Neural Cleanse.

R1: Thank you for this comment! We are deeply sorry that our previous submission may lead you to some misunderstandings that we want to clarify here.

• We speculate that you have this misunderstanding mainly because we all have a trigger inversion process before conducting further defenses. However, this is the characteristic of all backdoor trigger inversion (BTI)-based methods. We argue that it is unfair to claim our method is almost the same as neural cleanse simply based on it.
• Our BTI is quite different from neural cleanse in trigger inversion. Neural cleanse used $x' = (1-m)\cdot t + m\cdot x$ to generate backdoor samples. Their $m$ and $t$ are fixed after training. However, our method can generate different triggers via the UNet generator. Moreover, Neural Cleanse only considers the backdoor behavior in pixel space. Our method further considers the backdoor behavior in feature space and finds the difference between the benign features and the backdoor features. Therefore, our method can generate higher-quality triggers. As shown in the Table in our paper, our defenses can succeed even when the triggers are dynamic, but neural cleanse can't.
• Our work considers more defense variants. We apply our BTI in backdoor-removal defenses, pre-processing-based defenses, and detection-based defenses. However, neural cleanse did not consider the pre-processing-based variant. We have made certain breakthroughs in this regard.

Q2: The general idea of first decoupling backdoor features from benign ones and then performing trigger inversion on backdoor features is the same as in [ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation. CCS, 2019.] (though the formulation of the optimization problem is different).

R2: Thank you for this comment! We are deeply sorry that our previous submission may lead you to some misunderstandings that we want to clarify here.

• We never claimed that we were the first to work on backdoor trigger inversion by separating benign and backdoor features. In our paper, we intended to emphasize that all existing methods (including ABS) first fitted backdoor features before decoupling backdoor and benign features. In contrast, our method first fits benign features, avoiding the assumption of backdoor generation.
• Similar to neural cleanse, ABS also assumed that backdoor features can be completely separated from benign ones at the neuron-level. This method ignored the complex interactions between neurons. In contrast, we exploit a soft feature mask whose elements are from $[0, 1]$ instead of $\{0, 1\}$ to learn interaction effects.

Q3: "Revealing the low efficiency and low similarity nature of existing backdoor trigger inversion (BTI) methods and their intrinsic reason" cannot be regarded as a contribution even though you show your method performs better. Besides, there is no adequate discussion about the "intrinsic reason" in this paper.

R3: Thanks for your comments. We explained why exsiting BTI methods can't decouple the backdoor features in our Section 1. We are deeply sorry that we failed to describe it more clearly and lead you to some misunderstandings. We hereby provide more details about it.

• We argue that revealing the common underlying limitations of an important defense paradigm (i.e., backdoor trigger inversion) is also an important contribution. This can alert the field against blind optimism about such methods.
• Intrinsic Reasons: We reveal that both limitations are all because existing methods need to approximate and decouple backdoor features at first to separate benign and backdoor features, as required by BTI. Specifically, these methods need to 'scan' all potential classes to speculate the target label since defenders have no prior knowledge about attacks and poisoned samples. These processes are time-consuming since each scan requires iteratively solving a particular optimization problem.

Dear Reviewer gstB, thank you very much for your careful review of our paper and thoughtful comments. We are encouraged by your positive comments on our novel approach, efficiency, remarkable performance, resistance to adaptive attacks, and good paper presentation. We hope the following responses could alleviate your concerns.

Q1: Besides the discussed adaptive attack that blend features, some attacks, e.g., the composite attack, "Composite Backdoor Attack for Deep Neural Network by Mixing Existing Benign Features" from CCS 2020, also heavily mix benign and malicious features. Similarly, the paper can benefit from evaluating on other baselines, e.g., NONE (NeurIPS'22).

R1: Thank you for these constructive suggestions!

• We argue that composite attack is well-suited as the adaptive attack against our defenses. We reproduce the composite attack method with its offical code under the default setting. As shown in the following Table 1, both BTI-DBF (U) and BTI-DBF $P$ are still effective in defending against the composite attack. It is mostly because we use a soft mask $m \in [0, 1]$, and the benign features can still be decoupled from the backdoor features, even though they are mixed to some extent.

Table 1. The performance (%) of our methods in defending against the composite attack on CIFAR-10.

• Due to the time constraint, we use BadNets with different trigger sizes for discussions. We reproduce NONE based on its official code under the default setting. As shown in Table 2, our method is comparable with NONE under the trigger size in $7\times7$ and $9\times9$. However, we note that our defense is still effective when the trigger size is $15\times15$ while NONE does not work well.

Table 2. The performance (%) of our methods and NONE in defending against BadNets with different trigger size on CIFAR-10.

Please refer to Appendix P.2 of our revision for more details.

Please allow us to thank you again for reviewing our paper and the valuable feedback, and in particular for recognizing the strengths of our paper in terms of novelty, efficiency and effectiveness, resistance to adaptive attacks, and good writing.

Kindly let us know if our response and the new experiments have properly addressed your concerns. We are more than happy to answer any additional questions during the post-rebuttal period. Your feedback will be greatly appreciated.

We would like to thank the reviewer for the helpful discussion during the first round of the review. We hope our response has adequately addressed your comments related to the resistance of our methods to the composite attack and the comparison to NONE baseline. We take this as a great opportunity to improve our work and shall be grateful for any additional feedback you could give to us.

Dear Reviewer SUKT, thank you very much for your careful review of our paper and thoughtful comments. We are encouraged by your positive comments on our novel and different approach, different defense variants, comprehensive evaluation, and good paper presentation. We hope the following responses could help clarify potential misunderstandings and alleviate your concerns.

Q1: An intuition of existing backdoor trigger inversion method is that backdoor feature pattern is relatively fixed and in small size, e.g., a patch or a filter or a generative function. However, the feature space of benign samples can be huge, for example, for the class horse, there could be so many types of benign feature clusters. We are not sure if there is only one cluster in the feature space or there are actually many of them. Thus, the optimization directions can be relatively random. Have you tried different versions of benign features (e.g., different distance measurement)?

R1: Thanks for this insightful question! We are deeply sorry that our submission may lead you to some misunderstandings that we want to clarify.

• We did not assume that backdoor features are relatively fixed and in small size to a large extent. One of the difference between our method and existing methods is that we did not assume that poisoned samples are with the formulation $x' = (1-m) \cdot x + m\cdot t$ (with the regularization term $||m||$).
• In our method, for the backdoor features, we only assume that we can decouple them with benign features. In particular, we assume that all elements in our feature mask are from $[0,1]$ instead of $\{0, 1\}$. Accordingly, our approach can truly decouple benign and backdoor features at the feature level rather than at the neuron level. It alleviate the assumption of previous methods that backdoor features were only a very small group.
• In our trigger inversion step, we did assume that the distance between each poisoned sample and their benign version is small. However, it is even practical even for attacks with visible triggers (e.g., BadNets) since they only change a few pixels.
• However, we fully understand your concerns. To alleviate them, we conduct additional experiments with $L_1$ instead of $L_2$ norm as our distance measurement. As shown in the following table, our methods are still highly effective under the new measurement.

Table 1. The performance (%) of our methods in defending against six attacks when the distance measurements are L1 norm and L2 norm on CIFAR-10

We have provided more details in Appendix O of our revision.

Q2: The adaptive settings consider blending the benign and poisoned samples in the feature space. Have you considered triggers that naturally appear in the training dataset, i.e., natural triggers?

R2: Thank you for this insightful question! We have to admit that we did not think about it. Following similar settings provided in the open-sourced codes of [1], we conduct additional experiments on the MeGlass dataset. As shown in the following table, our methods are still highly effective in defending against this special attack. It is mostly because their backdoor features are still decoupled from benign features, and their triggers are still a small part of the whole image (please find more details in our R1). We will further explore it in our future works. Please refer to Appendix P.1 of our revision for more details.

Reference

1. Emily Wenger, et al. "Finding Naturally Occurring Physical Backdoors in Image Datasets." NeurIPS, 2022.

Table 2. The performance (%) of our methods in defending against natural backdoor attack on the MeGlass dataset.

Please allow us to thank you again for reviewing our paper and the valuable feedback, and in particular for recognizing the strengths of our paper in terms of novelty, comprehensive experments, and good writing.

Kindly let us know if our response and the new experiments have properly addressed your concerns. We are more than happy to answer any additional questions during the post-rebuttal period. Your feedback will be greatly appreciated.

We would like to thank the reviewer for the helpful discussion during the first round of the review. We hope our response has adequately addressed your comments related to our designs for decoupling benign features and the resistance of our methods to the natural backdoor attack. We take this as a great opportunity to improve our work and shall be grateful for any additional feedback you could give to us.

Dear Reviewer SUKT,

We greatly appreciate your initial comments. We totally understand that you may be extremely busy at this time. But we still hope that you could have a quick look at our responses to your concerns. We appreciate any feedback you could give to us. We also hope that you could kindly update the rating if your questions have been addressed. We are also happy to answer any additional questions before the rebuttal ends.

Best Regards,

Paper2275 Authors

Dear Reviewer pvdL, thank you very much for your careful review of our paper and thoughtful comments. We are encouraged by your positive comments on our insightful and novel method, effectiveness, efficiency, comprehensive evaluation, and good paper presentation. We hope the following responses could help clarify potential misunderstandings and alleviate your concerns.

Q1: Table 8 shows that the evaluation only picks 100 classes from ImageNet. This is wired. Has the method been tested on 1000 classes? What is the scalability of the proposed method? How does the method perform compared to other methods when the number of classes increases?

R1: Thank you for the insightful question! We are deeply sorry that our dataset selection may lead you to concerns about our effectiveness.

• Using ImageNet subset is a common setting in existing backdoor-related works (e.g., [1, 2]). For example, Fu et al. [1] picked 20 classes and Huang et al. [2] picked 30 classes from ImageNet. In our work, we deliberately chose a relatively large number of categories (i.e., 100) to evaluate the scalability of all methods.
• However, we do understand your concerns. To further alleviate them, we conduct additional experiments on the ImageNet-1K dataset. Due to the time constraint, we use BadNets and WaNet as attack baselines. Specifically, we set the poison rate to 5% and finetune the pre-trained ResNet-18 on ImageNet-1k to implant backdoors. As shown in Table 1, both BTI-DBF (U) and BTI-DBF $P$ are still effective (BA drops less than 3%, while the ASR is within 5%), even if the number of classes is huge.

We have provided more details in Appendix M of our revision.

Reference

1. Kunzhe Huang, et al. "Backdoor Defense via Decoupling the Training Process." ICLR 2022.
2. Chong Fu, et al. "FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases." USENIX Security 2023.

Table 1. The performance (%) of our methods in defending against BadNets and WaNet on ImageNet-1K.

Q2: Section 2.2 misses some latest work on feature level BTI：(1) SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning, 2023. (2) Detecting Backdoors in Pre-trained Encoders, 2023. Although these 2 works focus on self-supervised learning, they are highly related to the feature level BTI. It would be better to discuss them in Section 2.2.

R2: Thank you for this constructive suggestion! After carefully reading their papers, we have added some discussions in Section 2.2, as follows: Besides, two recent research discussed how to conduct BTI in the feature space under self-supervised learning, inspired by neural cleanse. Besides, we summarize the differences between their works and our method as follows.

• Motivation: These two works focused on the BTI in self-supervised Learning in which input labels are not available. However, our work primarily address the limitations in low efficiency and effectiveness of existing BTI method under supervised learning.
• Method. These two works adopted the same 'blended strategy' as neural cleanse in approximating poisoned samples, although the approximation is in the feature space instead of the input space. In contrast, our method only relies on benign samples without needing to assign a particular poisoning form for approximation.

Q3: No discussion on the limitations.

R3: Thank you for this insightful question! We are deeply sorry that we missed the discussion about potential limitations of our method. We hereby provide it, as follows.

• As illustrated in Section 3.1, our defense mainly focuses on using third-party pre-trained models. In particular, similar to existing baseline methods, we assume that defenders have a few local benign samples. Accordingly, our method is not feasible without benign samples. Besides, we need to train a model for the scenarios using third-party datasets before conducting trigger inversion and follow-up defenses, which is computation- and time-consuming. We will further explore how to conduct BTI under few/zero-shot settings in our future works.
• Our method needs to obtain the feature layer of the backdoored model to decouple the benign features and inverse the backdoor triggers. Besides, the optimization process in our method also relies on a white-box setting. Accordingly, it does not apply to black-box scenarios in which the defenders can only access the final output of the backdoored model. We will continue the exploration of the black-box BTI in our future works.

We have added the disscussion in Appendix N of our revision.