Confidence-driven Sampling for Backdoor Attacks

We thank the reviewer for raising these concerns. We will address these concerns by discussing the following questions: (1) evaluation on larger datasets such as ImageNet; (2) evaluation on defenses that do not depend on outlier detection; (3) clarification for FUS.

Q1: Evaluation on larger datasets such as ImageNet.

We thank the reviewer for this suggestion. We conduct experiments on Tiny-ImageNet. Success rates of random sampling and CBS combined with 3 types of attacks and against various defenses are shown in the following tables. These results demonstrate that CBS is applicable to larger datasets and consistently improves the stealthiness of different attacks.

Performance on Type I backdoor attacks:

Performance on Type II backdoor attacks:

Performance on Type III backdoor attacks:

Q2: Evaluatuion on defenses that do not depend on outlier detection.

We would like to politely point out that we evaluate defenses that do not depend on outlier detections such as Neural Cleanse[3], Fine Pruning[4], Anti-Backdoor Learning[5], in this work. We present partial results in the main text Table 1-3 and full results in the Appendix Table 4-7. According to these results, our method can also effectively improve the attacks' resistance against non-outlier-detection defenses. We are grateful for the defense in [1] pointed out by the reviewer. We notice that the authors published an updated version in [2] so we follow the latest paper. In detail, we test on CIFAR-10 with model ResNet18, and evaluate the performance of attacks BadNet and Clean-label attack. We compare our method with random sampling. We present both success rate (ASR) and clean accuracy (Clean Acc) in the following Table. We include another defense Neural Cleanse for a convenient comparison. The results show that under this powerful defense, our CBS can still significantly improve the stealthiness of backbone methods.

Q3: Clarification for FUS

As mentioned in the Introduction of the original paper, FUS [2] is a method to improve the efficiency of poisoning by a rational selection of poisoned samples. We follow the strategy in Sections 3.3, 4.1 and 4.2 in the original paper where they compare FUS with random sampling for fixed poison rates (which is referred to as mixing ratio in the original paper). In this case, we can compare it with random sampling and CBS for fixed poisoned rates.

Additional comments: We follow the reviewer's suggestion and conduct experiments on Tiny-ImageNet.

We hope our response can address your concerns. We are grateful for your valuable reviews and look forward to further feedback.

References

[1] Provable Guarantees against Data Poisoning Using Self-Expansion and Compatibility

[2] Data-Efficient Backdoor Attacks

[3] Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

[4] Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

[5] Anti-Backdoor Learning: Training Clean Models on Poisoned Data

We are grateful for your valuable comments. We hope that our responses have addressed your concerns. If you have any further concerns, please let us know. We are looking forward to hearing from you.

Q3: Discuss the defenses mentioned by the reviewer.

We are grateful for the defenses highlighted by the reviewer. To begin, we would like to clarify that our evaluation includes not only outlier-based defenses but also non-outlier-based ones such as NC, ABL, and FP. While we present a portion of these results in the main paper (Tables 1-3), comprehensive details can be found in the Appendix (Tables 4-7). According to these results, our method can improve the attacks’ resistance against both types of defenses, and we do not observe a drop in performance against non-outlier-detection defenses.

Regarding the defense method mentioned in [5], we encountered significant challenges in implementing it due to its complexity. The method requires training an extensive number of models (approximately 100,000) on randomly selected 50% subsets of the poisoned dataset. This process is exceedingly time-intensive. Thus, we focus our experimental efforts on the defense proposed in [6]. In detail, we test on CIFAR-10 with model ResNet18, and evaluate the performance of attacks BadNet and Clean-label attack. We compare our method with random sampling. We present both success rate (ASR) and clean accuracy (Clean Acc) in the following Table. We include another defense Neural Cleanse for a convenient comparison. The results show that under this powerful defense, our CBS can still significantly improve the stealthiness of backbone methods.

Q4: Why does the performance drop in the absence of defenses in Table 1?

Intuitively, our method selects samples close to the boundary, and this targeted selection results in a more constrained selection space compared to random sampling, which may lead to a slightly reduced performance in the absence of defenses. For instance, when high-confidence samples for their true classes are used as poisoning samples, there is a significant shift in the decision boundary, typically leading to a higher success rate. Moreover, in Section 4.3, we theoretically illustrate that boundary samples (with lower confidence) have a smaller success rate than samples that are further from the boundary (with higher confidence), which indicates that our methods slightly compromise the performance without defenses. Our analysis and experimental results show that there exists a trade-off between clean performance and stealthiness, and our method is stronger against defenses.

We hope our responses can address the reviewer's concerns. We are grateful for the insightful reviews and look forward to further interactions.

References

[1] Backdoor Learning: A Survey

[2] Hidden Trigger Backdoor Attacks

[3] Label-Consistent Backdoor Attacks

[4] Backdoor Attack with Imperceptible Input and Latent Modification

[5] Rethinking Backdoor Attacks

[6] Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks

[7] Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch

We thank the reviewer for raising these concerns. We address these concerns by discussing the following questions: (1) Does the attacker have "edit access" to the entire training set? Is this assumption practical? (2) Does the attacker have access to the surrogate model? Is this scenario practical? (3) discuss the defenses mentioned by the reviewer; (4) Why does the performance drop in the absence of defenses in Table 1?

Q1: Does the attacker have "edit access" to the entire training set? Is this assumption practical?

We acknowledge that there exist some practical scenarios when the attacker has edit access to the entire training set. For example, the attacker can be a model provider or a training platform provider as described in scenarios 1 and 2 in [1]. Under these scenarios, the attacker have the capacity to modify any data in the training set as he wishes. Therefore, our method can be easily adapted for selection to improve the stealthiness of the backdoor.

Moreover, our work does not necessarily assume the attacker has full edit access to the training set. When only allowed to insert triggers to part of the training data, the attacker can still select samples from his own data via our proposed method while keeping other users' data clean if the attacker is concerned about the stealthiness of the backdoor. To validate the feasibility of our proposed method under this scenario, we conducted an additional experiment on CIFAR-10. We allocated 10% of the training set as the attacker's private data, and the attacker is not allowed to perturb the rest 90%. We assume that the attacker selects 10% from his own data to insert triggers. To apply our method, we train a ResNet18 as the surrogate model and select samples from the private data. We evaluated the poisoning effect on both ResNet18 and VGG16. We considered the BadNet attack and defenses such as Spectral Signature(SS), Strip, and Neural Cleanse(NC). Our results (success rates), detailed in the following Table, suggest that our approach can indeed enhance the stealth of backbone attacks, even when the full training set is not accessible.

Q2: Does the attacker have access to the surrogate model? Is this scenario practical?

First, we would like to clarify that there exist some practical scenarios where surrogate models are available. When the attacker is a model provider or a training platform provider, he has access to the model information and thus can easily utilize a surrogate model for selection via our proposed method.

Second, we would like to highlight that it is a common assumption to have a surrogate model. The previous works in [2][3][4] also consider poisoning attacks in the same setting to leverage a surrogate model for generating poisoning samples. Notably, the architecture or parameters of the surrogate model do not need to be the same as the victim model. In our work, in Table 1 and 2, we present experiments where poisons are generated from a ResNet18 model, and we test the poisoning performance on the victim model VGG16. These results show that poisons can be transferred to different models. Existing work such as [7] also provides empirical evidence for the effectiveness of surrogate models under the black-box setting. In Tabel 4 in [7], poisons are generated from the surrogate model ResNet18 and transferred to MonileNet-V2, VGG11. The poisons are still effective. Therefore, we think the usage of surrogate models is practical.

We are grateful for your valuable comments. We hope that our responses have addressed your concerns. If you have any further concerns, please let us know. We are looking forward to hearing from you.

We appreciate your reviews. We hope that our responses have adequately addressed your concerns. As the deadline for open discussion nears, we kindly remind you to share any additional feedback you may have. We are keen to engage in further discussion.

Q3. Does the proposed method perform better against defenses depending on outlier detection than other backdoor defense methods?

In this work, we consider various defenses including outlier-detection methods like Spectral Signature[4], Activation Clustering[5], SPECTRE[6], and other methods like Neural Cleanse[7], Fine Pruning[8], and Anti-Backdoor Learning[9]. We present part of the results in the main paper Table 1-3 and full results in Appendix Table 4-7. According to these results, our method can improve the attacks' resistance against both types of defenses, and we do not observe a drop in performance against non-outlier-detection defenses. For instance, in Table 1, BadNet+CBS can achieve a success rate of 24.6% against NC which is not an outlier-detection defense, while achieving 23.7% against STRIP which is an outlier-detection defense.

Q4: Are there same observations for clean-label attacks as for dirty-label attacks in Figure 1?

Yes, we have similar observations for clean-label attacks such as Clean-label attack in [3]. We will include visualizations for Clean-label attacks in the appendix. Existing work [9] provides additional evidence. As shown in the introduction and Figure 1 in [10], poison and clean samples consistently form two separate clusters in the latent space, even for the Clean-label attack[3].

Q5:Why the performance drop when no defenses?

Intuitively, our method selects samples close to the boundary, and this targeted selection results in a more constrained selection space compared to random sampling, which may lead to a slightly reduced performance in the absence of defenses. For instance, when high-confidence samples for their true classes are used as poisoning samples, there is a significant shift in the decision boundary, typically leading to a higher success rate. Moreover, in Section 4.3, we theoretically illustrate that boundary samples (with lower confidence) have a smaller success rate than samples that are further from the boundary (with higher confidence), which indicates that our methods slightly compromise the performance without defenses. Our analysis and experimental results show that there exists a trade-off between clean performance and stealthiness, and our method is stronger against defenses.

We hope our responses can address the reviewer's concerns. We are grateful for the insightful reviews and look forward to further interactions.

References

[1] Adversarial Unlearning of Backdoors via Implicit Hypergradient

[2] Hidden Trigger Backdoor Attacks

[3] Label-consistent backdoor attacks.

[4] Spectral Signatures in Backdoor Attacks

[5] Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

[6] SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics

[7] Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks

[8] Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

[9] Anti-Backdoor Learning: Training Clean Models on Poisoned Data

[10] Revisiting the Assumption of Latent Separability for Backdoor Defenses

We thank the reviewer for raising these concerns. To address these concerns, we will discuss the following questions: (1) Is the proposed method inefficient, especially for training surrogate models on large datasets; (2) Evaluations on some recent backdoor attacks e.g. i-bau; (3) Does the proposed method perform better against defenses depending on outlier detection than other backdoor defense methods; (4) Are there same observations for clean-label attacks as for dirty-label attacks in Figure 1; and (5) Why the performance drop when no defenses.

Q1: Is the proposed inefficient, especially for training surrogate models on large datasets?

We would like to clarify that the proposed method does not suffer much from the time efficiency problem. First, we do not need to train the surrogate model till the end, because shorter training epochs are sufficient for selection and have the added benefit of reducing the risk of the model overfitting to the training set which may reduce the poisoning effect. As mentioned in Section 5.1, we only train the surrogate model for 60 epochs. We provide an ablation study about training epochs for the surrogate model in the CIFAR-10 dataset and ResNet18 model in Table below.

According to this table, when the surrogate model is trained for few epochs(20), we can not have a good estimation for boundary thus the performance is much decreased; when we train for more epochs(80), the performance is similar to 60 epochs. Second, many existing attacking methods such as hidden-trigger[2] and clean-label[3], utilize a surrogate model. When applying the proposed method to these attacks, we can directly leverage surrogate models of these attacks for selection, thus avoiding inefficiency. Moreover, for some large datasets such as ImageNet, there exist pre-trained models that can be directly used by the proposed method. We conduct experiments on Tiny-ImageNet where the surrogate model is trained for 60 epochs within 2 hours. The results are shown in Q1 in the response for Reviewer ycqn, and our method can consistently improve the attack's resistance against various defenses.

Q2: Evaluations on some recent backdoor attacks e.g. i-bau.

We thank the reviewer for suggesting the defense, I-BAU[1]. We compare our method (CBS) and random sampling (Random) on the CIFAR-10 dataset and ResNet18 model. We test various attacks including BadNet, Blend, WaNet and Hidden-trigger. We present both success rate(ASR) and clean accuracy(Clean Acc, accuracy on samples without trigger) in the following Table.

We include another defense Neural Cleanse[7] for convenient comparison. We notice that I-BAU is much more powerful than other defenses such as Neural Cleanse[6] but compromises the clean performance. However, according to the results, our method can still improve the stealthiness of the backbone methods under this powerful defense.

We are grateful for your valuable comments. We hope that our responses have addressed your concerns. If you have any further concerns, please let us know. We are looking forward to hearing from you.