Wicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks

We really appreciate your insightful feedback. Please see our response below.

Q1: Numerous studies [1,2,3,4,5,6,7,8] have addressed sample selection in backdoor attacks, several of which [6,8] specifically focus on sample selection for clean-label backdoor attacks. Omitting these key relevant works is a significant oversight and should be addressed to ensure a comprehensive discussion of the literature.

A: Thank you for your comment. Previous works [1,2,3,4,5,7] ([2,3,7] are archived, unpublished papers) discuss sample selection for dirty-label backdoor attacks, which have a different threat model than in our work. As discussed in the paper, we focus on a restricted yet practical threat model where the clean-label attacker only has access to the target class.

Li et al. (2024) (or [6], an also archived, but not yet published paper) also discuss the importance of sample selection for backdoor attacks, but for textual data. Our work investigates the effect of sample selection and proposes a novel strategy to boost the success rate of poisoning vision models, which are intrinsically different from NLP models. Hung-Quang et al. (2024) (or [8], a workshop paper) discuss a similar problem, but its practicality is limited due to its only exploration of pretrained models and the limited number of backdoor attacks and defenses.

Li, Ziqiang, et al. "Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks." arXiv preprint arXiv:2408.11587. 2024.

Hung-Quang, Nguyen, et al. “Clean-Label Backdoor Attacks by Selectively Poisoning with Limited Information from Target Class.” NeurIPS 2023 Workshop on Backdoors in Deep Learning - The Good, the Bad, and the Ugly. 2024.

Q2: The novelty of this paper is limited, as it leverages a pre-trained model to identify "hard samples" for poisoning—a concept already explored in several studies [6,7,9]. However, the distinctions between this approach and prior work are not clearly articulated.

A: Existing works [Gao et al. (2023) (or [9]), He et al. (2023) (or [7])] have studied the threat model where the attacker can access data from all classes and proposed corresponding poison-sample selection algorithms that rely on information from all classes.

On the other hand, our work focuses on the threat model where the attackers can only access the data from the target class. As discussed in Section 3 and acknowledged by the reviewers, this threat model is popular in practice, where the victim struggles to or cannot collect the dataset and needs to rely on the third-party. More importantly, our work demonstrates that even with limited knowledge (knowing data from one class) of the task, the attacker can still launch highly effective attacks using our novel sample-selection method. In other words, there exists a harmful backdoor threat (ours) that requires much less resources and we urge backdoor researchers to develop countermeasures for this type of attack.

As discussed in the previous answer, Li et al. (2024) (or [6]) study the importance of sample selection for backdoor attacks, however, for textual data. Our work investigates the effect of sample selection and proposes a novel strategy to boost the success rate of poisoning vision models, which are intrinsically different from NLP models.

He, Pengfei, et al. "Confidence-driven Sampling for Backdoor Attacks." arXiv preprint arXiv:2310.05263. 2023.

Li, Ziqiang, et al. "Large Language Models are Good Attackers: Efficient and Stealthy Textual Backdoor Attacks." arXiv preprint arXiv:2408.1158. 2024.

Q3: The first contribution claimed by this paper is the introduction of a new backdoor threat model, where an attacker. Consequently, the originality and contribution of this paper raise some concerns.

A: Thank you for the comment. First, we'd like to emphasize the importance, and significant contributions, of our work for the backdoor domain.

We believe that the detailed discussions on "Why are dirty-label attacks more effective than clean-label attacks?" point out the "degree" of harm that each type of attack may cause, assuming the threat model is satisfied. On the other hand, the capability or the tools the attacker can have or use to launch the attack depends greatly on the assumptions of the threat model; for example, if the victim "inspects" each image to filter out the mismatched label (the main motivation behind all clean-label attacks), launching the dirty label attack will become significantly challenging for the attacker; another example is that if the attacker can only collect/annotate data from 1 class (which is in our threat model), launching dirty label attack will also become impossible. The threat model studied in our paper is a practical threat model, but none of the existing attack methods can be employed to launch attacks with sufficiently harmful consequences (as we have already demonstrated in our paper). Consequently, does it mean that this attack setting/threat model is not harmful?

This is the important question we are answering with our paper. Specifically, we extensively demonstrated the existence of a highly stealthy (clean-label) and highly effective (high ASRs) attacker with the discoveries of the proposed sampling strategies. Note that, these sampling strategies are attack agnostic; i.e., an attacker can employ many existing types of attacks (e.g., badnet or sig triggeres), making the capabilities of the attacker even more notable.

More specifically, our paper studies two threat models: (1) the adversary can only access the target class's data and (2) besides the target class's data, the adversary can have access to an OOD dataset. The former threat model is the most restricted case in the class-constrained threat model studied in [10], in which the attacker can only access a single class. This latter threat model (2) is similar to the threat model studied by Narcissus, while the threat model (1) imposes even more constraints on the attack.

On the other hand, previous works [10, 11] suggest augmenting data or optimizing the trigger to increase the success rate in those constrained threat models. In contrast, our work proposes an orthogonal approach, that is to select suitable samples to poison. As shown in Section B.6, our strategy can be combined with other approaches to further boost the performance of the attack.

Q4: The discussion of backdoor attacks and defenses in the related work sections of this paper is outdated.

A: We believe that our paper has compared our work with a wide range of backdoor attacks and defenses, from the most representative to the most recent ones. As backdoor attack is an extensively studied area, it is not possible for us to discuss or include all recent papers. Consequently, we have to focus on the most relevant works to ours in the paper. Nevertheless, we are more than happy to promptly discuss and evaluate any missing and relevant work that the reviewer may recommend during the rebuttal process.

Q5: There are some potential over-claims. For example, Line 156-159: Accessing only samples from a single non-target class is more difficult setting than yours.

A: We'd like to clarify that our threat model is indeed one of the most constrained data poisoning threat models in clean-label as discussed in the paper. The goal of backdoor attacks is to insert a trigger that makes the victim model return the target label when the trigger is presented in the samples. This is not achievable without access to the target class, for example when the attacker only has samples from non-target classes as in the scenario suggested by the reviewer.

Note that, the "constraint" here refers to the "information" the attacker can use to launch the attacks; additionally having access to non-target class means "more information" available to the attacker, while our threat model allows the attacker to have significantly less information (i.e., only from the target class) to launch the attack.

Some related works such as HTBA and Witches's Brew perform clean-label attacks by poisoning data from non-target classes. However, it is worth noting that those methods still require data from the target class to optimize the trigger.

Saha, Aniruddha, et al. "Hidden trigger backdoor attacks." Proceedings of the AAAI conference on artificial intelligence. 2020.

Geiping, Jonas, et al. "Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching." International Conference on Learning Representations. 2021

Q6: The authors should also include the results of methods using all training samples for references, although you have a different setting.

A: While the method in Gao et al. (2023) does not work in our threat model, where the attacker does not have access to data from other classes, we are happy to provide the results of the performance of that strategy, as suggested by the reviewer. More particularly, we train a clean model on CIFAR10 and compute the loss value to select hard samples.

As can be observed, although having access to all training samples, the approach in Gao et al. does not outperform our strategy, which demonstrates the significance of our contributions.

Q7: It would be better to include the results of Narcissus here instead of in the appendix.

A: Thank you for your valuable suggestion. We will update the organization in the camera-ready version accordingly.

Q8: I would like to see whether the proposed method is also effective for untargeted clean-label backdoor attacks (e.g., UBW-C in [12])

A: Thank you for your suggestion. We believe that UBW-C serves a different purpose, which is dataset ownership verification, whereas our study focuses on attacking the model. Consequently, we believe that explicitly providing the comparison with UBW-C may lead to potential confusion among the readers; additionally, we may also have to compare with additional dataset ownership verification methods for fair evaluation, which significantly enlarges the scope of our paper beyond clean-label backdoor attacks.

We, however, believe that studying the applicability of our method in dataset ownership verification is an interesting extension of our work, which deserves an independent study. For example, as the algorithm in UBW-C does not select samples randomly but instead incorporates a selection strategy designed specifically for its optimization process, studying a better selection strategy for UBW-C, based on our work, is an interesting future direction.

Q9: The Resistance to Defenses: The authors should evaluate their methods on more advanced backdoor defenses (such as [13, 14] and their baselines).

A: We'd like to note that in our paper, we have already evaluated our strategy with 10 representative and recent backdoor defenses, ranging from backdoor erasing, training-data and inference-time backdoor detection, to anti-backdoor learning defenses; this makes our evaluation quite comprehensive in the backdoor domain with hundreds of backdoor defenses. Nevertheless, following the suggestion by the reviewer, we report the performance of our strategy against IBD PSC [13], a backdoor detection method. The results show that IBD PSC is not effective against low poisoning rate clean-label attacks, indicated by low AUC and F1 score; and our method does not make the attack less stealthy.

Thank you for your thoughtful comment and positive support. We'd like to further clarify our answers below.

Q1: To the best of my knowledge, there are some works [1, 2] have discussed the setting of accessing samples only from the target class. Thus, although these methods were not designed under the task of sample selection, this threat model is not novel.

A: We agree with the reviewer that our threat models share some resemblance to [1,2], as also discussed in our initial response; nevertheless, the true extent of the danger of these threat models were not fully exposed and in some cases, the attacks in these threat models are not even considered as serious.

Specifically, one of the threat models studied in [1] is the class-constrained threat model, whose most restricted case is similar to our threat model. However, they observe that backdoor attacks in this threat model are impractical due to the low ASR or low stealthiness. In contrast, our work shows that smartly selecting suitable samples to poison significantly boosts the ASRs of several existing attacks (BadNets, Refool, etc...) while keeping the attacks' resilience against backdoor defenses; thus, our work is the first study that exposes the full extent of the vulnerability in this threat model.

On the other hand, the threat model in Narcissus is similar to the second threat model in our work, which assumes that besides the target class's data, the adversary can have access to an OOD dataset. This threat model provides the attacker with more information, thus being more relaxed than the first threat model in our work where the attacker does not have any other data. Again, our work shows that by selecting data smartly for Narcissus, its ASR also improves significantly; in other words, our work again exposes an even more damaging attack based on Narcissus.

In summary, our work has outlined complete scenarios for the threat model where the attacker only has access to the target class data, and has exposed the true danger of these scenarios when the attacker strategically selects the data to poison. We will include this discussion in the camera-ready version of the paper.

Q2: The technique of using OOD samples for optimizing the sample selection is similar to that used for optimizing trigger patterns used in [1, 2] to some extent, although under different settings.

A: We'd like to note that Narcissus [2] proposes a method to find the trigger pattern, whereas our sample selection strategy is an orthogonal approach and can be used in conjunction with different types of triggers, including Narcissus, as shown in Section B.6.

Q3: As far as I know, the author only tested defenses that are 2 years old or even older in their original submission. As such, I think it is unfair to claim that 'We believe that our paper has compared our work with a wide range of backdoor attacks and defenses, from the most representative to the most recent ones.' in the rebuttal.

A: In our paper, we have evaluated our strategy with recent backdoor defenses released last year, such as FT-SAM and RNP. Since our method preserves the utility of the attacks on 11 representative defenses, we can generally conclude that the expected performance of the attacks using our sampling strategy on other types of defense should be preserved. We will include the new results suggested by the reviewer in the camera-ready version of the paper.

Thank you for your invaluable comments and for acknowledging our experiments. Please see our response below.

Q1: The paper trains for 300 epochs which is significantly longer than it should take to train the model on CIFAR-10/GSTRB [1,2] and makes attack success due to over-fitting very likely. Around 100 epochs seems to be more standard. Ideally, to simulate a competent defender early stopping should be employed. I.e. stopping the run when validation loss plateaus. Where are the training settings used in experiments adapted from?

A: We'd like to confirm that, for a fair evaluation, we follow the experimental setup used in the relevant works in the literature, such as Lin, et al. (300 epochs) and Zeng et al. (200 epochs), all of which train the model with more than 100 epochs. Some implementations, such as Wanet, even train the victim model with 1000 epochs. Nevertheless, we follow the reviewer's suggestion and conduct experiments with 100 epochs and report the results below; the experiments show that the number of epochs does not influence the effectiveness of our method.

Lin, Tao, et al. "Don't Use Large Mini-batches, Use Local SGD." International Conference on Learning Representations. 2020.

Zeng, Yi, et al. "Narcissus: A practical clean-label backdoor attack with limited information." Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 2023.

Q2: The experiments use very weak baselines. The paper only evaluates how the method performs compared to random sampling. At minimum the paper should compare against [3]. Especially because [3] could easily be adapted to adhere to this paper's threat model by using a pretrained model. Therefore, the experiments are not sufficent to jusifty that the proposed method is stronger than a slightly adapted version of [3].

A: [3] or Gao et al. (2023), cited and already discussed in our paper, propose three strategies to select samples by examining 1) loss value, 2) forgetting event, and 3) gradient norm, all of which cannot be adapted to launch the attack in our threat model (even when the pretrained model is used). More particularly, computing the forgetting event requires monitoring the training process, while loss value and gradient norm can only be computed when the pretrained model can "return the target class". For example, the attacker cannot use a model pretrained on ImageNet to compute the loss value of images in the GTSRB dataset. Our strategy overcomes this challenge by detecting hard samples based on their features, or training a model on the target class and OOD data to compute the loss value.

To the best of our knowledge, our sampling strategy is the only suitable method for the proposed threat model. Nevertheless, we are more than happy to promptly discuss and evaluate any missing and relevant work that the reviewer may recommend during the rebuttal process.

Nevertheless, as suggested by the reviewer, we now report the performance of [3] when we relax the assumption of our threat model to allow access to other classes. More particularly, we train a clean model on CIFAR10 and compute the loss value to select hard samples. As can be observed, although having access to all training samples, the approach in Gao et al. does not outperform our strategy.

Q3: The paper claims that it's threat model represents "the most constrained data-poisoning threat." However, there are other perfectly reasonable threat models that would make this attack unrealistic. For example, an opportunistic attacker that doesn't get to choose the subset of samples in the dataset they are able to manipulate.

A: Thank you for your comment. When the attacker cannot choose the subset of samples to manipulate, there are 2 cases: (1) the subset they can manipulate does not contain the target class samples, and (2) the subset contains some target class samples and several samples from other classes.

The setting of (1) does not adhere to the conventional clean-label setting, which assumes access to target-class samples for manipulating the images but not changing the target label; consequently, we believe that it's extremely hard (if not impossible) to launch clean label attacks. For the setting of (2), we already provided such experiments in Section B.10, showing that when the attacker only has access to a subset of the target class our method is still effective.

Note that, the "constraint" in backdoor attacks refers to the "information" the attacker can use to launch the attacks; while having less number of target class samples in (2) leads to less information on the target class, additionally having access to non-target class data also means "more information" from the other classes. Nevertheless, our strategy is still effective even if we only choose to use the information from a smaller set of target class data (as in the experiments in B.10).

Q4: When evaluating the attack against defenses the paper does not describe the hyperparameter settings used by each defense nor how those settings were derived.

A: Thank you for the comment. The results for all the defenses reported in our paper are obtained from the experimental settings in the corresponding original papers. We will add this statement in the camera-ready version of the paper.

Q5: Why would an attacker use the OOD strategy proposed in section 4.4, as it requires training a surrogate model and appears to work worse than using a pretrained model?

A: Thank you for your comment. The two data selection approaches cover two complementary scenarios for the attacker: when the pretrained model is available and when the pretrained model is not available (thus, one can be built from an OOD dataset). Choosing which attack approach consequently depends on this availability. When both options are available to the attacker, due to its observed higher performance in our experiments, the pretrained approach is a preferred choice, as mentioned in the comment. The second approach, i.e., OOD-strategy is indispensable when access to the pretrained model is unavailable. Here, the attacker can easily collect an OOD dataset; this attack is slightly less potent but still damaging, even when the OOD dataset is significantly distant as demonstrated in the paper.

Q6: Why use a latent space clustering approach instead of using the loss from a pretrained zero-shot image classifier like CLIP? Why use VICReg instead of a more general feature extractor like CLIP?

A: Thank you for your suggestion. The reason for not using multimodal pretrained models such as CLIP is that those models aim to align image features with textual features, thus, ignoring subtle visual details. For instance, CLIP tries to match an image of an English Springer with the prompt "a photo of a dog", therefore, CLIP is less likely to discriminate different dog species. This characteristic makes CLIP less effective in detecting outliers. To verify that, we conduct experiments where we select samples to poison by CLIP loss or kNN with CLIP features. The results show that this strategy is still better than the random baseline, however, it underperforms different methods where we use vision-only models to extract features. We will include this discussion in the camera-ready version of our paper to make the choice of pretrained models clearer.

Q7: Bolding of best methods or aggregation would make Tables 2 and 3 more interpretable. There are many typos in the manuscript.

A: Thank you for your comment. We will update them in the camera-ready version of the paper.

We are grateful for your insightful comments, and for appreciating our threat model, analysis, and experiments and acknowledging the novelty of our strategies. Please see our response below.

Q1: The Narcissus results, while interesting, are different from what reported in their original paper. Can the authors explain why there are such differences?

A: We'd like to confirm that we used the official implementation of these attacks to poison the datasets. As indicated in Narcissus's paper and in their official repository (https://github.com/reds-lab/Narcissus/issues/3), Narcissus's results have a very high variance; consequently, the results in our paper are averaged over 3 random seeds under our experimental setting using their official implementation. As discussed in Section B.6, we conjecture that the high variance is due to sample selections; for easy samples, the ASR (13.06%) is significantly lower than that (56.16%) of random samples, which is significantly lower than the ASR (89.65%) with samples proposed by our method. This behavior further supports our analysis, showing the importance of selecting samples to poison.

Q2: The OOD approach rely on out-of-distribution data but it’s not clear how this dataset could be obtained, or whether there are any specific requirements of the datasets to maintain the effectiveness of the attacks?

A: As has been shown in the results on GTSRB in Table 3, and especially PubFig and ImageNet-Sketch in Section B.2, our OOD data approach can boost the attack success rate even when the OOD dataset is significantly far from the target dataset, for example, the attacker can employ a general dataset such as TinyImagenet while the victim is training the model on face recognition task. We conjecture that surrogate models pretrained on a general dataset can detect task-agnostic features, such as edges, shapes, colors, etc..., which are sufficient to find outliers in the target class. Therefore, the assumption of access to OOD data can be easily satisfied, showing the practicality of our method.

Q3: Assuming that the victim could distribute the target class data collection to multiple sources, how does the proposed attacks perform in this case?

A: Thank you for your comment. This scenario has already been studied in Section B.10 of our paper. Specifically, Tab. 20 shows that if the attacker only provides 20% of the target class, selecting hard samples in that set still increases the success rate by more than 20%.

Q4: Do the authors have any suggestions about potential mitigation approaches against the proposed attacks in the studied threat model?

A: Thank you for the interesting question. If the victim is aware of this type of attack, one defensive possibility is to utilize the same sample-selection strategy discussed in Section 4 to detect and remove outliers in the dataset. However, the attacker can also counter this adaptive defense by poisoning medium-hard samples; for example, this adversary can sort the training samples from hard to easy and choose those below the top 20%. Figure 2 demonstrates that this strategy is still an effective attack, which performs even better than the baseline.

Currently, due to the important findings of our work, we encourage the model user to source dataset collection/annotation to trusted partners, to reduce the risks of the malicious actors using our methodology to launch a harmful backdoor attack. Otherwise, we believe that developing the countermeasure against our proposed attacks is non-trivial and deserves a separate, independent study as a future extension of our paper.

We really appreciate your insightful feedback. Please see our response below.

Q1: In my opinion, the method primarily introduces a data selection strategy, which lacks sufficient novelty.

A: Thank you for the comment. As discussed in the paper and also discussed by Reviewers BTAB and F14C, the main novelty of the paper (and thus the contributions) includes the discovery of a new and extremely constrained threat model and the proposal of novel sample selection algorithms that can empower existing backdoor attacks to achieve harmful attack success rates. These contributions are significant in the backdoor domain, as they demonstrate the existence of a very stealthy and harmful attacker.

Note that, existing works [Xia et al. (2022), Gao et al. (2023)] also study data selection but for the threat model where the attacker accesses data from all classes. Our work, on the other hand, demonstrates that even with limited knowledge (knowing data from one class) of the task, the attacker can still launch highly effective attacks using our novel sample-selection method. As discussed in Section 3, this threat model is popular in practice, where the victim struggles to or cannot collect the dataset and needs to rely on the third party.

Q2: The evaluation is conducted only on CIFAR-10 and GTSRB datasets, limiting insight into the method's performance across other dataset types and application domains.

A: In the paper, we have evaluated our approach with a wide range of datasets. More specifically, Sec. B.2 shows that even in the scenarios with extreme distribution shifts such as Imagenet-Sketch and PubFig, our method still increases the attack success rate, exposing a serious threat in practice.

Q3: The paper primarily tests against older defense strategies. Implementing more recent and sophisticated defenses, including adaptive methods like sample-specific anomaly detection, would strengthen the evaluation.

A: Thank you for your comment. We'd like to note that in our paper, we have already evaluated our strategy with 10 backdoor defenses, including recent defenses such as FT-SAM and RNP, ranging from backdoor erasing, training-data and inference-time backdoor detection, to anti-backdoor learning defenses. Experimental results show that our strategy is resilient to existing backdoor defenses while boosting the success rate significantly. Table 5 also indicates that current sample detection defenses are not effective against our method.

On the other hand, the user can perform adaptive defense by utilizing the same sample-selection strategy discussed in Section 4 to detect and remove outliers in the dataset. However, the attacker can also counter this adaptive defense by poisoning medium-hard samples; for example, this adversary can sort the training samples from hard to easy and choose those below the top 20%. Figure 2 demonstrates that this strategy is still an effective attack, which performs even better than the baseline. We believe that developing the countermeasure against this type of attack is non-trivial and deserves a separate, independent study as a future extension of our paper.

Q4: The pretrained model strategy relies on the availability of pretrained models in similar domains, which may not always be accessible in real-world applications.

A: As discussed in the paper, when the pretrained models are not available, the attacker can employ an OOD dataset, with similar successes. We also show that, this strategy can still improve the success rate even when the distribution of the training dataset is significantly different from pretrained models (with GTSRB, PubFig, and Imagenet-Sketch). The attacker can employ general pretrained models to detect hard samples in the target class to poison. Therefore, the assumption of pretrained models can be easily satisfied, showing the practicality of our method.

Thank you for your detailed responses. I appreciate the inclusion of more defenses in your evaluation, which provides a broader understanding of your method's performance. However, I strongly suggest conducting further experiments under varied settings, such as different poisoning ratios, architectures, and other factors, to provide a more comprehensive analysis of the attack against those defenses. Regarding the novelty of the attack and the threat model, I remain unconvinced. While I acknowledge your efforts to highlight their significance, I find the arguments presented insufficient to alter my current perspective. Therefore, I will maintain my score.