Universal Backdoor Attacks

We thank the reviewer for their positive assessment of our work and valuable feedback. Please find our responses below.

the required number of poisoned samples seems a bit high, even for imagnet. other papers have shown that around 300-500 samples are enough to mount an effective backdoor attack [1, 2]. this is in contrast with the results observed in Table 1, where the baseline attack is not successful even with 2k poisoned samples. can you please look into a setup with fewer poisoned samples? it should be possible to have a successful backdoor attack with close to 500 samples on ImageNet

We thank the reviewer for spotting this detail. It is difficult to compare across papers because the training parameters (learning rate, model architecture, and number of epochs) differ across papers. Our setup is described on p5 in Section 4.1 - Model Training. Geiping et al. poison 0.1% of ImageNet-1K to target a single class, whereas we poison 0.15% on the same dataset to target all classes. They attack a ResNet-34 model, whereas we attack a slightly smaller ResNet-18 model. Carlini et al. propose attacks against multimodal, contrastive models on different datasets with different training objectives. Their targeted backdoor attacks are substantially more effective than any attacks against models trained in a supervised manner. Carlini et al.’s attack requires poisoning only 0.0001% of samples, which is a 100 times reduction compared to Geiping et al.’s attack (and ours). We do not believe these results are comparable since the experimental setup is too different.

We are happy to include this discussion in the revised paper.

the authors only consider a single baseline model against which their attack is compared. this comparison is helpful, however, given the large number of poisoned samples required, it would be nice to see how other baselines would compare at that scale if possible, can you provide a good baseline for attacks to compare against?

We only consider a single baseline we created because we are the first to propose a universal backdoor attack and there are no baselines in the literature. Tables 1, 2, and 3 show the improvement of our work against our created baseline under many different settings, and Figure 3 shows the difference between both approaches during training. We believe that our baselines already provide a comprehensive overview, but we are happy to compare our work to more natural baselines if the reviewer has a suggestion.

the parameters of the defenses were tuned for a simple baseline (BadNets). the effectiveness of the attack might be very different if the parameters of the defense were tuned to the authors' attack can you also tune the parameters of the defense against each attack you are considering?

As stated in the paper, we used the same procedure as Lukas et al. [A] to tune the parameters of a defense. Lukas et al. assume the defender has access to one backdoor attack (BadNets [B]) against which they tune the parameters of their defense. Then, the defender tests the defense against many unknown attacks.

We replicate this setup in the paper, where the defender does not know of the existence of our attack but instead tunes against one known BadNets attack. On further analysis, we have optimized our defense hyperparameters for fine-tuning and pruning defenses over the ranges described in [A] and found no significant improvement in any of the defense’s effectiveness when optimizing its hyperparameters against our attacks. Our approach follows Lukas et al., and our paper shows conditions under which our backdoor lacks robustness (see Figure 5).

We will revise the paper to emphasize these points.

---

[A] Lukas, Nils, and Florian Kerschbaum. "Pick your Poison: Undetectability versus Robustness in Data Poisoning Attacks against Deep Image Classification." arXiv preprint arXiv:2305.09671 (2023).

[B] Gu, Tianyu, et al. "Badnets: Evaluating backdooring attacks on deep neural networks." IEEE Access 7 (2019): 47230-47244.

We thank the reviewer for their valuable suggestions that further improve the paper. Please find our responses to the questions below.

It is not clear why the proposed method improves the inter-class poison transferability and, in particular, how it ensures that an increase in attack success against one class improves attack success against other classes. Does the proposed method increase the transferability (attack success rate) of any two classes, even if these two classes differ significantly in the latent space?

Thank you for asking this clarifying question! On a high level, the main idea of our attack is to associate the trigger with principal components in the (surrogate) classifier’s latent space instead of associating a trigger with a target class. An attacker creates a trigger for each target class by encoding its location in the latent space.

Existing attacks, such as BadNets [A], associate a trigger with a target class by stamping the trigger on an image and labeling it with the target label. This poisoning strategy causes the model to associate the presence of a trigger with the target class. Thus, an effective attack requires injecting at least as many poisoned samples as there are classes. In our attack, we associate triggers with principal components in the dimensionality-reduced latent space of the surrogate classifier, which allows for more sample-efficient attacks. We use Latent Discriminant Analysis to obtain principal components from the latent space. Then, we bit-wise encode these components with the trigger and inject poisoned samples into the training data. During inference, given any target label, the attacker can create a trigger that encodes it as long as they can binary encode it with the principal components.

Upon further analysis, we tested whether our attack succeeded in poisoning all classes and found that a few classes had only a low success rate. Please see the table below for the success rate across 1000 classes from ImageNet.

The table shows the min, max, mean, and median values for the attack success rate against ImageNet-1K using 2000 or 5000 poisoned samples. The min value is 0%, indicating that some classes cannot be successfully attacked. Upon further investigation, we see that the number of principal components in our reduced latent space is too low and that a small number of classes (the ones our attack cannot reach) have the same encoding. Increasing the number of principal components could increase the number of reached classes but may require injecting more samples (as the model needs to learn to associate a trigger with more principal components).

We will include these results in the revised paper and clarify the motivation of inter-class poison transferability.

The formula in Section 3.2 needs to be formulated more appropriately and clearly. Specifically, do y' and y in the formula refer to any two categories or any two similar categories? If they refer to any two categories, please explain why categories that are very different in the latent space can also improve the success rate of the attack; otherwise, if they refer to any two similar categories, please give a clear definition of similarity.

Thank you for your feedback! This definition is too vague and can absolutely be reformulated to encompass the concept of inter-class transferability better. To do this, we refine inter-class transferability over two distinct sets of classes, that is, how improving an attack on one set of classes A indirectly improves the attack on the second set of classes B, which matches our experiments:

$$\frac{1}{|**A**|} \sum\_{a \in **A**} \text{ASR}_{a}\uparrow \implies \frac{1}{|**B**|} \sum\_{b \in B} \text{ASR}\_{b} \uparrow$$

For our experiments in Section 4.4, we choose A and B randomly, where A contains 90% of classes and B contains 10% of classes. We will update that section to use the new definition. What set of classes (A) is best for improving attack success against a certain set of classes B is a very interesting question. We have run an additional experiment that indicates that the choice of B does not strongly influence the magnitude of inter-class transferability. Specifically, whether 90%, 60%, or 30% of classes are used in B does not have a large impact on attack success in A (total poisoning rate held constant).

We thank the reviewer for their valuable suggestions for further improvement of the paper. Please find our detailed responses below.

The related work lacks a specific conceptual description of "many-to-many" and an introduction to recent works in this area.

Thank you for bringing this to our attention. We will revise the paper to include a conceptual description of “many-to-many” attacks. If the reviewer would kindly point us to related work that we might have overlooked, we will be happy to include it in the revised paper.

In Section 3.3, the encoding method used in the latent feature space is rather simplistic, where values greater than the mean are encoded as 1 and others as 0. What is the motivation behind this encoding method, and how does it contribute to improving the transferability of inter-class data poisoning?

We chose a binary encoding method for two reasons. The first reason is to demonstrate the existence of an effective and sample-efficient Universal Backdoor Attack. Our attacks require only a small increase in the number of injected samples. As we state in the paper, one would naïvely expect that such attacks require vastly more poisoned samples. The second reason is to have a robust encoding of the principal components in the model’s latent space. It is possible that improved encoding strategies exist that we did not consider in the paper (e.g., by encoding principal components with floating point numbers), which we leave to future work.

We will revise the paper’s discussion section to address this question.

A) The author employs a patch and blend approach to add triggers, resulting in poor concealment of the backdoor triggers. Visually, the differences between poisoned and clean samples can be distinguished. Has the author considered more covert methods for backdoor implantation, such as injecting triggers in the latent space and decoding them back to the original samples to reduce the dissimilarity between poisoned and clean samples?

B) The authors validated the effectiveness of the method under model-side defense measures. It is recommended to include defense methods in data-side.

Thank you for raising this question! Our attack is agnostic to the trigger, meaning that any trigger can be used if it can encode a multi-bit payload (the encoding of the principal components). We agree that the triggers we use are visually perceptible and could be detected by data-side defenses, but then the attacker could modify their trigger to break this defense, as shown by Koh et al. [B], and we believe this cat-and-mouse game is outside of the scope of our work.

The ability to detect some triggers does not protect the defender from Universal Backdoor Attacks, as the attacker could have used another trigger that the defense does not detect. This comes at a trade-off, as shown by Frederickson et al. [A], where an attacker may need to inject more poisoned samples to achieve the same effectiveness. Unless a data-side defense exists that can detect any trigger, a defender always needs to consider our Universal Backdoor. Hence, the study of data-side defenses is an important problem but orthogonal to the contributions made in our paper.

We refer to the excellent analysis by Koh et al. [B] for further information on this topic and will revise the paper to include this discussion.

Selection of baselines. The chosen comparative methods are both from 2017. It is recommended to include comparative experiments with the latest backdoor attack methods.

We created the baselines that we compare by re-using the trigger from a known attack from Gu et al. [C]. Since we are the first to propose Universal Backdoor Attacks against vision classifiers, there are no other baselines to compare our attack. We will revise the paper to mention the lack of existing baselines in our discussion section.

---

[A] Frederickson, Christopher, et al. "Attack strength vs. detectability dilemma in adversarial machine learning." 2018 international joint conference on neural networks (IJCNN). IEEE, 2018.

[B] Koh, Pang Wei, Jacob Steinhardt, and Percy Liang. "Stronger data poisoning attacks break data sanitization defenses." Machine Learning (2022): 1-47.

[C] Gu, Tianyu, et al. "Badnets: Evaluating backdooring attacks on deep neural networks." IEEE Access 7 (2019): 47230-47244.

We appreciate that the reviewer liked our paper and gave us many valuable suggestions for further improvements. Please find our point-by-point responses below.

In general, each experiment should be averaged over multiple seeds for statistical significance

We agree with the reviewer, and the only reason we did not average over multiple random seeds is the high running time required for training an ImageNet classifier from scratch. Training a single model requires around 100 GPU hours. While we do not repeat experiments for the same parameters with different seeds, our paper contains ablation studies where we repeat the experiment while varying one parameter. To further enhance reproducibility, we promise to release our source code, which can be used to reproduce all experiments shown in the paper.

A major part of the backdoor attack regime is the preservation of clean accuracy, and there is no analysis on how well the proposed method protects a model's clean accuracy. This should certainly be included in future versions of the paper.

We thank the reviewer for raising this question. We have included a table showing the clean accuracy for each model for various poisoning rates and methods below. The clean accuracy of the official HuggingFace ResNet-18 model trained on ImageNet is 69.7%. Since we do not know if Huggingface selected the best model from multiple runs or used a single run, we are re-training a clean model from scratch with our training parameters and will report on its clean accuracy when the training has finished. Please find our table below for the poisoned model’s clean accuracies.

The table shows that all backdoor attacks studied in our paper do not substantially degrade the model’s accuracy.

A) The proposed triggers in Fig. 2 seem quite obvious to the human eye and may be susceptible to input-space defenses. I would like to see some analysis on the necessary intensity of these triggers and their brittleness to input-space defenses like STRIP.

B) In addition to the above, the attack was not evaluated on data-cleaning defenses like SPECTRE, which I think would be particularly effective against this regime. I would like to see these defenses evaluated as well--and not limited to specific target classes.

We agree with the reviewer that there may exist data-cleaning defenses that remove our backdoor when using visible triggers. Our proposed Universal Backdoor Attack is agnostic to the trigger, meaning that the attacker can choose any trigger that can encode multiple bits and use it to poison the model. Consequently, if one specific trigger can be detected reliably, any attack that uses it becomes detectable and ineffective. However, Universal Backdoor Attacks remain a threat because there are other triggers an attacker could have used that are not easily detectable by a defender. This cat-and-mouse game has been studied by Koh et al. [A]. However, there exists a trade-off that an attacker has to consider, which we do not study in our paper: While choosing more stealthy triggers can make it more difficult to detect and remove poisoned samples, this likely comes at the expense of injecting more poisoned samples (with less visible perturbations) to achieve comparable effectiveness (this was studied by Frederickson et al. [B]).

We will expand on this trade-off in the discussion section of our revised paper.

Halting a defense at 2% has the potential to straw-man some defense mechanisms.

We have run additional experiments to measure the trade-off between the clean accuracy (CDA) and the attack success rate (ASR) for a weight decay defense up to 3.5% model accuracy. Please see a table summarizing these results below.

The table shows that the ASR decreases linearly with the CDA, and the Pearson correlation coefficient between the line of best fit to our CDA/ASR data is 0.9794. We used the same 2% cutoff that Lukas et al. [C] use.

We will include a scatter plot of the results in the revised paper.

---

[A] Koh, Pang Wei, Jacob Steinhardt, and Percy Liang. "Stronger data poisoning attacks break data sanitization defenses." Machine Learning (2022): 1-47.

[B] Frederickson, Christopher, et al. "Attack strength vs. detectability dilemma in adversarial machine learning." 2018 international joint conference on neural networks (IJCNN). IEEE, 2018.