Towards Reliable Backdoor Attacks on Vision Transformers

Dear reviewer Vd1v:

Thank you for your valuable feedback. We are very sorry for the ambiguity in section 5.2. We have re-organized this section to make it easier to read.

Q1: Thus, the possibility exists that existing ViT-specific backdoor attacks can also evade well-adapted backdoor defenses.

A1: TrojViT [1] and DBIA [2] are the existing two kinds of ViT-specific attacks. In Table 5 and Table 10 in the paper, we have shown that both of them fail to evade well-adapted backdoor defenses: they only achieve a very low ASR (<1%) across all model-agnostic defense approaches while the benign accuracy maintains a relatively high level (>75%). Here we further illuminate that our proposed adapted approaches remain essential for better defending TrojViT and DBIA. The experiments are performed on ImageNet. The ASR and ACC before and after adaptations are shown as follows:

ASR (%)

TrojViT:

DBIA:

ACC (%)

TrojViT:

DBIA:

The results demonstrate that although vanilla settings can successfully decrease the ASR, they also impair the benign function of the backdoored models a lot. In contrast, our proposed adaptations better preserve the benign ACC. For example, AWM achieves the ACC of 77.18% after adaptations against TrojViT while it totally collapses without any adaptations.

Q2: The authors only consider purified-based backdoor defenses. How about the detection-based backdoor defenses? Are they also over-estimated?

A2: Actually, the detection-based defenses are underestimated for current attacks, but our proposed CAT can better bypass those defenses. Take NC [3] as an example, which is one of the most popular detection-based backdoor defenses. It is composed of two stages: It first searches the possible trigger through outlier detection. Then, the backdoor model is mitigated through unlearning with the reversed trigger. Performing experiments on the CIFAR-10 dataset for ViT-B, we explore the performance of NC by dividing the two stages separately:

Stage 1: Detection

NC reconstructs potential triggers for each class and uses an anomaly index to determine if one of them is a valid trigger. The larger the anomaly index, the more likely it is to be a real backdoor trigger. Here, we calculate the anomaly indexes of the attack with or without CAT for comparison:

The results show that CAT can always obtain lower anomaly indexes: for example, the vanilla badnets attack obtains the anomaly indexes of 7.45 which is quite larger than those after combing CAT (5.04). It means CAT can help existing attacks better bypass the detection of NC.

Stage 2: Unlearning

Next, the defenders use the reconstructed triggers to mitigate the backdoor behavior once the reconstructed triggers are identified. Specifically, they fine-tune the model to predict ground-truth labels in the presence of the triggers, i.e., unlearning the backdoor behavior. Here, we explore whether CAT makes existing attacks more resistant to unlearning. According to previous research [1] which observes that the unlearning process of NC with CNNs’ default settings will decrease the benign acc a lot (~50%), we make the following adaptations based on the observations in our paper:

• Use AdamW optimizer to unlearn the backdoored models.
• Unlearn the backdoored model only for 20 epochs.

We summarize the results as follows:

ASR (%)

ACC (%)

We first observe that NC successfully mitigates the effect of all existing attacks and maintains high ACC. Secondly, CAT can help current attacks bypass the unlearning process better: for example, the ASR of Badnets is improved from 1.08% to 99.99%.

Q3: After proposing a new backdoor attack, the authors should compare it with SOTA backdoor attacks, especially advanced ViT-specific backdoor attacks, to show its superiority.

A3: In fact, we have compared CAT with two ViT-specific attacks: TrojViT [1] and DBIA [2] in Table 5 and 10 in the paper. We summarize the average ASR of all attack methods after performing five model-agnostic defenses in the following table.

The results demonstrate that current ViT-specific attacks can not be immune to the adapted defenses and CAT can outperform them with a higher ASR.

Q4: There is insufficient evaluation to explore whether the proposed attack can evade the SOTA backdoor defenses designed for ViT.

A4: In Table 5 and Table 10 in the paper, we have shown that CAT can evade the AB (Attention block) [5] which is one of the ViT-specific backdoor defenses. Here we further perform experiments on patch-processing [6] which is another ViT-specific defense. Following their original paper, we select TPR (the proportion of benign samples that are successfully identified) and TNR (the proportion of backdoor samples that are successfully detected) as our metrics. The experiments are performed on ImageNet with ViT-B architecture.

Compared to the model-agnostic defense in Table 5, patch-processing only obtains limited performance on both Blend and Blend+CAT. We speculate this is because ImageNet is a large and complex dataset. It makes the classification results more susceptible to change if certain patches of the image are masked, thereby increasing the difficulty of distinguishing between clean and backdoor images. In addition, combining with CAT helps Blend obtain lower TNR, representing the backdoor samples are harder to detect. We also perform experiments on CIFAR-10 to further investigate whether CAT can provide better robustness on small datasets. The experiments are also performed on ViT-B:

Similar to the experiments on Imagenet, combining with CAT can obtain comparable TPR and lower TNR (<50%). This demonstrates that CAT can evade the SOTA backdoor defenses designed for ViT.

Q5: There is a lack of enough complex datasets, such as ImageNet, to evaluate the effectiveness of the proposed attacks.

A5: We actually perform experiments on ImageNet in Section 5.2 in the paper. The results show that our proposed attacks are still effective on large datasets.

We really hope our explanation can eliminate your doubts. We are happy for the further discussion with you if any concerns remain.

[1] TrojViT: Trojan Insertion in Vision Transformers, Mengxin Zheng et al., In CVPR 2023.

[2] Dbia: Data-free backdoor injection attack against transformer networks, Peizhuo Lv et al., In Arxiv 2021.

[3] Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks, Bolun Wang et al., In S&P 2019.

[4] BackdoorBench: A Comprehensive Benchmark of Backdoor Learning, Baoyuan Wu et al., In NeurIPS 2022.

[5] Backdoor attacks on vision transformers. Akshayvarun Subramanya et al., In Arxiv 2022.

[6] Defending Backdoor Attacks on Vision Transformer via Patch Processing, Khoa D. Doan et al., in AAAI 2023.

Dear reviewer FbZc:

Thanks very much for your constructive and detailed comments. Here are our responses to address your concerns:

Q1: The hypothesis lacks enough evidence. However, the authors don’t cite papers that use SGD to mitigate backdoors in ViT. When transferring the defense methods on CNN to ViT, the most straightforward scheme is to use the same optimizer as when training the model, i.e., SGD for CNN and AdamW for ViT.

A1: We apologize for the confusion in Section 3.2. In fact, we cited previous research [1], which used the SGD for defensive fine-tuning while Adam for pretraining. To avoid misunderstanding, we revised our paper and discussed previous research in section 3.2 on page 4. Note that the earliest work [2], which first introduced transformers to computer vision, used AdamW for pre-training and SGD for fine-tuning. Following studies [3, 4, 5] naturally inherit this strategy. Unfortunately, in backdoor defense, we found that this natural choice caused issues. We want to use this article to remind everyone to pay attention to making reasonable choices rather than blindly adopting default or natural options.

Q2: The attack methods used in Table 2 are all CNN-specific attack methods. Authors should conduct experiments on ViT-specific backdoor attacks [6,7,8].

A2: Thank you for your suggestion. First, we argue that these backdoor attacks in Table 2 are actually model-agnostic and general rather than CNN-specific. Second, we conduct experiments on ViT-specific attacks to make our conclusion more convincing. Here, we consider the effect of the optimizer on FT for TrojViT [6] and DBIA attacks [7]. We do not include previous research [8] since it proposed to use attention to alleviate the backdoor effect for defense. The results are shown in the following table. The model is DeiT-B, and all experiments are performed on ImageNet. It is easily found that for ViT-specific attacks, AdamW is also a better choice, which better preserves the ACC and reduces the ASR to an extremely low level. This phenomenon is the same as those with model-agnostic attacks in Table 2.

ASR (%)

ACC (%)

Q3: It is not clear if this attack can bypass detection technologies that don’t rely on the difference in activation, such as Neural Cleanse [9] which is based on reverse engineering and outlier detection.

A3: Here, we use Neural Cleanse (NC) as an example. NC is a backdoor consisting of two steps: (1) It first reconstructs all possible triggers through optimization and determines whether a backdoor attack exists via outlier detection. (2) it mitigates backdoor behavior through unlearning with the reconstructed trigger, i.e., restoring the performance even with the presence of the trigger. We examine whether CAT can better bypass NC in these two stages, and all experiments are performed on the CIFAR-10 dataset under ViT-B.

Stage 1: Detection

NC reconstructs potential triggers for each class and uses an anomaly index to determine if one of them is a valid trigger. The larger the anomaly index, the more likely it is to be a real backdoor trigger. Here, we calculate the anomaly indexes of the attack with or without CAT for comparison,

The results show that CAT can always lower anomaly indexes, making the attack stealthier. For example, the vanilla badnets attack obtains anomaly indexes of 7.45, which is larger than those after combing CAT (5.04). It means CAT can help existing attacks better bypass the detection of NC.

Dear reviewer jtUw:

Thank you for your invaluable reviews. For your concerns, our replies are as follows:

Q1: The study of backdoor attacks with only CIFAR10 and single vision transformer architecture is not convincing, and any conclusion based on these limited settings won’t be accurate. Note that study of backdoor attack on the Vision Transformer has been conducted before.

A1: Firstly, we would like to reclaim our contributions compared to previous works [1-4] studying the backdoor robustness for ViTs:

• Backdoor Attack: Zheng et al. in [1] and Lv et al. in [2] propose two ViT-specific attacks respectively: TrojViT and DBIA. However, our works show that both of them fail to evade the backdoor defenses after adaptations. Only our proposed CAT can maintain a high ASR.
• Backdoor Defense: In [3], they first apply model-agnostic backdoor defense on ViTs. However, due to the lack of any adaptation, they either significantly reduce ACC or are unable to reduce ASR. Therefore, they conclude that: the effective defense that has been verified on the CNN architecture may not be suitable for the ViT architecture. In contrast to their conclusions, we find that with proper adaptation, current defense methods are still demonstrated to be effective on ViTs. Subramanya et al. in [4] propose a ViT-specific defense: AB (Attention Blocking). We show that CAT can improve the ASR even for ViT-specific defense like AB.

Furthermore, to illustrate that our conclusion is general and convincing, we conduct experiments on the CIFAR-100 dataset with the ViT-B and Swin-B architectures as follows:

Comparison of Optimizers

We perform FT using SGD and AdamW optimizer, respectively in the following tables:

ViT-B

Swin-B

While the SGD optimizer can reduce the ASR to almost zero, it will significantly sacrifice the benign ACC, which makes the purified models unusable. By contrast, the ACC for fine-tuning using AdamW (69.12%) is much higher than the ACC for fine-tuning using SGD (42.76%). This observation is consistent with our findings on the CIFAR-10 dataset.

Comparison of Fine-tuning Epochs

On CIFAR-100, we also fine-tuned ViT-B and Swin-B with various epochs to investigate the effect of longer training on ViTs’ performance. The benign accuracy is shown in the following table,

where we find the severe overfitting issue caused by longer training. For example, the ACC drop of 100 epochs with ViT-B is more than 10% (69.12 → 58.93). This is consistent with the phenomenon in our paper.

Comparison of Puning Granularity

Taking AWM as an example, we investigate the impact of pruning granularity on the performance of defense approaches. Here, we compare the default granularity and our adapted granularity on CIFAR-100 with both ViT-B and Swin-B in the following tables.

ViT-B

Swin-B

Similarly, our proposed coarse granularity (pruning the output channel of the linear projection layer) surpasses the default granularity (pruning ViTs using an element-wise mask) in preserving ACC. For example, our proposed adaptation improves ACC of ViT-B by a notable margin (61.04% → 82.07%) while keeping the ASR almost unchanged. The results demonstrate that pruning the output channel is a better choice across datasets or architectures.

Q4: Some symbols used in the formulas lack explanations. Appendix C Figure 7 should refer to Table 7.

A4: Thank you for the correction. We have read the entire paper word by word and two minor errors have been corrected (The corrections are highlighted in blue.):

• In Appendix C, we substitute “Figure 7” with “Table 7”. Figure 7 is correctly referred to and explained in Appendix G.
• In Section 4, we correct “there exists” with “there exist”.

We sincerely hope that our response could answer your questions and you could reconsider your rating. Look forward to your replies!

[1] An Image is Worth 16$\times$16 Words: Transformer for Image Recognition at Scale, Dosovitskiy et al, in ICLR 2021.

[2] BackdoorBench: A Comprehensive Benchmark of Backdoor Learning, Baoyuan Wu et al., In NeurIPS 2022.

Dear reviewer FJ73:

First, I would like to express my gratitude to you for your insightful feedback. For your raised questions, here are our responses:

Q1: The contributions of this paper seem incremental, especially in the defense part. There is no methodology to guide us on how to pick good hyperparameters.

A1: We believe that when solving problems, the greatest contribution to the community is to explore the fundamental issues and solve them with straightforward strategies, rather than a seemingly novel and complex approach. Here, we attribute the failure of the current defense to the following two points:

• The mismatch between training and defense: In fact, the first paper proposing ViTs [1] uses AdamW for pretraining and finetuning with SGD. Therefore, multiple works such as [2] adopt this strategy as default and they only obtain 42.00% ACC. We first identify this problem and propose that the AdamW optimizer is essential for fine-tuning-based defense.
• The overfitting problem during the defense: There is a typical problem when only a small amount of data is available for defense. In this situation, fine-tuning a large-capacity model is risky as it can easily lead to overfitting, especially for ViTs which are known to lack inductive bias [1]. Therefore, applying coarser granularity to reduce the number of learnable parameters and decreasing the number of iterations to lower the actual exploration parameter space can both reduce overfitting issues.

Therefore, we believe that our approaches provide the community with sufficient insight and contributions. From our investigations, there are three methodologies that can be concluded to guide us in picking good hyperparameters.

• If the defense methods are based on fine-tuning, the AdamW optimizer is recommended.
• If the defense methods are based on pruning, pruning the channel of the linear layer might be a good choice.
• For all defense methods, additional measures are needed to be taken to prevent overfitting.

Q2: The curve for the first 20 epochs differs significantly from the first 20 epochs when setting the experiment to 100 epochs, particularly in the left plot of (a) left. The variability in experimental results raises concerns about the reliability of the findings, considering the potential instability.

A2: This is because we employ a cosine learning rate (lr) schedule, which starts with a very large learning rate (0.0003) and gradually decreases until it reaches 0 in the final epoch. lr decreases at a different pace between training for 20 epochs and training for 100 epochs. Specifically, in the former case, lr drops to 0 at the 20th epoch (20 epochs in total), whereas in the latter case, it still keeps a large value (0.000274) at the 20th epoch (100 epochs in total). As a result, the curve seems quite different for the first 20 epochs.

In addition, we repeated our experiments 3 times with various numbers of epochs in the following table, in which we can find our method is stable with small variability and the findings are reliable.

Badnets Attack

Q3: The enhancement of CAT is limited.

A3: Note that even a single successful attack can lead to substantial losses. When ASR increases from 1% to 10%, the incident frequency becomes 10 times larger, turning a negligible risk into a significant threat. In contrast, models with ACC of either 1% or 10% are both considered unusable in practice.

In the following table, we have listed the average improvement of CAT against various defense algorithms under a given attack method, along with the changes in the incident frequency. It is evident that our method significantly increases the incident frequency. Notably, it even increases the incident frequency of BadNets to ~34 times larger compared to its original value.