A Closer Look at Backdoor Attacks on CLIP

Thank you so much for your valuable comments!

W1: The authors classify the backdoor triggers into two categories, namely local-patch and global-noise, and then, explore their effects against CLIP. Is such a trigger pattern classification generic enough to validate the effectiveness of the revealed findings? To make the finding more persuasive, the authors should better give a more comprehensive discussion on the reasonability of the selection of attack types, or provide experiments to prove it.

A: Thanks for your helpful suggestion. We argue that the taxonomy of triggers is general enough to current backdoor attacks on CLIP. We really understand your concern about other types of triggers. To address your concern, we have tried our best to implement two more backdoor attacks on CLIP called BadNet-M (multiple triggers located in the four corners of the image) and Kitty (the trigger is a natural object of the kitty). The experimental results are shown in the following tables.

Table: BadNet-M AH (ASR)

Table: BadNet-M (ASR)

Table: kitty AH (ASR)

Table: kitty MLP (ASR)

The above tables show that the two backdoor attacks mainly affect attention heads in the last layer and have little effect on MLPs. Therefore, they could be classified as the local patch-based type.

W2: The proposed defense works in two pipelines (against two different attack types), and it looks like implementing these two pipelines simultaneously is time-consuming. So, how does the defender know which defense pipeline should be selected in practice? Seem to be impossible.

A: Indeed, a defender does not know the types of triggers in practice. In this case, we argue that we can defend against both two types of backdoor attacks by first checking whether AHs or MLPs are infected and repairing infected AHs or MLPs correspondingly. We would like to emphasize that this strategy would not expend too much additional inference cost because we just need to forward images into the network once and repair the representation on AHs or MLPs for different types of backdoor images simultaneously.

W3: A confusing conflict: the 2nd finding says the local patch backdoor infects AH, while the last finding claims that AHs are not greatly infected. So, are the AHs infected at all? If yes, the last finding seems to be meaningless. If not, what about the correctness of the 2nd finding?

A: We would like to explain that the second and the last findings are not in conflict. Specifically, the second finding mainly reveals that many attention heads have infected in terms of their attention weights (i.e., they pay more attention to the trigger) and MMD scores (significant distribution differences compared with clean attention heads), while the last finding further reveals that some of these infected attention heads still maintain the original functionality which is so-called "not greatly infected" (Noe that different attention heads have various roles (e.g., color and location) in models [1].). For example, as shown in Figure 2, the 4th infected attention head still maintains the role of color because its descriptive texts are both related to color, while descriptive texts of other attention heads have significantly changed. In summary, we argue that these two findings have revealed the different characteristics of infected attention heads and are not in conflict.

W4: By the way, what is the meaning of the last finding? This manuscript seems to have no relation or motivation to derive the final defense method.

A: We would like to explain that the motivation for the last finding stems from the research of understanding CLIP's image representations by text explanations [1, 2]. Specifically, they revealed how the internal image token acts in the model by using natural texts, which are more intuitive and comprehensible for humans to understand than only using visual attention maps. They found that some attention heads have property-specific roles (e.g. location or shape), indicating their inherent functionality in the model [1]. Based on this observation, we would like to clarify whether or how backdoor attacks have affected the functionality of infected attention heads. If this is the case, the infected attention heads may be destroyed completely. Otherwise, the infected attention heads may be not greatly affected and still maintain the original functionality. We believe that exploring this point could motivate more future work to design attack or defense methods for this concern.

W5: I really appreciate the efforts of the authors to do these experiments to reveal their findings. However, there is no novel approach involved in the study procedure but only invocations of different previous tools. Moreover, the final findings are similar to an early work, i.e., the well-known ''fine-pruning" defenses.

A: Thanks for affirming our efforts. We indeed use the existing method of representation decomposition [1] to explore the characteristics of backdoor attacks on CLIP. Despite this, we would like to emphasize that our findings are indeed novel and significantly important in the area of backdoor attacks and defenses. Besides, we would like to summarize the basic differences between our work and the mentioned paper [3] in terms of the following aspects.

1. Different target models. Our work focused on prevalent attention-based multimodal models (e.g., CLIP), while the paper [3] focused on simple non-attention-based networks (e.g., AlexNet). The findings and the defense method in the paper [3] do not apply to CLIP directly. Exploring the effect of backdoor attacks on CLIP has a broader impact on the safety concern of vision-language models.

2. Different empirical findings. Although the paper [3] also revealed the characteristics of the infected neurons in the backdoored model, our findings revealed that different types of backdoor attacks would affect different model components in CLIP, which has not been disclosed by the paper [3] or any other existing papers. Furthermore, we used CLIP's texts to explore the functional change of infected model components. The resulting finding has also not been revealed yet in the existing research.

3. Different defense methods. Specifically, Fine-pruning [3] first prunes the backdoored network and then fine-tunes the pruned network, while our proposed method i.e., Decomp-Rep, aims to decompose and modify the representation, which is training-free and more explainable.

In summary, we argue that our findings are indeed novel and significantly important to the area of backdoor attacks and defenses. Besides, the proposed defense methods are also novel and well-motivated.

Thank you so much for your insightful comments!

W1: I think there are some issues with this paper, particularly concerning the generality of certain parameters in the CLIP domain and its performance in addressing different types of backdoor attacks.

A: Thanks for your comment. We would like to explain that the selected value of parameters can apply to different types of backdoor attacks. Specifically, instead of separately searching varying values of parameters for different backdoor attacks, we just set the parameter $\epsilon$ to 0.002 in Decomp-Rep to defend against all BadNet, Label Consistent, and BadCLIP. Our experimental results shown in Table 1 indicate that using this value of $\epsilon$ in our proposed Decomp-Rep can effectively decrease ASR while maintaining CACC for these backdoor attacks. It is worth noting that we chose the visual transformer as the backbone and achieved superior defense performance compared with the existing fine-tuning-based method CleanCLIP [1], especially for the current advanced attack BadCLIP.

Q1: I am wondering whether the "dynamic" attack yields the same experimental conclusions. Should it be classified as a local patch-based or global noise-based type?

A: Thanks for your interesting question. To solve your concern, we conducted experiments with another dynamic attack ISSBA [2] where the trigger is also dynamic and generated by an encoder-decoder network. We would like to first explain the reason for not using "Input-Aware Dynamic Backdoor Attack" [3] (IAD) is that the IAD attack is not targeted at CLIP and it is technically challenging to directly adapt the IAD attack to CLIP. Specifically, IAD originally targeted simple networks (i.e., PreActRes-18) and tiny datasets (i.e., MNIST, CIFAR-10, and GTSRB), and simultaneously trained the trigger generator and the victim model by a multitask loss. However, adapting the two tasks by the multitask loss in CLIP (fine-tuning on the dataset from CC3M) fails to train a satisfying trigger generator due to the huge differences in various aspects, including architectures (deep vs shallow), model training (fine-tuning vs training from scratch), and datasets (large-scale vs small-scale).

In order to address your concern, we have tried our best to successfully adapt another dynamic attack ISSBA [2] to CLIP. Specifically, we first generated poisoned images from CC3M with sample-specific triggers by an encoder-decoder network and then fine-tuned CLIP on the poisoned dataset. To better show the effect of ISSBA, we improve the illustration of Figure 2 in the revised manuscript. The figure shows that the dynamic trigger (globally optimized pixels) does not affect attention heads but only MLPs (as shown in the following table) in the last few layers. This observation is consistent with our findings.

Table: AH ablation (ASR)

Table: MLP ablation (ASR)

We abbreviate Forward/Backward/Separate-MLP as F-MLP/B-MLP/S-MLP. The above tables show that only ablating MLPs in the last few layers can significantly decrease the ASR. In summary, we conclude that this dynamic trigger could be classified as a global noise-based type.

Q2: Finding 3 states, "descriptive texts of MLPs in the last five layers have a distinct semantic difference, while that of MLPs in other layers have negligible changes in semantics." Why specifically the last five layers? Is this conclusion solely based on the experimental results presented in Figure 3?

A: Actually, this conclusion depends on not only the descriptive texts of MLPs but also some quantitative metrics. Specifically, we used Maximum Mean Discrepancy (MMD) to quantify the distribution difference between representations on clean and backdoored MLPs. The result is shown in Figure 6. (d-2) (in Appendix). We can see that the last few MLPs have larger MMD scores, which indicates these MLPs have been affected significantly. On the other hand, we also used descriptive texts of MLPs to characterize the semantic change in MLPs. To further quantify the semantic change, we calculate the cosine similarity between text embeddings of clean (green) and backdoor (red) descriptive texts. The experimental result is shown in the following table.

Table: Average text similarity

The above table shows that the cosine similarity values in the last five layers are significantly smaller than those in the previous layers, thereby indicating the significant semantic change in the last five MLPs. In summary, we conclude that global noise-based attacks mainly infect the last five MLPs.

Q3: As a defender, how can I determine whether a backdoor type is local patch-based or global noise-based?

A: Indeed, you do not know the types of triggers when you are a defender in real-world applications. In this case, we argue that you can defend against both two types of backdoor attacks by first checking whether AHs or MLPs are infected and repairing infected AHs or MLPs correspondingly. We would like to emphasize that this strategy would not expend too much additional inference cost because we just need to forward images into the network once and repair the representation on AHs or MLPs for different types of backdoor images simultaneously.

Q4: Additionally, the choice of the parameter in Figure 7 seems to struggle to achieve a good balance between attack success rate and clean accuracy.

A: Figure 7 indeed indicates that achieving the perfect trade-off between ASR and CACC (i.e., keeping CACC unchanged and decreasing ASR to zero) is considerably challenging. Actually, this dilemma is also faced by most backdoor attack methods. In the parameter selection procedure, we prioritize no significant decrease in CACC, which is significant and practical for downstream tasks. In this case, our proposed method still substantially makes the resulting ASR decrease, and the selected parameter applies to different backdoor attacks, i.e., we do not need to separately select parameters for varying backdoor attacks. Therefore, we believe that parameter selection is not a big barrier when using our method in real-world applications.

Reference:

[1] Cleanclip: Mitigating data poisoning attacks in multimodal contrastive learning. ICCV 2023.

[2] Invisible Backdoor Attack with Sample-Specific Triggers. ICCV 2021.

[3] Input-Aware Dynamic Backdoor Attack. NeurIPS 2020.

Thank you so much for your valuable comments!

W1: Limited novelty in some key findings. The observation that local patch-based attacks mainly affect AHs while global noise-based attacks mainly affect MLPs seems intuitive and somewhat expected.

A: Thanks for your comment. We would like to emphasize that this finding has not been revealed yet in the existing research and thus is indeed novel. Although there are several existing papers [1, 2] that have also identified the infected neurons in the backdoored model, they mainly focused on simple deep neural networks such as AlexNet and ResNet-18 (non-attention-based models). In contrast, our work focused on exploring backdoor attacks on CLIP (the prevalent attention-based model). Furthermore, they did not deeply explore the characteristics of the infected components, while we revealed that different backdoor attacks would affect different model components in CLIP. Therefore, we argue that this finding is indeed novel in the area of backdoor attacks.

Besides, we would like to emphasize that this finding is significantly important to the area of backdoor defense. Specifically, existing defense methods (such as CleanCLIP [3]) mainly focus on fine-tuning all model parameters to destroy the implanted backdoors of all backdoor attacks, neglecting the difference in infected model components of different backdoor attacks. Our findings can motivate future research to design more efficient and effective defense methods that specially handle the infected model components against different backdoor attacks instead of blindly treating all model components against all backdoor attacks.

W2: In Figure 2, it is difficult to distinguish the original images. Figure 3, while presenting a lot of descriptive texts, lacks intuitive clarity, making it challenging to draw conclusions directly from it.

A: We are sorry for the limited clarity in Figures 2 and 3. We have improved the illustration in Figures 2 and 3 in the revised manuscript. Specifically, we comparatively show the original images with various triggers and their head-specific attention maps and corresponding descriptive texts. To quantitatively illustrate the observation, we use head-specific MMD scores (indicating whether backdoor attacks significantly affect the representation of attention heads) and average text similarity (indicating whether the text semantics of infected components have changed dramatically).

W3: Defining the threat model at the beginning will make the paper clearer.

A: Thanks for your suggestion. We have formally defined the threat model in the preliminary section of the revised manuscript. Specifically, we define the thread model as {$\widetilde{\mathcal{V}}(\cdot),\widetilde{\mathcal{T}}(\cdot)$} where $\widetilde{\mathcal{V}}(\cdot)$ is associated with a implanted trigger $\Theta$.

Q1: How would the conclusions hold for semantic triggers rather than simple patches or textures? How might the affected layers change with semantic triggers?

A: Thanks for providing the interesting idea. To explore this point, we planned to use an image of "kitty" as the semantic trigger. Following our original experimental setting, we selected 500,000 images from CC3M as the fine-tuning dataset and poisoned 1,500 of them. Specifically, we first resized the "kitty" image to the patch size (16) and attached (random location) it to the poisoned images as the semantic trigger. After fine-tuning the dataset on a clean CLIP, the backdoored CLIP was associated with the semantic trigger "kitty". Then, we conducted the mean-ablation experiment to explore which model components were infected. The experimental results are shown in the following tables. Note that we have the following abbreviations: Forward-AH/MLP (F-AH/MLP), Backward-AH/MLP (B-AH/MLP), Separate-AH/MLP (S-AH/MLP).

Table: kitty AH (ASR)

Table: kitty MLP (ASR)

From the above tables, we can see that the ASRs have a great decrease after ablating the AHs in the 12th layer while having little effect after ablating MLPs. Therefore, our finding still holds for the semantic trigger.

Reference:

[1] Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. RAID 2018.

[2] Adversarial Neuron Pruning Purifies Backdoored Deep Models. NeurIPS 2021.

[3] Cleanclip: Mitigating data poisoning attacks in multimodal contrastive learning. ICCV 2023.

W4: Although the paper's experimentation is extensive, the authors could have incorporated stronger attacks like dynamic triggers.

A: Thanks for your valuable suggestion. We have conducted experiments with more backdoor attacks such as Reflection [1] and ISSBA [2] (dynamic triggers). Note that the reflection attack [1] originally targeted conventional neural networks (such as ResNet-34 and DenseNet) and we have tried our best to successfully adapt it to attack CLIP. Specifically, following the original setting, we used the images from PascalVOC as the reflection set to poison the training images (1,500 out of 500,000) from CC3M by the reflection method. Then, following our experimental setting, we fine-tuned a clean CLIP on the poisoned dataset and caused a backdoored CLIP (ASR is close to 99%). The results are shown in the following tables.

Table: AH ablation (reflection)

Table: MLP ablation (reflection)

Table: Defense performances (CACC/ASR)

From the above tables, we can see that the reflection attack and ISSBA mainly affect MLPs, and our proposed method can effectively defend against these attacks.

Q1: The authors fail to address the applicability of their analysis to non-CLIP-based multi-modal and attention-based models.

A: Thanks for providing the interesting idea. As shown in our response to W2, the experimental results verify the generality of our findings to other non-CLIP-based multi-modal and attention-based models (i.e., unimodal vision transformer) and CLIP with the large vision transformer.

Reference:

[1] Reflection backdoor: A natural backdoor attack on deep neural networks. ECCV 2020.

[2] Invisible Backdoor Attack with Sample-Specific Triggers. ICCV 2021.

[3] Hidden trigger backdoor attacks. AAAI 2020.

Thank you so much for your insightful comments!

W1: In conjunction with strength 3, I will note that although the experimentation is extensive, the authors do not do a good job of explaining the results or giving readers a better understanding of what the experiments show.

A: Thanks for your helpful suggestion. We have improved the illustration of Figure 2 in the revised manuscript, which compares the effect of various backdoor attacks more intuitively. We can see that local patch-based attacks (i.e., BadNet and BadCLIP) make many AHs pay more attention to the trigger compared with global noise-based attacks (i.e., Blended and ISSBA), thereby leading to larger MMD scores (indicating whether backdoor attacks significantly affect the representation of these AHs) and smaller text similarity (indicating whether the text semantic of these AHs have changed dramatically). We believe that the improved illustration in Figure 2 could give the readers a better understanding of our findings.

W2: For a more thorough ablation study, the authors could maybe experiment with varying architectures, or modify existing networks to change layer orders to understand their claims.

A: Thanks for your valuable suggestion. we have conducted more experiments on two types of models: Uni-ViT-B and Mul-ViT-L. Uni-ViT-B indicates the vision transformer (ViT-B/32) in supervised learning, while Mul-ViT-L means the vision transformer (ViT-L/14) in multimodal contrastive learning. Following the original attack setting, we used BadNet to attack these two models and explored the location of infected AHs.

Table: AH ablation on Uni-ViT-B (ASR)

Table: AH ablation on Mul-ViT-L (ASR)

From the above tables, we can see that ablating the last layer in Uni-ViT-B or Mul-ViT-L significantly decreases the ASR, which indicates that the infected AHs are still centered in the last layer.

W3: In conjunction with strength 4, although the paper's language is lucid, the math hampers readability.

A: Thanks for your helpful suggestion. We have simplified the mathematical symbols in the preliminary and the method sections. Specifically, we rewrote the cosine similarity as $S _{ij}=\phi(\widetilde{x} _{i},\widetilde{t} _{j})$ and thus simplify Eq. (1) in the preliminary section. In addition, we omitted the symbol $L$ and rewrote the symbols of the prototype $\Psi _{h}$ and the representation $C _{i} ^{h}$ in the method section, thereby simplifying Eq. (6) and Eq. (7) for better understanding. The detailed revision can be found in the highlighted part of the revised manuscript.