Representation Confusion: Towards Representation Backdoor on CLIP via Concept Activation

W1: Motivation Complexity

A1: Thanks for your comments. To make our paper clearer, we have substantially revised our discussion of cognitive science by removing Section 4.1 (preliminary of cognitive science) as it was less relevant to our method. We streamlined the introduction while strengthening the connection between the Hopfieldian view and current concept extraction methods in explainable AI. This revision better illustrates how the Hopfieldian perspective inspired explainable AI methodologies aimed at revealing what deep neural networks learn during training—specifically, how they develop representations through visual features and human-interpretable concepts that drive their predictive decision-making. Our empirical findings demonstrate that in computer vision models, neural activations are fundamentally driven by distinct visual features, providing a strong theoretical foundation and motivation for our proposed concept-based attack method.

W2: Novelty of the method

A2: Our work fundamentally differs from physical backdoor attacks in trigger methodology. Physical backdoor attacks rely on real-world triggers (e.g., stickers in facial recognition) that function across various environmental conditions while maintaining their essential triggering nature. In contrast, our approach operates without embedding any digital or physical triggers in the input data, instead manipulating the model's behavior through inherent concepts. This trigger-free design circumvents traditional detection methods and represents a significant departure from conventional backdoor attack paradigms.

W3: Method Generality

A3: Our method is transferable to various image-based tasks, with CLIP serving as a demonstration framework. We specifically focused on the image encoder to ensure broad generalizability, as this approach facilitates easier adaptation across different image tasks, while modifying both modalities would potentially limit the attack's versatility. The single-modality design choice aligns with current trends in multimodal attacks, particularly in MLLM jailbreaking research, where image-based interventions are prevalent [7][8].

W4： Experiment Limitations

A4: Our revised experiments demonstrate that our method surpasses BadCLIP[9] in attack effectiveness, with BadCLIP showing greater vulnerability to existing defenses. Further comparisons with advanced attack and defense baselines are in progress, and results will be updated accordingly. The BadCLIP performance data is presented below.

Clean Accuracy (CACC) (%) and Attack Success Rate (ASR) (%) of BadCLIP against different kinds of defense baselines

W5：Defense Limitations [2][3].

A5:The defense method [2] evaluates modality alignment through QA pairs generated by large models, but fails against our attack for two reasons. First, image-specific questions may not target the attacked concept (e.g., asking about a house when the attack concept is "water"). Second, the defense is designed for text-generation models and incompatible with CLIP's label-based predictions. Furthermore, question quality affects reliability even for normal images. The defense in [3] is inapplicable as our method doesn't utilize the text encoder.

References:

[1] Backdoor Attacks Against Deep Learning Systems in the Physical World.

[2] VDC: Versatile Data Cleanser based on Visual-Linguistic Inconsistency by Multimodal Large Language Models.

[3] Robust contrastive language-image pretraining against data poisoning and backdoor attacks.

[4] Kim, Been, et al. "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)." International conference on machine learning. PMLR, 2018.

[5] Bhalla, Usha, et al. "Interpreting clip with sparse linear concept embeddings (splice)." NeurIPS 2024.

[6] Cheng Y A, Rodriguez I F, Chen S, et al. RTify: Aligning Deep Neural Networks with Human Behavioral Decisions. NeurIPS 2024.

[7] Niu Z, Sun Y, Ren H, et al. Efficient LLM-Jailbreaking by Introducing Visual Modality[J]. arXiv preprint arXiv:2405.20015, 2024.

[8] Tao X, Zhong S, Li L, et al. ImgTrojan: Jailbreaking Vision-Language Models with ONE Image[J]. arXiv preprint arXiv:2403.02910, 2024.

[9] Bai, Jiawang, et al. "BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

W1: The specific technical differences from [1]

A1: The difference between our work and physical backdoor attacks lies in the use and nature of triggers. Physical backdoor attacks typically use physical triggers that exist in the real world. These triggers are designed to work across varying conditions like angles, lighting, and distances. While the trigger itself may vary in appearance due to these conditions, its essence remains consistent with traditional backdoor attacks: the model is conditioned to misbehave when this specific trigger is present, whether digital or physical. Our method manipulates the model’s behavior in a more subtle and trigger-free way. This means that there is no external cue in the input data that the model depends on to exhibit the backdoor behavior. This represents a significant conceptual difference, as we bypass the need for any inserted trigger altogether. Our approach addresses limitations inherent to traditional and physical backdoor attacks, particularly in scenarios where the presence of a trigger can be detected or mitigated.

W2: The cognitive science section seems unnecessary.

A2: To enhance clarity, we revised our discussion of cognitive science, removing Section 4.1 and streamlining the introduction to focus on essential connections between the Hopfieldian view and modern explainable AI. The Hopfieldian perspective has inspired methods for understanding neural networks' learned representations—specifically, how visual features and human-interpretable concepts drive model decisions. This understanding of concept-driven neural activations provides the key motivation for our proposed attack method.

W3: The practicality of the proposed attack.

A3:We have set a threshold of the target concept in the training process, and in practice, only when the concept score of the target concept is higher than the threshold, then the attack will success. We conducted additional experiments using multiple combined concepts as triggers. After the backdoor attack, if any one of these concepts in the output image exceeds its threshold, the model predicts the target label. Using mixed concepts as triggers significantly enhances the practicality of this backdoor attack. The results of the mixed concept as a trigger is shown below.

Table: Combined concepts as backdoor triggers: performance on CIFAR-10 using TCAV as concept score calculator with high ASR(%) and CACC(%)

W4: Concept size

A4: Our method doesn't require a fixed concept size. Our methodology is invariant to the input dimensions or concept sizes, as it operates solely on the extracted concept scores rather than the intermediate visual representations.

W5: Threshold σ

A5:We calculate the target concept scores for all training samples. We select the top 1% (poisoning ratio) of samples, and use the score of the last sample in this top 1% as threshold σ. In the test set, any sample with a concept score above σ will be predicted as the target label by the model.

W6: Defense strategies for SSL, such as SSL-Cleanse[2] and DECREE[3]

A6:We evaluated the effectiveness of the two proposed reference defenses against our attack. These defenses fail to even detect whether our encoder is backdoored. The results are shown below.

Backdoor Detection Methods Comparison, and "true" indicates detected backdoors, "false" indicates undetected.

References:

[1] Wenger et al., Finding Naturally-Occurring Physical Backdoors in Image Datasets, NeurIPS '22.

[2] Zheng et al., SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning, ECCV '24.

[3] Feng et al., Detecting Backdoors in Pre-trained Encoders, CVPR '23.

[4] Kim, Been, et al. "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)." International conference on machine learning. PMLR, 2018.

[5] Bhalla, Usha, et al. "Interpreting clip with sparse linear concept embeddings (splice)." NeurIPS 2024.

[6] Cheng Y A, Rodriguez I F, Chen S, et al. RTify: Aligning Deep Neural Networks with Human Behavioral Decisions. NeurIPS 2024.

W1：Leaving other potential tasks unexplored

A1: This paper focuses on backdoor attacks in CLIP's classification tasks, which is consistent with most existing backdoor attacks on CLIP[1]. Thus we think focusing on classification already has significant contributions compared to those related work. In future work, we plan to extend our approach to prompt generation tasks, where concepts will serve as triggers to make multimodal large models generate specific biased responses, enabling more sophisticated backdoor attacks.

W2：Claims to target CLIP but lacks comparison with representative CLIP-specific attacks like BadCLIP.

A2: In the revised version we have included BadCLIP as a baseline for comparison. Our results show that its attack performance remains inferior to our proposed attack method. Moreover, BadCLIP can be detected and defended against by various defense methods while our attack remains effective. The results of BadCLIP as a attack baseline is shown below. Clean Accuracy (CACC) (%) and Attack Success Rate (ASR) (%) of BadCLIP against different kinds of defense baselines

W3：Insufficient justification for concept selection threshold (σ) choice

A3: If we use "water" as a trigger concept, we calculate concept scores for all samples in the training set. We select the top 1% (the poisoning ratio) of samples with the highest scores. The concept score of the last sample in this top 1% becomes our threshold σ (different concepts have different thresholds). For the test set, any sample with a concept score higher than σ will be predicted as the target label by the model.

W4：Limited sensitivity analysis on poisoning rate

A4: We do not agree. We have presented the ablation experiments on different poisoning rates in Table 7 of the paper. Table 7 shows that even with reduced poisoning ratios, our attack maintains nearly 100% ASR while keeping CACC above 97%, demonstrating it effectiveness even with minimal poisoned data

Q1: How to select optimal trigger concepts?

A5: Indeed, some concepts work better as triggers. For example, using "engine" as a trigger with "airplane" as the target label leads to more successful attacks. While using triggers that are very different from the original label may slightly reduce effectiveness, the attack success rate generally remains above 90%.

Q2: Using combinations of multiple concepts as triggers

A6: In the revised version, we added experiments using mixed concepts as triggers. See page 8 for details. When using two concepts as triggers, the model predicts the target label if either concept's score exceeds its threshold in test data. This mixed-concept approach makes backdoor attacks more stealthy by diversifying trigger conditions, making detection and defense more challenging. This attack method could be extended to use more combined concepts for even more covert backdoor attacks. The results is shown in the Table below.

Table: Combined concepts as backdoor triggers: performance on CIFAR-10 using TCAV as concept score calculator with high ASR(%) and CACC(%)

Q3: Can this method be extended to other types of multimodal models?

A7: This work can definitely be extended to multimodal text generation. In MLLMs (Multimodal Large Language Models), if an input image contains certain concepts, it could trigger a backdoor that causes the model to generate text with specific biases, violent content, or jailbreak responses. This creates a subtle form of backdoor attack in the multimodal domain, where visual triggers influence text generation behavior. However, as in this paper we mainly focus on CLIP. Extending to other MLLMs will be future work.

Q4 :How does the method perform on larger-scale datasets?

A8: We are currently validating our attack on the Caltech-101 dataset. While ImageNet would be ideal for evaluation, its scale exceeds our GPU capacity. Results will be updated upon completion of these experiments.

References:

[1] Liang S, Zhu M, Liu A, et al. Badclip: Dual-embedding guided backdoor attack on multimodal contrastive learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 24645-24654.

W1: How to implement it in practice

A1: Since the target image has many concepts, our method supports composite concepts with conjunctions, i.e., the attack will succeed if any one concept is higher than the threshold. Thus, if the concept vector threshold of the target image is indeed lower than the pre-defined value, we can use composite concepts, i.e., find the target image with one concept higher threshold, to make the attack effective. As we mention in the introduction, the internal states of image classifiers can be represented as human-understandable concepts. Thus, it is impossible for all concepts to be lower than the threshold, as if all concepts/representations are irrelevant to the class, then this image should not belong to the specific class.

We appreciate your suggestion to deliberately alter the concept vector score of the target image, but we want to clarify that the concept score is an inherent property of the images and the classifier and cannot be changed directly. This aligns with our notion that concepts are intrinsic to the neural network’s understanding and must be activated rather than modified.

Additionally, we emphasize that our scenario is more practical, as the targeted label has a strong relationship with the “explicit concept,” making it more intuitive and human-understandable. For example, both ducks and swans are typically associated with water, so misclassifying a duck as a swan appears more natural. In contrast, the traditional approach of misclassifying a duck as a tiger, while stealthy, is less intuitive and more confusing.

W2.1: How reliable is the concept score?

A2.1: Firstly, we respectfully argue that the adversary CANNOT change the concept vector scores of any images since these concept scores are inherent properties of the corresponding images and the networks. Specifically, for a given image and its concept vector, a high score of a specific concept means that this image contains such a concept and vice versa. Since our backdoor attack would not modify any training images, the adversary thus could not change any concept vector scores.

Secondly, we note that the concept score is reliable across different concept extraction methods through extensive experiments in Table 8 in the paper.

W2.2: When does the proposed attack fail, and is it caused by the concept score calculation?

A2.2: We clarify our proposed attack will fail ONLY in the case where all training and targeted images do not contain any concept from our pre-selected concept set $C$. However, such a case is almost impossible to happen in real-world threat scenarios since to make the attack effective, the adversary will select a concept set to cover concepts in real-world images as much as possible.

W2.3:Can the same strategy be used in feature space for random statistics instead of calculating the concept vector score?

A2.3: Our strategy cannot adopt concept vectors calculated from random feature spaces. Specifically, concepts are natural features that align well with human perception. The reason that our proposed backdoor attack can work is that it leverages these natural features (i.e., concepts) that widely exist in real-world images. Unfortunately, these natural features apparently do not exist in random feature spaces, thus concept vectors calculated from such spaces would not have meaningful information, and our backdoor attack could not leverage such meaningless concept vectors.

W3: Adaptive defenses.

A3: We further evaluated our backdoor attack against two recently proposed defense methods: SSL-Cleanse[1] and DECREE[2]. Our results show that both defense methods fail to detect whether the encoder has been backdoored with the target concept, let alone remove the backdoor effectively. The result is shown in the Table below.

Backdoor Detection Methods Comparison, and "true" indicates detected backdoors, "false" indicates undetected.

For the adaptive defenses, note that this is the paper introducing confusing the concept vectors as attacks, we think it is a good question but it is beyond the scope of the paper.

Minor: Minor: Line 219, bold -> green. The font size in Figure 4 is too small.

A4: Thanks & addressed.

Questions: Please clarify about triggering on a randomly selected test image, the reliability of the concept score, and adaptive defenses.

A5: Please refer to the replies above

References:

[1] Zheng et al., SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning, ECCV '24.

[2] Feng et al., Detecting Backdoors in Pre-trained Encoders, CVPR '23.