Stealthy Targeted Backdoor Attack Against Image Captioning

Dear Reviewer, we address each of your questions below.

Q1: More baselines and some backdoor attacks in visual question answering (VQA) can also be considered by removing the trigger of the textual question part.

A1: Thanks for the suggestion. We adopt two image caption backdoor attacks [1,2] and two backdoor attacks in the image domain [3,4] to our scenario on the Flickr8k dataset, where the trigger in the image corresponds to the object name in the caption. The poisoning rates are set at 5%. The results show the ASR of these methods is very low, which does not apply to our scenario. The VQA backdoor attack [5] considers that the backdoor will be triggered when the trigger pattern in the image and the keyword in the question exist simultaneously. Therefore, it will be ineffective after removing the question part, so we did not do this experiment.

[1] Hyun Kwon and Sanghyun Lee. "Toward backdoor attacks for image captioning model in deep neural networks." Security and Communication Networks, 2022.

[2] Li et al. "Object-oriented backdoor attack against image captioning." ICASSP, 2022.

[3] Nguyen et al. "Wanet-imperceptible warping-based backdoor attack." ICLR, 2021.

[4] Liu et al. "Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks." ECCV, 2020.

[5] Matthew et al. "Dual-key multimodal backdoors for visual question answering." CVPR, 2022.

Q2: Experiments on COCO dataset.

A2: Thanks for the comments. We have added experiments on the COCO dataset in the global response. The results show that our method can also achieve high ASR on the COCO dataset.

Q3: More advanced backdoor defenses.

A3: Thanks for the comments. We have added two backdoor defenses Implicit Backdoor Adversarial Unlearning (I-BAU) [6] and Channel Lipschitzness-based Pruning (CLP) [7] to measure the effects against our attack. The results show that ASRs do not drop to very low values, suggesting that these defense methods are ineffective against our attacks.

[6] Zeng et al. "Adversarial Unlearning of Backdoors via Implicit Hypergradient." ICLR, 2022.

[7] Zheng et al. "Data-free Backdoor Removal based on Channel Lipschitzness." ECCV, 2022.

Q4: Add the LPIPS metric to measure the distance between the original samples and poisoned samples.

A4: Thanks for the suggestion. We have added experiments about LPIPS distance on Flickr8k and Flickr30k datasets. We select 50 clean samples and generate corresponding poisoned samples to calculate LPIPS distance. A lower value of LPIPS indicates that the two images are more similar, which means that the poisoned samples we generated are similar to the corresponding clean samples. And we provide some poisoned examples in Figure 8 in Appendix. The trigger in poisoned samples is difficult for humans to detect.

Q5: Some typos in the paper.

A5: Thanks for pointing out these typos. We have fixed them in the revised version of our paper.

Dear Reviewer, we address each of your questions below.

Q1: More advanced target models.

A1: Thanks for the comments. We have added two transformer-based architectures, Unified VLP and ViT-GPT2, to evaluate our method in the global response. The results show that our method can still achieve high ASR on these advanced target models.

Q2: More metrics to evaluate the model’s performance on benign samples, such as METEOR and CIDEr.

A2: Thanks for the suggestion. We have added these two metrics of ResNet101-LSTM results to Tables 4 and 5 in the Appendix of the updated version of our paper.

Q3: The consistency with the ASR values.

A3: Thanks for pointing out this issue. In the revised version, we have updated Table 2 and Table 3, which scale to a 0-100 range to maintain consistency.

Q4: Are the two output sentences distinguishable only by the object name, like the training samples?

A4: Thanks for the comments. The two output sentences may not necessarily be the same as the training samples except for the object name, but the semantics of the sentences match the image content well except for the object name. We provide some examples in Figure 4 in the Appendix of the revision.

Q3: The experimental setup and results fall short of comprehensiveness. The authors' experiments do not benchmark against prevalent backdoor attacks and defense strategies.

A3: Thanks for the comments. In the revision, we have added four attacks and two defenses. Specifically, we adapt two image caption backdoor attacks [2,3] and two backdoor attacks in the image domain [4,5] to our scenario on the Flickr8k dataset, where the trigger in the image corresponds to the object name in the caption. The poisoning rates are set at 5%. The results show the ASR of these methods is very low, which does not apply to our scenario.

We add two backdoor defenses Implicit Backdoor Adversarial Unlearning (I-BAU) [6] and Channel Lipschitzness-based Pruning (CLP) [7] to measure the effects against our attack. The results show that ASRs do not drop to very low values, suggesting that these defense methods are ineffective against our attacks.

[2] Hyun Kwon and Sanghyun Lee. "Toward backdoor attacks for image captioning model in deep neural networks." Security and Communication Networks, 2022.

[3] Li et al. "Object-oriented backdoor attack against image captioning." ICASSP, 2022.

[4] Nguyen et al. "Wanet-imperceptible warping-based backdoor attack." ICLR, 2021.

[5] Liu et al. "Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks." ECCV, 2020.

[6] Zeng et al. "Adversarial Unlearning of Backdoors via Implicit Hypergradient." ICLR, 2022.

[7] Zheng et al. "Data-free Backdoor Removal based on Channel Lipschitzness." ECCV, 2022.

Dear Reviewer, we address each of your questions below.

Q1: The specific motivations behind choosing this model and the challenges in contrast to VQA-targeted backdoor attacks.

A1: Thanks for the valuable comments. Image captioning has many applications such as usage in virtual assistants, for visually impaired persons, image annotation, and etc. The security of image caption models in these applications requires people's attention. For instance, an image caption model coupled with a speech system can help a visually impaired person to perceive the external environment. If the image caption model is embedded with a backdoor, it may adversely affect the visually impaired person. As mentioned in the Discussion section, it motivates us to study the backdoor attack in image captioning tasks. For backdoor attacks, image captioning is more challenging than VQA. First, the attacker is assumed to have no access to the model training process and does not know the model architecture of the target model, whose ability is weaker compared to the VQA backdoor attack. In contrast, the VQA backdoor attack assumes the attacker can control the whole training process. Second, in the image caption domain, it's challenging to bind the trigger to a specific word in the caption rather than the whole output caption while maintaining stealthiness in the image domain. However, in the VQA task, the trigger in the image is visible to the human eye, and the trigger corresponds to the entire specific answer, which is easier to achieve. Third, in the VQA task, the poisoned samples generated by the attacker only target a certain model that the attacker wants to train. In the image caption, the attacker's goal is to generate model-agnostic poisoned samples, which can successfully attack a variety of model architectures, which is also challenging. In addition, the backdoor attack in VQA cannot be directly migrated to the backdoor attack in the considered image caption scenario, suggesting that the design of a new attack scheme is needed.

Q2: The proposed methodology lacks distinct innovation.

A2: Thanks for the comments. A naive solution is to simply use a trigger pattern and a specific weird caption as a backdoor. However, this would make poisoned samples easy for people to observe as anomalies in the image and text domains. We would like to emphasize our work's novelty from both image and caption perspectives. For the image, we have to ensure that the trigger is very small and will not affect other parts of the image. We also need to ensure the stealthiness of the trigger. Thus, we propose to optimize the trigger pattern to make it invisible. When optimizing the trigger pattern, we leverage the idea of adversarial examples on object detection, but it is still different from backdoor attacks on object detection. In the object detection backdoor attack [1], the trigger is usually randomly chosen without optimization, such as a chessboard. And their trigger is visible to the human eye. For the caption, we connect the trigger pattern to an object name in the caption instead of the whole specific caption, which can make the poisoned sample invisible in the text domain as well. This task is difficult because our trigger only affects selected object names in the caption. Combining the above, we make poisoned samples similar to normal samples on both domains. And the poisoned samples we generate are effective across different model architectures, which demonstrates the generalizability of our approach.

[1] Chan et al. "BadDet: Backdoor Attacks on Object Detection." ECCV, 2022.

Thank you for the effort put into addressing the reviews for your submission to ICLR. I appreciate the time and work that went into your responses and the revisions of your manuscript. However, the author's response did not meet my expectations in terms of innovative and challenging explanations, so I left the original score unchanged.

Q6: The definition of $ASR=\frac{N_p}{N_t}$. where $N_p$ is the number of sentences containing the target object and $N_t$ is the number of sentences containing the ground truth source object. Here, every sample's output would be counted as one sentence, or there might be multiple sentences. In the former case, we denote $N_p$ as the number of outputs in which the target object appears and $N_t$ as the number of total samples would be the same.

A6: Thanks for the comments. It's the former case. Every sample's output would be counted as one sentence. Your understanding is correct.

Dear Reviewer, we address each of your questions below.

Q1: What is the main difference between the proposed method and another paper "Dual-Key Multimodal Backdoors for Visual Question Answering" (We refer to Dual-Key paper for simplicity)?

A1: Thanks for the interesting insight. We would like to illustrate the significant differences between our work and the Dual-Key paper in terms of task, methodology and threat model:

Task: The model's inputs in Dual-Key paper are an image and its corresponding question, and the output is the answer. For the image caption task, the input is an image without the question part, and the output is a sentence describing the image. And we would like to clarify the difference in the use of object detector: The models in Dual-Key paper first process images through a fixed, pretrained object detector. The object detector is part of their backdoored model. Our work uses the object detector (YOLOv5s) to optimize the trigger pattern and is not part of the image caption model.

Methodology: The Dual-Key paper unites a patch in the image and a word in the question as a trigger, and the output is a specific answer (usually a word). The trigger is triggered only if a patch is in the image and a word is in the question. Our trigger is a patch, while the output corresponds to a word in the caption, not the whole sentence. Moreover, our patch is stealthy when added to the image, whereas the patch in Dual-Key paper is visible to the human eye. The backdoor attack in VQA cannot be directly migrated to the backdoor attack in the image caption scenario we considered.

Threat model: The attacker has no access to the model training process and does not know the model architecture of the target model, whose ability is weaker compared to the VQA backdoor attack. In contrast, The Dual-Key paper assumes the attacker can control the whole training process of the VQA model, which is a white-box attack scenario. In the VQA task, the poisoned samples generated by the attacker only target a certain model that the attacker wants to train. But in the image caption, the attacker's goal is to generate model-agnostic poisoned samples, which can successfully attack a variety of model architectures.

Q2: The Figure 1 is not very intuitive. The image caption models maybe common, but involving the specific architecture in Figure 1 (The Experimental Setup Section is not clear to me) would help to understand the attack procedures. I assume the input images go into the object detector first, then the output features would be forword to ResNet101-LSTM model for text generation? Are there any other modules between object detector and ResNEt101-LSTM? Also, the proposed method adds the trigger to the center of bounding box generated by object detector. It would be better to illustrate it in the Figure 1 (for better understanding).

A2: We apologize for the potential confusion. We have updated Figure 1 for readers to understand the process better. We give a simple explanation based on Figure 1. First, the object detector is only used to optimize the trigger pattern. After generating the trigger pattern, we add it to clean samples and modify the object names in the corresponding captions to get our poisoned samples. Then, the clean and poisoned samples are directly fed to the ResNet101-LSTM model for training. There are no other modules between the object detector and ResNet101-LSTM.

Q3: Experiments on other CNN-LSTM and transformer based archs.

A3: Thanks for the comments. We have added some experiments on ResNet50-LSTM, Unified VLP and ViT-GPT2. The results show that our method can still achieve high ASR on these target models. Please refer to the global comments.

Q4: Experiments on COCO dataset.

A4: Thanks for the comments. We have added some experiments on COCO dataset. The results show that our method can also achieve high ASR on the COCO dataset. Please refer to the global comments.

Q5: How important is the Trigger location?

A5: Thanks for the comments. The location of the trigger within the object is important. We set Flickr8k and ResNet101-LSTM as the dataset and the architecture, then change the trigger location to the bottom right corner and the top left corner of the object bounding box for comparisons.

From the experimental results, we observe a significant drop in ASR, this is because the corners of the bounding box don't necessarily contain objects, and it could be the background of the image. We add the trigger to the center of the bounding box, because we aim to involve the trigger in the object but without affecting the rest of the image.