An Image Is Worth 1000 Lies: Transferability of Adversarial Images across Prompts on Vision-Language Models

Q1: The technical contribution and novelty may be limited.

A: In this work, we have multiple contributions. The first contribution of our paper is that we introduce a new problem, namely, transferability across prompts. The novelty of this contribution is recognized by all other reviewers. As the reviewer realized, we also proposed a method to improve the transferability across prompts. This method is straightforward yet proves to be highly effective and our experiments have consistently demonstrated its effectiveness.

Q2: What is the performance with a different ε?

A: This is a good question. In our paper, the perturbation size is set to 16/255, and we included this information in our paper. We conducted experiments with different perturbation sizes, and the results are shown in Table 7 of Appendix C. The results have shown that our method still consistently outperforms the baseline methods.

Q3: Transferability of the generated perturbations across different VLMs.

A: Yes, our results show that the perturbations can be transferred across different VLMs. We have expanded our discussion on this topic. Our experimental results in Table 8 in Appendix C show an overall ASR of close to 10% for cross-model transferability in addition to the cross-prompt transferability, despite differences in architecture and size, such as between OPT-2.7b and Vicuna-7b. The transferability across different VLMs depends on the similarities between the language models. We expect the cross-model transferability to be stronger if the language model components are highly similar (e.g. one model is the instruction fine-tuned variant of the source model).

Q4: Questions about the formula for non-targeted attack after equation (2).

A: Thank you for your careful observation. Indeed, it was a typo in the paper. The ultimate goal of the max-min process is to make the generation result of adversarial examples different from the original one, and the maximization operations should be applied to the image perturbation. We have switched the order of image perturbation $\delta_v$ and prompt perturbation $\delta_t$ in the formula.

This is a friendly reminder for the reviewer to update their comments. We are happy to provide more information if the reviewer still has some concerns. Thank you again for your time on our rebuttal.

Q1.1: The method may not be generalized to all mainstream VLMs as they are fast scaling up.

A: This is a good question. We discuss the challenges brought by scaling up the models in the following two perspectives.

1. Accurate gradient: With larger VLMs, it can be more challenging to obtain accurate gradient information. In our community, several approaches have also been proposed to address this problem such as EOT [1]. These approaches can be combined to effectively address the gradient accuracy challenge in large VLMs. In this work, we applied the standard attack method.
2. The memory issue caused by large VLMs: our community has actively proposed methods to allow for the usage of increasingly large models. Hugging Face, for instance, allows users to load models by distributing model shards across multiple graphics cards.

Q1.2: The applicability of CroPA under the black-box setting

A: Achieving strong transferability across models is indeed challenging. We also studied the performance under the black-box setting. Our experimental results in Table 7 in Appendix C indicate an ASR of close to 10% for transferability across models, despite differences in architecture and size, such as between OPT-2.7b and vicuna-7b. We expect the cross-model transferability to be stronger if the language model components are highly similar (e.g. one model is the instruction fine-tuned variant of the source model). Our study is expected to have broad applications as many models are built based on prevalent open-source vision models (e.g. OpenCLIP) and language models (e.g. LlaMA).

Q2: The framework learns adversarial prompts to enhance the attack's effectiveness. However, very few text prompt instances are shown in the paper, and the quality of the result text prompts is not sufficiently evaluated.

A: Thank the reviewer for pointing this out. The adversarial prompts are learnable embeddings instead of tokens. The following are more detailed clarifications regarding this question:

1. In optimization, we update the embedding of the prompt perturbations instead of updating the discrete tokens of the prompts. That's the reason why we are not able to show the adversarial prompts directly.
2. We also decoded the perturbed prompt embeddings by approximating the tokens in terms of cosine similarity. The approximate tokens of the perturbed prompt embeddings are the same as the original prompts, which can be found in Section 4.6.

Q3: Discussion about the difference with another work [2] under the black-box settings.

A: The study [2] under the black box settings indeed provides meaningful insights into the cross-model transferability, but our work focuses on a new perspective transferability across. Our works differ in the following aspects.

1. The goals of these two studies are different. The study [2] explores the adversarial robustness of VLMs from an evaluation perspective. It cannot be trivially applied to solve the cross-prompt transferability, which is the focus of this paper.

2. The evaluation methods in the two papers are different as well. The ASR (Attack Success Rate) discussed in our paper adheres to a strict standard: an attack is considered successful only if the model is deceived into generating the exact target text. The evaluation metric used in [2] is CLIP-score, which does not require the exact match of the generated text and the target text.

Moreover, our method makes sense in both black-box and white-box. The black-box attacks on VLMs can be combined with our CroPA method to enable our methods to work in a black-box fashion.

Q4: Typo "leanable" in the introduction (page 2)

A: Thanks for pointing this out, and we have corrected the typo.

[1] Athalye, Anish, Nicholas Carlini, and David Wagner. "Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples." International conference on machine learning. PMLR, 2018.

[2] Zhao, Yunqing, et al. "On evaluating adversarial robustness of large vision-language models." arXiv preprint arXiv:2305.16934 (2023).

Q1: Whether the proposed attack can indeed raise significant security issues？ This security concern would be more valid if the attack successfully tricks VLMs into producing contradictory predictions or harmful instructions.

A: We agree that tricking the model into generating harmful instructions is indeed a security concern, but our goal is different. The goal of this paper is to study the adversarial transferability across prompts of the VLMs.

There is some connection between the two topics. When the target output is set to harmful content, such as “suicide”, the model can output inappropriate sentences for images.

The detailed data are shown as follows, which is also presented in Table 9 in Appendix C.

Therefore, setting the target sentence to harmful words can pose threats to the applications of large VLMs. We would like to point out again that harmful content generation is not the focus of this work. But, we agree it is interesting to discuss the relationship between them.

Q2: Questions about the statement "prevent malicious usage of large VLMs for unauthorized extraction of sensitive information from personal images" and comparisons with the transferability across models in the context of jailbreak.

A: Thank you for highlighting this aspect.

1. The white-box setting also makes sense for this statement. For example, staff in technology companies may have access to users’ images, and there is potential for them to maliciously use their company’s models to extract users' information. Our method can serve as a privacy protection method if the images are pre-processed by adding invisible perturbations designed for their company’s models.

2. Our setting is much more stringent than the jailbreak scenarios in Large Language Models (LLMs). In this work, we assume that only images can be manipulated while the jailbreak-related study in LLMs allows for modifying prompts directly. Manipulating VLM outputs through images is challenging due to the difficulty in aligning visual information with text, as detailed in [1]. It is also not entirely clear how to achieve jailbreaking by modifying images only. However, it is a really good question to explore in our future work.

Q3: Whether some straightforward defense techniques can defeat the proposed attack, such as incorporating data augmentation on the input images?

A: Thanks for bringing up this question.

1. We tested the ASR (Attack Success Rate) of our method by incorporating common data augmentation techniques, such as random rotation on the input images, during the testing phase only. As detailed in Table 6 in Appendix C, the overall ASR of CroPA drops 5% while the performance of Multi-P drops 8%. Please note that we do not include any data augmentation during the optimization stage.

2. There is an arms race between attack and defense strategies. To counter data augmentation defense techniques, data augmentation can be easily integrated into the optimization process. Better attack and defense methods are orthogonal to this work. In other words, these techniques can all be combined with our work readily.

Q4: Is the improvement achieved by CroPA due to the increased number of textual prompts generated through perturbations? One approach to achieve a similar effect is to utilize another language model to rephrase the text prompts into new ones, thus serving as data augmentation.

A: This is a very good question.

1. Firstly, as shown in Figure 2 in the paper, the improvement from adding more prompts quickly saturates. Beyond the point where the number of prompts reaches ten, the increase in cross-prompt transferability becomes marginal, especially for Multi-P. Therefore, using more prompts does not necessarily mean a stronger performance.
2. Given the same number of prompts and the same number of optimization iterations as shown in Figure 3 in the paper, the proposed CroPA method clearly outperforms the Multi-P method.

[1] Lu, Dong, et al. "Set-level guidance attack: Boosting adversarial transferability of vision-language pre-training models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

This is a friendly reminder for the reviewer to update their comments. We are happy to provide more information if the reviewer still has some concerns. Thank you again for your time on our rebuttal.

Q1: The term "prompt perturbation" initially appears a bit confusing.

A: We thank the reviewer for pointing out this issue. We included clarifications on this in our paper to avoid confusion. The term “prompt perturbation” refers to the perturbations that are added to the embeddings of the prompts for obtaining image perturbation with strong cross-prompt transferability. It is not optimized to deceive the model.

Q2: How does the proposed method work for the longer prompt? Does the transferability still hold?

A: Yes, our method still outperforms the baseline approaches for longer target texts. The detailed data can also be found in Table 10 in Appendix C.

Q1: The method does not achieve transferability across models or images in addition to cross-prompt transferability. This might limit its practical applicability.

A: Thank you for your insightful observation regarding the transferability in other aspects. We appreciate the opportunity to clarify and expand upon these points.

1. The transferability across different images is orthogonal to the transferability across different prompts. Transferability across prompts can be combined with transferability across images by utilizing more images during the optimization stage. As stated in the paper [1], the perturbation computed over a larger dataset can increase the cross-image transferability.

2. The transferability across different models depends on the similarities between the model architecture. Our study is expected to have broad application as many models are built based on common open-source vision models (e.g. OpenCLIP) and language models (e.g. LlaMA).

Q2.1: Clarifications are required in some parts of the paper. Certain statements about the “textual prompts” are misleading.

A: Thanks for pointing this out. We agree with the reviewer about the potential misunderstanding. Following the suggestion, we replaced "learnable textual prompts" with "learnable prompts" across the paper.

Q2.2： More clarifications about the underlying setting of the experiments (e.g. optimization algorithm, the norm, and the perturbation size )

A: We follow the standard setting, and adopt the PGD as our optimization algorithm with L-infinity norm. The perturbation size is fixed to 16/255. We have updated our paper accordingly. We added the ablation experiments with different perturbation sizes as shown in Table 7 in Appendix C.

Q3: Typos in the conclusion part (“baseline approaches only archive” -> “achieve”).

A: We have fixed this typo, thanks for the correction.

Q4: Do you only add the prompt perturbations during the optimization phase or do you also add them during evaluation?

A: Prompt perturbations are only added during the optimization phase, and during the evaluation phase we do not add the prompt perturbation. The reason for using prompt perturbation during the optimization is to obtain the image perturbation with stronger cross-model transferability. During the test stage, the image perturbation is fixed and the prompt perturbations are not added.

Q5: What is the ASR of Multi-P when transferred across models or images?

A: The ASRs of the Multi-P are all zeros when transferred across images. The ASRs of Multi-P across different images are around 3% and the detailed data can be found in Table 8 in Appendix C. As the table shows, the Multi-P underperforms CroPA, even though the cross-model transferability performance of CroPA is limited. Please note that the techniques for enhancing cross-image transferability are orthogonal to our study, and they can be combined with our method.

Q6: Do you have a conjecture regarding the inability of CroPA to achieve transferability across models or images?

A: As reviewers realized, the transferability across models or images of both Multi-P and CroPA is not strong. A recent work [2] made initial explorations of the challenges of adversarial transferability across VLMs. It stated that it is difficult to align the visual information to text, namely the visual information can correspond to multiple texts. We conjecture that our methods on VLMs face similar challenges, but it leaves a new direction to improve transferability across the models or images with existing transferability-enhancing methods.

[1] Moosavi-Dezfooli, Seyed-Mohsen, et al. "Universal adversarial perturbations." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

[2] Lu, Dong, et al. "Set-level guidance attack: Boosting adversarial transferability of vision-language pre-training models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.