PoisonedEye: Knowledge Poisoning Attack on Retrieval-Augmented Generation based Large Vision-Language Models

We sincerely thank your constructive comments.

Q1. The retriever part can be improved such as using multimodal retriever (e.g., UniIR).

We conduct additional experiments specifically on the UniIR_CLIP_SF retriever model, which is the best-performing model in the UniIR framework. As the results shown in the table below, our attack achieves 99.45% PSR on the UniIR model, demonstrating our method's effectiveness on different retriever models.

* This experiment is conducted on LLaVA-v1.6-Mistral 7B LVLM, and Places-365 dataset.

Q2. How to defend such attack? The current work focus on exploring this attack.

We conduct additional experiments on the following possible defense strategies to demonsrate the effectiveness of our attacks. 1) add noise on poison image; 2) random crop poison image; 3) apply RoCLIP [1] that rematches every image with the text that is most similar to it in the database for retrieved samples. As the results shown in the table below, noise and random crop reduce the attack PSR to some extent. However, even after the random crop defense, the PSR remains at 45.20%, indicating that nearly half of the attacks are still successful. For RoCLIP, the defense is effective for the original attack. However, this defense can be easily bypassed by an enhanced attack that maximizes the poison image-text relation in the poison crafting process. As shown in the last line of the table, the RoCLIP only reduce the enhanced attack PSR by 38.11%, indicating that our poisoning attack framework can not be effectively defended so far.

* This experiment is conducted on Siglip-so400m retriever, LLaVA-v1.6-Mistral 7B LVLM, and Places-365 dataset.

Q3. How would attacker attack a private multimodal database? It seems the current attack requires access to insert data into a database, how does such attack work in real world setting where people/service host a private database or just use the web.

In most existing literature on poisoning attacks, the following real-world settings are commonly considered, where poisoning attacks can work for a private database: 1) Direct database access (e.g., insiders or hackers); 2) Malicious data vendors. Injecting poison samples into datasets sold to victims; 3) Public data poisoning. Attackers publish poison data on the Internet, which victims unknowingly collect to build their private databases. Therefore, our paper adopts a consistent setting for the attacker, aligning with existing poisoning attacks.

Referneces

[1] Robust contrastive language-image pretraining against data poisoning and backdoor attacks. NeurIPS'23.

We sincerely thank your constructive comments.

Q1. The ablation study on α, s, and ϵ is missing.

We conduct additional ablation studies on α, s, and ϵ as the table shown below.

* All experiments are conducted on Siglip-so400m, LLaVA-v1.6-Mistral 7B, and Places-365.

Q2. Why cannot also include a similar Class Query Targeted attack setting for the text part? Is there any challenge?

Under the class query targeted attack setting for texts, the attack should be activated when user asks similar questions to the target text. However, the inducibility of the poison text becomes very weak when user asks different questions, as shown in the rebuttal of reviewer adRg Q4. This is mainly due to the black box assumption of the VLM, which makes it hard to alter the response by crafting poison prompts. Therefore, enhancing attack capability on text side under the class query targeted attack scenario will be a valuable research direction for future works.

Q3. The writing should not focus on MuRAG, which could include other modalities (e.g., audio).

Thanks for your comment. We will revise the use of "MuRAG" in the final version of our paper.

Q4. The reason for using L2-distance as similarity metric is not justified.

Current works on RAG like DPR [1], UniIR [2], FAISS [3] mainly employ inner product and L2-distance as similarity search metrics. The L2-distance has equivalent effect as the inner product, because it can be mathematically transformed to the L2-distance by the equation shown below when the embeddings are normalized. A small L2-distance indicates closer proximity and stronger feature similarity. Therefore, the L2-distance metric is reasonable for retrieval process.

$$L_2(v_1,v_2) = ||v_1-v_2||_2 = \sqrt{||v_1||^2+||v_2||^2-2v_1⋅v_2} = \sqrt{2-2InnerProduct(v_1,v_2)}$$

Q5. Could the authors further explain the drop in PSR when retrieval number K increases? Does this mean the attack could be defended by increasing retrieval number? Could this issue be solved?

As retrieval number K increases, more clean samples from the database is retrieved and integrated into the prompt, then the VLM is provided with more information that could lead to the correct answer. Therefore, the VLM is less likely to produce the target answer and the PSR decreases.

There are indeed some drop in PSR as retrieval number K increases. However, this issue can be solved by increasing poison injecting numbers. In our main experiments, we only inject one poison sample into the knowledge database. We conduct additional experiments with more poison samples when K=8. As the results shown in the table below, the PSR increases when poison number is larger than 1, even when poison number is 2, the PSR holds at 85.20%, solving the PSR dropping issue.

References

[1] Dense Passage Retrieval, EMNLP'20

[2] UniIR, ECCV'24

[3] The Faiss library, arXiv:2401.08281

We sincerely thank your constructive comments.

This work adapts PoisonRAG [1] from LLMs to VLMs. While the attack concept is similar, VLMs require retrieving image-text pairs from the database, which makes crafting the injected query different.

Thank you for recognizing our contributions and efforts in devising the poisoning attacks on VLMs, which require nontrivial technical development, as you pointed out.

Q1. How much computation is needed to craft the poisoning prompt? What is the performance of the attack when there is a limited quota for iterating the poisoning prompt?

Since the poison text is fixed, the majority computation in our attack lies in creating the poison image through the signed gradient descent algorithm. A key hyper-parameter to balance computation and poisoning effect is generation steps s. The crafted image could converge well when s is large enough; however, when s is small, the image may not converge well and becomes unstable. We conduct additional ablation experiments to evaluate balance between attack performance and time consumption across different s. As shown in the table below, the time required per poison sample is consistently under 20 seconds, demonstrating the efficiency of our attack. Besides, even with only 10 steps, the PSR holds at 69.04%, indicating the effectiveness of our attack under limited iterations.

Q2. Are you using "I don't know" as the only target response? Do you have experiments using an incorrectly predicted class as the target response?

We have explored 5 different types of target responses and conducted experiments in Section 5.3.3 of our paper. The results show that our attack remains effective on these different target responses. Please refer to Section 5.3.3 and Appendix.G for details. We will further highlight the flexible choice of target response and the corresponding experiments in the revised paper, as you suggested. Thank you for your thoughtful question and valuable advice!

We sincerely thank your constructive comments.

Q1: "Class" definition & class query attack perform on captions datasets like COCO

The "class" in this context denotes a group of images that have similar semantic meanings (i.e., close L2 distance on pre-trained encoders like CLIP). For image classification datasets, we assume that they pre-classify semantically similar images into groups and attack images in the same class of the target image. For image caption datasets like COCO, our attack will be also effective on semantically similar images of the target image.

For example, we randomly select a target image and evaluate our attack for its semantically similar images (e.g., with 100 closest images on CLIP image distance) in COCO dataset. As the result shown below, the PSR holds at 67.39%, demonstrating our attack effectiveness on semantically similar query images in caption datasets like COCO.

* This experiment is conducted on Siglip-so400m, and LLaVA-v1.6-Mistral 7B.

Q2: How are the images selected for the class query attack? How to ensure that these images are representative of the class?

We aims to alter the system's response for query images that have similar semantic meanings to the target image. Therefore, regarding the target image as the center, we find its semantic neighbors (images with close L2 distance) from an auxiliary dataset (WebQA) by measuring the CLIP distance and select them as representative samples of the "class" centered by the target image.

Q3: How valid are the results of baseline and single query attack in real-world scenarios where the user's image is not identical?

In our main evaluation, we adopt class query attack assumption where the target image is not identical to the user's query image. The results in Table 1 and Figure 2 of our paper is exactly what you are looking for. For the baseline and single query attack assumption where the target image is identical to the user's query image, the experiments is conducted in Appendix.D.

Q4: Text influence on the attack & experiments about text distance

For the text part, we conduct additional experiments to evaluate the attack effectiveness when user asks similar questions with similar images to the target. For initial target text = "Which scene category does this image belong to?", we add 4 relevant target texts and test three other user queries that are highly relevant, medium relevant, and low relevant (marked by an LLM) to the target text as the table shown below. The results show that our attack can succeed in certain cases when the user query is relevant to the target query. When the user asks irrelevant questions, the attack does not activate as expected with PSR=2.19%. Therefore, enhancing attack capability on text side under the class query attack scenario will be a valuable research direction for future works.

Q5: Are there any adaptive defense strategies (e.g., random noise and RoCLIP)?

We conduct experiments on defense strategies including random noise, random crop, and RoCLIP to demonsrate the effectiveness of our attack. The experimental results show that our poisoning attack framework can not be effectively defended so far. Please refer to the rebuttal of Reviewer vyyS, Q2 for details.

Q6: Are there any other baselines or related work that could be compared to?

There are indeed some studies [1-3] on text RAG attacks. For PoisonedRAG [1], we have adapted it to the vison-language modality as our baseline method PoisonedEye-B. The comparison between our baseline and our proposed methods has been shown in Table 1 and Figure 2 of our paper. The results show that our proposed methods outperform the baseline in all cases. TrojanRAG [2] and AgentPoison [3] are remotely related with our work but focus on different attack settings and tasks. e.g., they both requires the attackers have the capability to modify user queries and [3] studies the LLM agent task. This is not consistent with our poisoning attack threat model, where attackers cannot modify user queries.

[1] PoisonedRAG, USENIX'25

[2] TrojanRAG, arXiv:2405.13401

[3] AgentPoison, NeurIPS'24