On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains

We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.

Q: additional retrieval cases ..., especially in the legal domain.

R: We actually included the experiments on the legal domain in Table 10 in Section D in the supplementary material. Overall, we observe similar attack success rates in the legal domain as in the medical domain presented in the main text. We will highlight this in the revised version.

Q: l2 norm baseline relatively simple

R: $\ell_2$-norm is a simple but also the standard baseline in many existing literature [1,2,3]. To address your concern, following [1,2,3], we also include the perplexity filter, which examines the perplexity of the text, as an additional baseline. The results are shown in the Table below where we observe that the perplexity filter is not effective, thus validating the effectiveness of our method.

Table: Detection rate of the perplexity filter on the MedMCQA corpus. The results are averaged over 5 runs.

Q: application efficiency be improved for large-scale data and real-time systems?

R: We are not 100% certain regarding what you mean by "application efficiency." So we will refer to the computation cost/efficiency of our method and respond accordingly. The computation of our defense method is very light. We only need to compute the covariance offline once, and then during inference time, we only need to calculate the Mahalanobis distance once for each query, which is very efficient. To further improve efficiency, we can take a batch of queries and compute the Mahalanobis distance for all queries in the batch simultaneously.

Q: cosine similarity

R: The use of inner product is a common practice in current literature [1,2,3]. To address your concern, we also evaluated cosine similarity and the summarized results are shown in the table below. We observe similar high attack success rates when using cosine similarity. We will include this in the revised version.

Table: Top 2 retrieval success rates under cosine similarity with Contriever as the retriever and MedMCQA, PubMedQA as the query corpus.

Q: The description of the defense is not clearly explained, easy to make confuse to reader.

R: We will revise the description of the defense in the main text to make it clearer. The main workflow of our defense is for the defender to select a set of clean corpus (corresponding to a set of queries) to be protected. Then, the defender computes the covariance of the embeddings of the clean corpus. During inference time, for each query, the defender computes the Mahalanobis distance between the query and the clean corpus. If the distance is larger than a threshold, the query is rejected.

Q: How about top5, top10 retrieval performance

R: The retrieval rates reported in the main text are a non-decreasing function of the $k$ value used in the retrieval. So the top-5 and top-10 retrieval performance will be no worse than the top-2 retrieval performance. In particular, we report the results for $k=10$, where we observe near perfect retrieval rates. We will include this in the revised version.

Table: Top-10 retrieval success rates with Contriever as the retriever and MedMCQA, PubMedQA as the query corpus.

Q: More related work and flag in the abstract for potential harmful information

R: We will include more related work as you suggested. We will also add a flag in the abstract to indicate that the paper contains potentially harmful information.

Q: Math explanation

Providing rigorous mathematical justification is challenging due to the sparse, preliminary theoretical understanding of transformer-based models [4]. We aim to address this in future work.

Refs: [1] Xiong et al., "Benchmarking Retrieval-Augmented Generation for Medicine", ACL 2024

[2] Miao et al., "Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications", Medicina (Kaunas). 2024

[3] Zou et al., "PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models", USENIX Security, 2025

[4] Tian et al., "Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer", NeurIPS 2023.

We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.

Q: In Lines 105 (left) and 150 (right), the notation refers to the retrievers and the embedding function, respectively. Could the authors please provide some explanation regarding the definitions of the retrievers?

R: In our paper, we use the terms "retriever" and "embedding function" interchangeably, which take an input text and output a vector representation of the text. We will clarify this point to avoid confusion in the revised version.

Q: Besides, the results in Sections 3-5 heavily rely on the embedding function of the input. However, my past experience implies that the performance of those embedding models released before 2024, especially those token-level models like BERT, is far from satisfactory. As mentioned in the "Experimental Designs Or Analyses" part, I suppose the retrievers used in this paper are slightly out of date. (P.S. I am unfamiliar with RAG's research, and thus, I cannot provide exact references.) Are there any retrieval systems that are based on SOTA embedding models, e.g., text-embedding-3 from OpenAI?

R: The three retrievers used in the main text are the state-of-the-art models for RAG following very recent literature [1,2,3]. To address your concern, we also include the results of the text-embedding-3 from OpenAI in the Table below. We observe that the attack success rates are similar to those of the state-of-the-art models. We will include this in the revised version.

Table. Top 2 retrieval success rates with text-embedding-3 as the retriever and MedMCQA, PubMedQA as the query corpus.

Q: I suggest including more illustrative examples to further improve the presentation of this paper. The examples given in Figure 1 seem artificial. In Section A, some examples from the real dataset are presented. I suppose presenting some real-world examples can better illustrate the safety risk faced by the RAG systems.

R: We will follow your suggestions to replace the current Figure 1 and include some real-world examples in the main text to better represent the safety risks faced by RAG systems.

Refs: [1] Xiong et al., "Benchmarking Retrieval-Augmented Generation for Medicine", ACL 2024

[2] Miao et al., "Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications", Medicina (Kaunas). 2024

[3] Zou et al., "PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models", USENIX Security, 2025

We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.

Q: Universal poisoning attacks against RAG have already been shown to be effective [1] ...

R: Thank you for your valuable suggestions regarding the work in [1]. Overall, our work differs from [1] in terms of goals, insights, and application scenarios, despite the similarity in attack implementations. Detailed discussions are provided as follows.

• Goal. Our objective is to investigate and comprehend the robustness of retrieval systems employed in RAG. We achieve this by injecting various types of information, encompassing both irrelevant and relevant content, into the corpus and evaluating the ease or difficulty of retrieval. The goal of [1], on the other hand, is to inject only query-relevant context to deceive the LLM's generation process upon retrieval.
• Insights. We provide explanations on the difficulty/easiness of the retrieval of different kinds of information, which is not covered in the work of [1]. Moreover, based on the developed insights, we propose a new defense that can effectively filter out the adversarial documents generated by [1].

Q: The orthogonal augmentation property essentially demonstrates a simple fact ...

R: The orthogonal augmentation property concerns the effects on changes in embedding vectors generated by text embedding models when manipulating the text input. Specifically, orthogonal augmentation examines the relationship between $f(q)$ and the change in the embedding space that occurs when shifting $q$ to $q \oplus p$, namely $v \triangleq f(q \oplus p) - f(q)$, for two documents $p$ and $q$, which can be either semantically relevant or irrelevant (as discussed in Section 4).

Overall, we feel that it is essential to understand the orthogonal augmentation property, as it can be seen in many poisoning attacks against text embedding models. We tried to investigate this property both in theoretical and empirical ways. It turns out the current literature on the theoretical understanding of transformer-based models seems to be sparse and preliminary. For example, recent research, such as the work [A] published in NeurIPS 2023, studied the weight dynamics of transformers under strong assumptions which may not align with real-world use cases, such as single-layer self-attention, no positional encoding, and excessively long input contexts. As a result, in the paper, we present the empirical study of the orthogonal augmentation property.

Q: The proposed defense requires access to a collection of clean documents ...

R: We feel that the assumption is reasonable in the following sense. First, it is straightforward (and can be theoretically proven) to see that the defender is impossible to defend all corpus/queries (Because that would make the whole input space to be clean). As a result, a more feasible solution is to find a set of important queries and their corresponding corpus and then prioritize protecting these selected ones. In fact, many RAG poisoning attacks are targeted attacks in the sense that the attacker only wants to poison/manipulate a small set of queries [2,3,4]. So we believe that such an assumption is reasonable. We will clarify this in the revised version.

Q: The paper focuses only on dense retrievers, leaving it unclear how the findings extend to models like ColBERT or retrieval architectures incorporating cross-encoder re-rankers ...

R: The MedCPT retriever used in our paper is actually of the cross-encoder re-rankers architecture tuned on medical data (https://github.com/ncbi/MedCPT). As a result, we think the observations and findings are applicable to the cross-encoder re-rankers. We also include the results of close-sourced text-embedding-3 from OpenAI in the Table below. We observe that the attack success rates are similar to those of the state-of-the-art models. We will include this in the revised version.

Table. Top 2 retrieval success rates with text-embedding-3 as the retriever and MedMCQA, PubMedQA as the query corpus.

Refs: [A] Tian et al., "Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer", NeurIPS 2023.

[2] Xiong et al., "Benchmarking Retrieval-Augmented Generation for Medicine", ACL 2024

[3] Miao et al., "Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications", Medicina (Kaunas). 2024

[4] Zou et al., "PoisonedRAG: Knowledge Poisoning Attacks to Retrieval-Augmented Generation of Large Language Models", USENIX Security, 2025

We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.

Q: Lack of Novelty. The method appears to be simply an integration of a poisoning attack within RAG, where the attacker-defined query consistently results in a high-ranking retrieval of the poisoned document.

R: The novelty of our work lies in the following aspects:

• We systematically investigate and comprehend the robustness of retrieval systems employed in RAG, by introducing a new attack. We achieve this by injecting various types of information, encompassing both irrelevant and relevant content, into the corpus and evaluating the ease or difficulty of retrieval.
• We propose the orthogonal augmentation property to explain the wide-spread success of the attack, and provide empirical evidence to support this property.
• We propose a new defense based on the orthogonal augmentation property, which can effectively filter out the adversarial documents.

Given the relatively new development of safety in RAG, e.g., poisoning attacks, we do believe the above listed contributions are novel and valuable. We will clarify this in the revised version.

Q: Although the paper partly relaxes the assumption by showing robustness under paraphrasing, the attack still relies on knowing the typical structure of medical queries.

R: We have included additional experiments on the legal domain in Table 10 in Section D of the supplementary material. Overall, we observe similar attack success rates in the legal domain as in the medical domain presented in the main text. Therefore, we believe that the proposed attacks can be generalized to a wide range of domain applications. We will clarify this in the revision.

Q: The proposed detection method based on Mahalanobis distance with covariance shrinkage appears effective empirically; however, its performance may be highly sensitive to the choice of the shrinkage parameter (β) and the quality of the anchor set.

R: We have included an ablation study on the selection of $\beta$ in Table 8 in Section C of the supplementary material. We observed that the detection performance remains stable across a wide range of $\beta$ selections. Regarding the anchor set, if the anchor set is non-informative, it becomes theoretically infeasible to implement any effective defenses. We will include more discussions and empirical results in the main text.

We sincerely thank the reviewers for investing their time and effort in reviewing our manuscript and providing valuable feedback. We will address their comments point by point in the following and incorporate them into our revision.

Q: One weakness is that the orthogonal augmentation property relies on the behavior of the embedding function, specifically its approximate linearity under concatenation. Given that Contriever and MedCPT have different sensitivity to document length, the claim might vary with retrieval architecture or training regimes, and does not necessarily support the notion of "universal". The authors select three retrievers based on their general ability / domain design. Does it make sense to have retrievers that have more architectural / training regime variations?

R: The three retrievers used in our paper themselves have different architectures and training regimes. For example, the Contriever is a single-tower model with contrastive learning training regime, i.e., InfoNCE loss, trained on general purpose domain data. While the MedCPT is a cross-encoder re-ranker model with two-stage training (InfoNCE for contrastive learning + cross-entropy for reranker) for medical purposes. We will add results on closed-source models like text-embedding-3 from OpenAI (shown in Table below) in the revised version to further validate the generality of our findings.

Table. Top 2 retrieval success rates with text-embedding-3 as the retriever and MedMCQA, PubMedQA as the query corpus.

Q: The other weakness is that another underlying assumption for the claim is that the concatenated adversarial information is nearly orthogonal to the original query, but that might not be the case and being orthogonal does not necessarily mean the underlying documents are semantically unrelated.

R: We thank you for your extremely sharp comments. We addressed this point in Lines 352 - 362 (left) of the main text. We found that closeness to orthogonality between embeddings does not imply that their associated documents are semantically irrelevant. For example, we randomly sampled two nonoverlapping batches of questions from the MedQA dataset and found that the angle between their embeddings is around $70^\circ$. Yet, these batches of queries are all semantically related to biology research questions. As a result, we believe the stated assumption is reasonable to a certain degree. We will further clarify this in the revised version.

Q: The authors mention that they check the general attack areas / target datasets by using GPT-3 to check their semantic closeness. It could be helpful to include some summary in the appendix?

R: We will include the summary of the semantic closeness check in the appendix as you suggested.

Q: I have reviewed all supplementary materials - the authors mention that the details of the retrievers are in the appendix but I have not found those details.

R: We will include the details of the retrievers in the appendix as you suggested.