SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAG

We appreciate your insightful review and recommendations. We address the remarks in Weaknesses (W) and Questions (Q) below.

---

W1:Dependency on LLMs: EIRE and ssG rely on additional LLM calls, raising concerns about cost, latency, and bias.

Thank you for the feedback. We recognize the reviewer's apprehension about the dependence on LLMs in foundational elements like ssG and EIRE. The need for deeper semantic understanding that conventional symbolic or sparse methods cannot offer drove our design decision to use LLMs for semantic structure extraction and graph similarity. We do agree, though, that this adds overhead. Also, we notice the following: As shown in Section 5.4 and Figure 3, the additional runtime introduced by SeCon-RAG remains within a reasonable range (<1.5 minutes per batch). One query only requires an acceptable cost, which we believe is acceptable for many real-world applications that require robustness.

We also showed in Section 5.3 (Table 2) that SeCon-RAG performs well across various lightweight embedding models (e.g., MiniLM, SimCSE), indicating that the framework is adaptable to lower-cost alternatives. For future work, we are looking into more lightweight techniques to replace EIRE, reducing the reliance on full-scale LLM inference at runtime. We will clarify these points more explicitly in the revised version, as well as include additional discussion of scalability trade-offs in deployment scenarios.

---

 W2:Lack of standalone evaluation for EIRE in ablation study.

We agree that evaluating EIRE's standalone impact is critical for better understanding its contribution to overall performance.To address this, Table (shown below) contains a more detailed ablation analysis in which we compare performance with and without EIRE across PIA and multiple poisoning intensity levels (100%, 20%) and datasets (Hotpot, NQ, MS) use Mistral-12B. The results clearly demonstrate the efficacy of EIRE as a standalone defense component.

The table shows Accuracy (ACC↑) on the left and Attack Success Rate (ASR) on the right. When EIRE is enabled ("Y"), the model consistently achieves higher or comparable accuracy across all settings than when EIRE is turned off ("N"). Attack Success Rate (ASR↓) More importantly, EIRE significantly lowers the attack success rate, particularly in high-poisoning scenarios. For example, under 100% PIA, ASR falls from 16% to 10.2% in one setting and to 0% in others. These findings confirm that EIRE alone makes a significant contribution to the defense mechanism, both in terms of preventing attacks and maintaining model performance. We will clarify this in the revised manuscript and emphasize EIRE's standalone impact in the ablation study section.

---

W3: Threshold sensitivity analysis is missing.

We will add a supplementary plot showing how varying τ_cluster and τ_semantic affects accuracy and ASR. Preliminary results suggest that SeCon-RAG is relatively robust within a reasonable range (±0.1) of threshold values, due to the conservative AND-logic used in joint filtering.

We agree that thresholds are critical in determining the strictness of the Semantic Consistency Filter (SCF). In our experiments, we manually tuned these thresholds on a validation set. Unfortunately, due to space constraints, we did not include a detailed sensitivity analysis in the main paper.

To address this, we conducted a comprehensive threshold sensitivity analysis and included the findings in the supplementary material. Specifically, we varied τ_cluster and τ_semantic independently across a reasonable range and evaluated the impact on both model accuracy (ACC↑) and attack success rate (ASR↓) under different poisoning intensity levels (PIA: 100%, 60%, 20%) on datasets(Hotpot、NQ、MS). The findings show that SeCon-RAG is relatively resistant to small variations in these thresholds, owing to the conservative AND-logic used in joint filtering. Changing τ_cluster from 0.86 to 0.90 maintains accuracy and ASR stability across both LLaMA-3.1-8B and GPT-4o backbones. Similarly, changes in τ_semantic between 0.2 and 0.4 cause only minor fluctuations in performance.

---

W4: Real-world deployment discussion is lacking.

We agree that additional discussion of real-world deployment considerations would improve the paper. In Section 6 (Limitations), we will discuss the trade-off between robustness and latency, particularly for latency-sensitive applications (e.g., real-time QA). And we also want to point out that because of the SeCon-RAG's plug-and-play compatibility with existing RAG systems, the real-world can easily use it to improve the robustness of a realistic system.

---

W5: Font size in tables/figures is too small.

Thank you for this important formatting remark. We acknowledge that the font sizes in Table 1, Table 2, and Figures 1-4 are currently small due to layout constraints. In the camera-ready version, we will separate large tables into sub-tables for each dataset to improve readability. And increase the font size and spacing in all figures and tables, move detailed numerical tables to the appendix, and keep summary plots in the main text.

---

Q2: Semantic distinction between clean and toxic (poisoned) content is unclear.

Thank you for the feedback. The semantic difference between clean and toxic content in our SCF module can be understood from two key perspectives:

First, because poisoned documents frequently contain identical or highly similar content, their embedding distributions tend to cluster tightly together when retrieved. In contrast, clean content has more diverse semantic representations and thus does not form such tight clusters. This phenomena has previously been examined in past work, like as the paper "TrustRAG" which mentioned a similar viewpoint.

Second, we conducted a thorough analysis using semantic graphs created from entities, intents, and relations extracted using EIRE. Figure 2 and Figures 1, 2, and 3 in the Appendix demonstrate that poisoned documents consistently produce sparser and less coherent semantic graphs than clean documents. These structural inconsistencies occur because poisoned content frequently contains misleading or contradictory information that lacks semantic consistency and connectivity. Furthermore, the characteristics of the poisoning attack approach exacerbate this effect.

These findings support our targeted design of the SCF module, which uses semantic graph-based analysis and clustering-based filtering to effectively distinguish toxic from clean content.

We appreciate your insightful review and recommendations. We address the remarks in Weaknesses (W) and Questions (Q) below.

---

W1: Dependency on LLMs and Runtime Overhead.

Thank you for the feedback. We acknowledge that SeCon-RAG uses LLMs in several components (EIRE, semantic similarity scoring, and CAF's final judgment), resulting in a moderate runtime overhead. The need for deeper semantic understanding that conventional symbolic or sparse methods cannot offer drove our design decision to use LLMs for semantic structure extraction and graph similarity. We do agree, though, that this adds overhead. Also, we notice the following:

• A single query incurs only a reasonable computational cost, which we believe is acceptable for many real-world applications that require robustness against retrieval-time attacks—such as business consulting, legal or medical information Q&A, and other high-stakes domains where resistance to poisoning is critical.

• We also showed in Section 5.3 (Table 2) that SeCon-RAG performs well across various lightweight embedding models (e.g., MiniLM, SimCSE), indicating that the framework is adaptable to lower-cost alternatives. For future work, we are looking into more lightweight techniques to replace EIRE, reducing the reliance on full-scale LLM inference at runtime. We will clarify these points more explicitly in the revised version, as well as include additional discussion of scalability trade-offs in deployment scenarios. 

---

W2. Reliance on EIRE Quality.

Thank you for the feedback. You are correct in stating that the quality of EIRE affects downstream modules. The dual-layer design of SCF and CAF helps to reduce this dependency. So we evaluating EIRE's standalone impact is critical for better understanding its contribution to overall performance. Table contains a more detailed ablation analysis in which we compare performance with and without EIRE across PIA and multiple poisoning intensity levels (100%, 20%) and datasets (Hotpot, NQ, MS) use Mistral-12B. The results clearly demonstrate the efficacy of EIRE as a standalone defense component.

The table shows Accuracy (ACC↑) on the left and Attack Success Rate (ASR) on the right. When EIRE is enabled ("Y"), the model consistently achieves higher or comparable accuracy across all settings than when EIRE is turned off ("N"). More importantly, EIRE significantly lowers the ASR, particularly in high-poisoning scenarios. For example, in dataset ms, ASR drops from 5% to 0% when Poisoned is set to 100%.

These findings confirm that EIRE alone makes a significant contribution to the defense mechanism, both in terms of preventing attacks and maintaining model performance.

---

W3. Missing Error Bars.

Thank you for the feedback. We averaged our experimental results across multiple runs, but we did not include confidence intervals due to space constraints. This will be addressed as follows: The revised version now includes standard deviation/error bars in key tables and figures.In the appendix, we report the number of runs as well as details about our random seed and evaluation protocol.

---

Q1: LLMs used within SeCon-RAG: Are they general-purpose or specialized? How is prompting designed?.

Thank you for the feedback. The benchmark RAG model determines whether we use open or closed-source LLMs. When the RAG component is instantiated with model A, the resulting instruction model is also A, eliminating the need for a separate open or closed-source LLM. Our method is effective whether the LLM is accessed via external APIs or deployed locally, as demonstrated by the consistent performance across both settings in Table 2. And prompting was carefully designed for robustness and consistency, as shown in the appendix.

• EIRE extracts entities, intents, and relations using structured prompts (refer to Appendix A.3.1).

• For semantic graph comparison, we ask the LLM to compare a candidate document's semantic graph to reference graphs and return a normalized similarity score (see Appendix A.3.2).

• In CAF, we use explicit semantic criteria to assess trustworthiness across query, corpus, and model dimensions (see Appendix A.4).

---

Q2. Clarification on “Supervise” in Figure 1 for EIRE

Thank you for noticing the ambiguity. The "Supervise" label in Figure 1 indicates that EIRE guides both SCF and CAF.

• In SCF, EIRE structures a semantic graph and compares it to a reference set.
• In CAF, EIRE's outputs serve as the semantic foundation for consistency checks.

The term "supervise" does not refer to supervised learning, but rather semantic guidance. We will change the figure caption and legend to avoid confusion.

---

Q3. How were τ_cluster and τ_semantic determined? Sensitivity?

Thank you for the feedback. For example, the value of τ_cluster is set to 0.88, which was empirically determined based on our own experimental results.. In contrast, τ_semantic is determined using the semantic similarity scores we created from predefined prompt words (as shown in the appendix). Notably, τsemantic is set independently and does not rely on a validation set.

Specifically, we varied τ_cluster and τ_semantic independently across a reasonable range and evaluated the impact on both model accuracy (ACC↑) and attack success rate (ASR↓) under different poisoning intensity levels (PIA: 100%, 60%, 20%) on datasets(Hotpot、NQ、MS). The findings show that SeCon-RAG is relatively resistant to small variations in these thresholds, owing to the conservative AND-logic used in joint filtering. Changing τ_cluster from 0.86 to 0.90 maintains accuracy and ASR stability across both LLaMA-3.1-8B and GPT-4o backbones. Similarly, changes in τ_semantic between 0.2 and 0.4 cause only minor fluctuations in performance.

---

Q4. Composition and size of Dcor (verified correct documents)

Thank you for the feedback. The Dcor subset is a small collection of documents that have been manually verified as relevant and accurate to the query. These documents were human-verified from datasets to ensure high semantic relevance and correctness. Dcor guides the semantic filtering and CAF process, especially in the construction of the semantic graph used to identify poisoned or inconsistent documents. Its primary function is to provide a reliable semantic anchor for calculating similarity scores.

To assess the impact of Dcor, we conducted ablation experiments that compared performance with and without this Dcor subset. As shown in the table below (with and without cor), enabling Dcor ("Y") consistently improves accuracy or reduces ASR across PIA and different poisoning intensities (100%, 20%) on three datasets (Hotpot 、NQ、MS) .

For example, when Dcor is used at 100% poisoning intensity in MS, the ASR falls from 5% to 0% and accuracy improves from 85 to 88. These findings show that even a small, high-quality Dcor set can significantly improve semantic filtering performance by anchoring the semantic space and reducing noise from poisoned documents. In conclusion, while Dcor is small in size, its careful curation adds significantly to the robustness of our filtering mechanism.

---

Q5. How is the base LLM’s “internal knowledge” used for model consistency in CAF?

---

Thank you for the feedback. In CAF's model consistency check, we ask the LLM to determine whether a document's semantic content corresponds to its internal knowledge. At this stage, reinforcement learning was not used, nor was its bias towards knowledge reinforced.

Importantly, the LLM is expected to reason based on extracted entities and relations, not memorized facts. We reduce hallucination risk by comparing multiple retrieved documents and relying on consistency across sources.Only documents marked "trustable" across all dimensions are used in the final generation, including those that support or confirm the LLM's own prior knowledge.

We sincerely thank the reviewer for the thoughtful and constructive comments. We address the remarks in Weaknesses (W) and Questions (Q) below.

---

W1. Heuristic Nature and Lack of Motivation for Design Choices

---

Thank you for this feedback. Previously approaches for removing poisoned documents relied on filtering or voting, which disregarded semantic information. And we feel that the semantic information contained in papers is particularly useful for LLM. As a result, we decided to develop filtering methods from a semantic standpoint in order to improve the whole system's robustness and resistance to attacks.

More specifically, the use of clustering and semantic graph filtering in SCF is motivated by the observation that poisoned documents cluster frequently (captured via embeddings) while lacking coherent semantic structure (captured via EIRE). CAF ensures factual consistency from semantics by comparing retrieved documents to both the query and the LLM's own knowledge, making the final response more reliable.

---

W2. Technical Clarity

Thank you for this feedback. We recognize that some of the core mechanisms, particularly how semantic structure graphs are constructed and evaluated, could benefit from more explicit explanation. In the revised version, we will:

Include a step-by-step example of EIRE's output and how semantic graphs are created (see also Appendix A.3.1). Provide a more understandable figure that shows the difference between clean and poisoned documents in the semantic and embedding space. Extend Sections 4.2.2 and 4.3 to explain the reasoning behind using semantic similarity as a filtering signal. These additions will assist readers in understanding not only how the method works, but also why it is effective.

---

W3. Lack of Formal Complexity Analysis

The first stage of our proposed defense framework is the filtering phase, which includes cluster-based filtering on the retrieval database and semantic filtering on retrieved candidates. Before final inference, the Consistency-Aware Filtering (CAF) module performs additional checks on both the retrieved documents and the inference output. The pseudocode in the appendix provides a more detailed implementation and complexity analysis. In terms of complexity, the retrieval and inference processes are executed only once, whereas the additional filtering steps require two lightweight detection operations. This design ensures that overall computational overhead is minimal. Figure 3 illustrates a specific time comparison.

---

W4. Heavy Reliance on Prompting and LLM Capabilities

Thank you for the feedback. We recognize the reviewer's apprehension about the dependence on LLMs in foundational elements like ssG and EIRE. Our design decision is based on the observation that LLMs are currently the most effective tools for capturing deep semantic structures in natural language which traditional symbolic or sparse methods struggle to accomplish. Even when using smaller or less powerful models, the semantic signals extracted are sufficiently informative to allow our framework to function properly.

We also showed in Section 5.3 (Table 2) that SeCon-RAG performs well across various lightweight embedding models (e.g., MiniLM, SimCSE), indicating that the framework is adaptable to lower-cost alternatives. For future work, we are looking into more lightweight techniques to replace EIRE, reducing the reliance on full-scale LLM inference at runtime. We will clarify these points more explicitly in the revised version, as well as include additional discussion of scalability trade-offs in deployment scenarios.

---

Q1. Line 168 – Clustering of malicious documents in embedding space

Thank you for your feedback. Acturally Clustering of malicious documents in embedding space may not be universally applicable across all attack types, but in this paper, we will focus on the poisoned RAG, which will result in Clustering of malicious documents in embedding space. Furthermore, because poisoned documents frequently contain identical or very similar content, their embedding distributions tend to cluster tightly together when retrieved. Clean content, on the other hand, has a broader range of semantic representations and thus does not cluster as tightly. This phenomena has previously been examined in past work, like as the paper "TrustRAG" which mentioned a similar viewpoint. Furthermore, the characteristics of the poisoning attack strategy amplify this effect. We will include a visualization (PCA) in the appendix to back up this point.

---

Q2. Line 184 – Sparse or fragmented semantic graphs

Thank you for your feedback. As shown in Figure 2, correct documents generate densely connected semantic graphs, whereas adversarial documents produce noticeably sparser structures. We use embedding model to project documents into vector space, and the resulting semantic graphs for both correct and incorrect documents are shown in Appendix Figures 1–3.

This difference in semantic graph structure stems from the inherent characteristics of common attack methods. Although an adversary could create a poisoned document by simply flipping an edge or renaming an entity in Figure 2(a), such changes often reduce the attack's effectiveness or increase its cost. As a result, our defense approach, which takes advantage of these semantic and structural differences, proves particularly effective in such attack scenarios.

---

Q3. Line 192 – Source of verified documents

Thank you for this feedback. The "Verifying Correct Documents" (D_cor) used for semantic reference are a small set of samples chosen manually from the dataset. These documents are only intended to provide the large model with accurate semantic structures for reference. To assess the impact of Dcor, we conducted ablation experiments that compared performance with and without this Dcor subset. As shown in the table below (with and without cor), enabling Dcor ("Y") consistently improves accuracy or reduces ASR across PIA and different poisoning intensities (100%, 20%) on three datasets (Hotpot 、NQ、MS) .

We will include representative examples in the appendix and investigate the effects of including or excluding this candidate set on the experimental results.

---

Q4. Equation (5) and Graph Input to LLM

Thank you for this feedback. We appreciate the reviewer's observation that the description in lines 178-181 and Equation (5) is unclear. To clarify, the input to the large language model is natural language not raw numerical vectors. The compressed vectors are used solely for visualization and relationship construction purposes, but are not directly fed into the LLM. Instead, we use EIRE outputs (e.g., structured triplets, triplets like "Entity A—[Relation R]→ Entity B") to construct semantic graphs using LLM. The LLM then uses these serialized representations to compare candidate and reference documents.We will update Equation (5) and the appendix text to reflect this process and avoid confusion about LLM input types.

Thank you for your comment. Different from previous work，our defense framework incorporating structured semantics into both the retrieval filtering and response generation stages of RAG. To our knowledge, this is the first paper to explicitly use semantic-level understanding via entity-intent-relation graphs and LLM-based validation to defend against poisoned attacks in RAG.

This structured semantic filtering fills a critical gap in previous research, in which semantics were ignored in both the filtering and inference stages of RAG defense. Unlike clustering-based defense methods, our semantic filter enables fine-grained detection of semantically inconsistent or poisoned content, thereby improving RAG's robustness. 

Our ablation results (below) show that the proposed semantic filter, combined with clustering filter, significantly reduces attack success rates while improving answer accuracy. Specifically, our hybrid method, Semantic-Clustering Filtering (SCF), is more robust than either component alone, demonstrating the complementary strengths of clustering and semantic reasoning.

Dear Reviewer N6z7，

Thank you for reading our response and submitting the acknowledgement. We would greatly appreciate it if you could also let us know whether our response has addressed your concerns. If so, we kindly ask you to consider increasing the score accordingly, if you haven’t already done so. Otherwise, if any questions or uncertainties remain, we would be more than happy to provide further clarification.

Best, Authors

We sincerely thank the reviewer for the thoughtful and constructive comments. We address the remarks in Weaknesses (W) and Questions (Q) below.

---

W1. The explanation of the principles behind some key steps in this work's methodology is unclear, making it somewhat difficult to understand why this method is effective.

---

Thank you for this feedback. We clarify that the success of our strategy is derived from a two-stage semantic filtering framework: (1) To conservatively filter poisoned documents, SCF use both clustering and semantic graph matching (via output from our EIRE module); (2) CAF further filters retrieved content by ensuring semantic coherence between the query, retrieved documents, and the model’s internal knowledge. This layered design ensures robustness while minimizing information loss. We have offered clarifications and examples in the appendix to assist the reader in understanding.

---

W2. Lack of ablation for the necessity of combining both SCF components

Thank you for this feedback. We agree that a more explicit evaluation of SCF's two subcomponents—the clustering-based and semantic graph-based filtering modules—is beneficial. While Figure 4 in Section 5.5 already shows an ablation study comparing SCF and CAF as a whole, it does not separate the contributions of the two SCF submodules. To address this, we carried out additional experiments, evaluating each subcomponent independently. The findings are summarized in the table below.

Preliminary results show that combining both filters provides the best overall balance of clean accuracy and attack suppression. Specifically, while each module provides moderate improvements on its own, their combination results in significantly increased robustness (e.g., 0% ASR in several settings) with minimal effect on clean performance. This demonstrates the complementary nature of clustering and semantic-based filtering. These ablation results will be included in the revised appendix to support the empirical design choices underlying SCF.

---

W3. Inference latency may limit real-world applicability

Thank you for the feedback. We acknowledge that SeCon-RAG uses LLMs in several components (EIRE, semantic similarity scoring, and CAF's final judgment), resulting in a moderate runtime overhead. The need for deeper semantic understanding that conventional symbolic or sparse methods cannot offer drove our design decision to use LLMs for semantic structure extraction and graph similarity. We do agree, though, that this adds overhead. Also, we notice the following:

• As shown in Section 5.4 and Figure 3, the additional runtime introduced by SeCon-RAG remains within a reasonable range (<1.5 minutes per batch). One query only requires an acceptable cost, which we believe is acceptable for many real-world applications that require robustness.

• We also showed in Section 5.3 (Table 2) that SeCon-RAG performs well across various lightweight embedding models (e.g., MiniLM, SimCSE), indicating that the framework is adaptable to lower-cost alternatives. For future work, we are looking into more lightweight techniques to replace EIRE, reducing the reliance on full-scale LLM inference at runtime. We will clarify these points more explicitly in the revised version, as well as include additional discussion of scalability trade-offs in deployment scenarios.

---

Q1. Basis for the claim: "malicious documents cluster in embedding space" (line 168)

Thank you for the feedback. This insight is based on empirical observations that adversarially generated poisoning documents frequently use highly similar phrasing or templated structures, particularly when targeting the same query. Due to their similarities, they form tight clusters in the embedding space. This phenomena has previously been examined in past work, like as the paper "TrustRAG" which mentioned a similar viewpoint.

---

Q2. Reasoning behind: "poisoning documents generate sparse or fragmented graphs" (line 188)

Thank you for the feedback. This phenomenon arises for two reasons.

• On the one hand, it is owing to the attack mechanism described in the paper, "poisoning RAG" settings, which require a lower cost to attack.
• Poisonous content, on the other hand, frequently introduces isolated or deceptive claims that are not adequately supported by the surrounding semantic context. When EIRE extracts entities and their relationships from such documents, the resulting semantic graphs frequently include abrupt or unnatural relationships, as well as isolated nodes or disconnected subgraphs. This behavior differs from clean documents, which have well-structured, coherent semantic graphs. We demonstrate this qualitatively (Figure 2) and quantitatively (semantic similarity scores).

We will elaborate on this reasoning and provide additional examples in the appendix.

---

Q3. Is there experimental support for semantic graph similarity filtering? (line 194)

Thank you for the feedback. To assess the impact of "candidate document and verified correct documents" (D_cor) , we conducted ablation experiments that compared performance with and without this Dcor subset. As shown in the table below (with and without cor), enabling Dcor ("Y") consistently improves accuracy or reduces ASR across PIA and different poisoning intensities (100%, 20%) on three datasets (Hotpot 、NQ、MS) . 

We will include representative examples in the appendix and investigate the effects of including or excluding this candidate set on the experimental results.