Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

W1. Benchmarking large language models in retrieval-augmented generation. AAAI 24

We have acknowledged the contribution of previous work, including the AAAI '24 paper (lines 40 and 303). In fact, we have evaluated our method on the RGB benchmark introduced in that paper (see Figure 4). We find that, when the noise ratio is 1.0 (the worst-case where all passages are negative), all baseline RAG methods fall behind No RAG. Our method, Astute RAG, is the only RAG method that reaches a performance close to No RAG under the worst-case scenario, further supporting its effectiveness in addressing imperfect retrieval augmentation.

While prior works have identified the issues of retrieval and knowledge conflicts, we are the first to provide a comprehensive and quantitative analysis connecting imperfect retrieval, knowledge conflicts, and RAG failures in real-world scenarios. In contrast, RGB relies on a synthetic noise ratio. Notably, the AAAI '24 paper does not propose solutions to these problems.

W2. There are many previous studies. the authors should explain the differences/advantages of their methods over these studies.

We would be glad to include any overlooked related work if the reviewer could provide specific references.

Currently, we have discussed relevant prior studies in the related work section. The key differences are:

• We provide a comprehensive analysis connecting imperfect retrieval, knowledge conflicts, and RAG failures, going beyond previous analysis on isolated problems.
• Building upon the insights from the analyses, our method encompasses a broader framework to address these challenges comprehensively under a training-free and black-box setting
• Our method effectively consolidates knowledge from both internal and external sources and achieves more reliable answers with little additional cost.

W3. the baselines in this paper are all general RAG baselines

The baselines in this paper include those specifically designed to address noise and conflict issues in RAG, such as RobustRAG and InstructRAG. Additionally, Self-Route incorporates confidence awareness. These recent baselines cover the types specified by the reviewer. Below is a brief description of them:

• RobustRAG (Xiang et al., 2024) aggregates answers from each independent passage, using the majority consensus to prevent the negative influence of malicious passages.
• InstructRAG (Wei et al., 2024) instructs the LLM to provide a rationale to identify misleading or even erroneous information in the retrieved contents.
• Self-Route (Li et al., 2024) adaptively switches between LLMs with and without RAG based on the model confidence.

We are happy to add more baselines if the reviewer can specify which ones they have in mind.

Dear Reviewer VS5X,

As the rebuttal deadline approaches, please kindly check the papers' discussion threads and respond to the authors' rebuttals. If you haven't had a chance to respond yet, I’d greatly appreciate your input soon. Your insights are invaluable to the authors and the review process.

Thank you for your effort and support!

Best regards,

Area chair

W2. There are many previous studies. the authors should explain the differences/advantages of their methods over these studies.

We sincerely thank the reviewer for the insightful feedback on improving the literature review. Below, we provide a more detailed discussion of related work referenced in our paper to clarify our positioning and highlight distinctions from prior research. We would be glad to include any overlooked related work if the reviewer could provide specific references.

Overview of Related Work

The negative impact of noisy information within retrieved passages has been increasingly scrutinized. Several orthogonal research directions address this issue at the retrieval stage of RAG, as outlined below:

• Dynamic Retrieval: Iterative retrieval (Asai et al., 2023; Yan et al., 2024) retrieves additional information as needed during the generation process.
• Passage Reranking: Techniques like those by Glass et al. (2022) and Yu et al. (2024) enhance robustness by refining retrieval quality through reranking.
• Query Enhancement: Query rewriting methods, e.g., Ma et al. (2023), focus on dynamically refining input queries to improve retrieval outcomes.
• Speculative Drafting: Wang et al. (2024) propose generating intermediate outputs to guide and refine the retrieval results.
• Training Passage Filters: Approaches such as Wang et al. (2023c) train smaller models to identify and filter out unhelpful passages.

These efforts primarily focus on improving retrieval and preprocessing stages to optimize the quality of passages fed into LLMs. While these methods enhance RAG performance, they address different research questions and are complementary to our work.

Positioning of Our Work

We tackle challenges at the post-retrieval stage, operating under the assumption that retrieved passages are already preprocessed and provided. This stage is subsequent to the methods listed above, making our approach complementary and distinct.

Closely Related Work

Several recent approaches address the noisy retrieval result issues in this post-retrieval setting are compared as baselines:

• GenRead (Yu et al., 2023): Enhances retrieved passages by incorporating LLM-generated passages.
• RobustRAG (Xiang et al., 2024): Aggregates answers from independent passages to ensure robustness, though it does not resolve conflicts using internal knowledge.
• InstructRAG (Wei et al., 2024): Focuses on generating rationales that identify misleading or incorrect passages.
• Self-Route (Xu et al., 2024a): Dynamically switches between LLMs with and without RAG based on model confidence of retrieved passages.

Less Directly Comparable Work

Other works in related domains make additional assumptions or address distinct settings:

• Training LLMs with Noisy Contexts (Fang et al., 2024; Pan et al., 2024): Trains LLMs to directly handle noisy inputs, focusing on improving model robustness via supervised training.
• Contrastive Decoding (Jin et al., 2024; Zhao et al., 2024): Assumes a setup where model logits are accessible, which is not applicable to most black-box LLMs.
• Matching Generated and Retrieved Passages (Zhang et al., 2023): Involves training models to match retrieval outputs with generated content.

Summary

Our work uniquely focuses on the post-retrieval stage, addressing challenges that arise after preprocessing retrieved passages. By situating our method as the next step in the RAG pipeline, we build on and complement existing approaches.
Currently, we have discussed relevant prior studies in the related work section. The key differences are:

• We provide a comprehensive analysis connecting imperfect retrieval, knowledge conflicts, and RAG failures, going beyond previous analysis on isolated problems.
• Building upon the insights from the analyses, our method encompasses a broader framework to address these challenges comprehensively under a training-free and black-box setting
• Our method effectively consolidates knowledge from both internal and external sources and achieves more reliable answers with little additional cost.

We hope this expanded discussion adequately addresses the reviewer’s concerns about the literature review. Please let us know if further elaboration is needed.

Dear Reviewer VS5X,

Thank you for your valuable feedback on our paper. We have incorporated your suggestions and made the following updates to the pdf:

• Rephrased Contributions in a Broader Context: Revised lines 46–50, 520–526, and updated Figure 1 to better situate our contributions. The "Closely Related Work" are introduced and compared with in lines 310-342, the "Overview of Related Work" are introduced in lines 472-483, and the "Less Directly Comparable Work" are discussed in lines 483-526. Methodologically, our work provides the following novel insights/benefits:
  • Adaptive Generation: The internal knowledge of LLMs is not equally accessible to all queries. Leveraging adaptive generation improves overall performance.
  • Iterative Source-Aware Consolidation: Explicitly and iteratively instructing the model to consolidate internal and external knowledge in a source-aware manner enhances results. This approach differs from prior work, which typically resolves knowledge conflicts through methods such as pairwise matching or pre-defined switching rules.
  • Train-Free and Efficient: Our method is train-free, applicable to black-box models, and achieves an 11% relative performance improvement with only a 5% increase in token cost, demonstrating significantly higher efficiency compared to baselines.
• Comprehensive Analysis of Advantages: Provided a detailed comparison of AstuteRAG with baselines, focusing on adaptive generation, efficiency, and robustness to ordering strategies (lines 422–462, Tables 5, 6, and 7).
• Expanded Verification of AstuteRAG's Effectiveness: Tested its performance on more open-source models of varying sizes (Table 4).
• Demonstrated Generalizability: Evaluated AstuteRAG on long-form QA tasks, with results summarized in Table 2 and lines 400–402.

As the due for updating the PDF is approaching, we are eager to know if there are any remaining concerns or areas for improvement. We are eager to further enhance our comparisons with prior work. If the reviewer can provide additional references or suggest specific works to include, we would be happy to incorporate them. Your guidance will be invaluable as we finalize the revisions.

Q1. compare system performance with varying numbers of retrieved documents

We report below the results using 5 and 10 retrieved passages, respectively. Astute RAG demonstrates consistent performance across these configurations, showcasing its robustness and adaptability to varying context lengths.

Q2.1. how these refusal cases were handled in the results presented in Table 1?

Since the questions are answerable, refusals are counted as incorrect answers. This setting aligns with real-world helpfulness evaluation, where failing to provide an answer is considered a failure to assist.

Q2.2. provide specific statistics on the frequency of LLM refusals

To assess the refusal rate, we count the ratio of responses containing a clear "no" or "not" in the final answer. We observe that there is no direct correlation between model performance and refusal rate. A method with better overall performance may also be more effective at identifying when additional information is needed, while a weaker method could either refuse more often or make more incorrect guesses.

Q3. Please elaborate on the potential role of reranking

We apply different ordering strategies introduced by [1], on RAG and our AstuteRAG. We find that the improvement with AstuteRAG is significantly larger than the gap between different ordering strategies. Moreover, the consolidation process makes Astute RAG less sensitive to the order of passages.

[1] Alessio et al. 2024 Improving RAG Systems via Sentence Clustering and Reordering

W1. This process is rather time-consuming and consumes tokens.

We present the average number of tokens used per instance for Claude on our benchmark. Given that inference cost scales with the number of tokens, Astute RAG incurs only a marginal cost increase of less than 5% while delivering a substantial relative improvement of over 11% compared to the RAG baseline.

In addition, the efficiency of Astute RAG can be controlled by adjusting the consolidation iteration (t) and the max number of generated passages (m). Notably, even when both t and m are set to 1, the improvement is already significant.

W2. whether the confidence score of LLM shows an obvious bias when there is a conflict between internal and external knowledge?

While our method relies on the LLM to assign confidence scores, we aim to mitigate this issue by explicitly consolidating potentially conflicting internal and external knowledge. This step helps reduce the direct influence of bias. As demonstrated in Figure 7, our method successfully generates the correct answer when either side is correct in these cases. The quantitative results in Figure 6 further show that our method significantly improves model performance on conflicting sets, which require the model to balance both sides effectively.

However, we acknowledge that inherent biases in LLMs cannot be entirely eliminated, as they are intrinsic to the model's training and capabilities. The confidence scores generated by the LLM could indeed reflect biases related to token distribution, a known challenge in generative models [1,2]. Future work can explore robust training techniques to address these biases and improve the reliability of confidence assignments.

[1] Xie, Jian, et al. "Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts." arXiv preprint arXiv:2305.13300 (2023).

[2] Jin, Zhuoran, et al. "Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models." arXiv preprint arXiv:2402.14409 (2024).

W3/Q1. lacking analysis of some scenarios. evaluate whether the findings can be generalized to other formats,(e.g. Long-form QA)

We have conducted additional experiments on the long-form QA dataset, ASQA, following the setting in [1,2]. The results demonstrate that Astute RAG consistently achieves significant improvements in this new task, reinforcing its effectiveness across diverse scenarios.

[1] Wei, Zhepei, Wei-Lin Chen, and Yu Meng. "InstructRAG: Instructing Retrieval-Augmented Generation with Explicit Denoising." arXiv preprint arXiv:2406.13629 (2024).

[2] Asai, Akari, et al. "Self-rag: Learning to retrieve, generate, and critique through self-reflection." arXiv preprint arXiv:2310.11511 (2023)

W2 (cont) the knowledge consolidation process is fully completed by LLM. whether the confidence score of LLM shows an obvious bias

To further investigate whether the intermediate steps, especially knowledge consolidation and confidence assignment, exhibits obvious bias, we have evaluated the intermediate stages of knowledge consolidation and confidence assignment using LLM-as-a-judge with the following instruction:

**Task:** You are provided with the following:  
1. A question.  
2. The correct answer.  
3. The input context.  
4. The model's response, which contains:  
   - Consolidated context.  
   - Confidence scores for candidate answers.  

Your task is to:  
- Evaluate the **quality of the consolidated context** in the model's response and provide a label: `<consolidation> correct </consolidation>` or `<consolidation> incorrect </consolidation>`. This evaluation is only about whether the consolidation is correct given the input context.  
- Evaluate the **accuracy of the confidence score** (whether it aligns with the confidence of the supporting context) and provide a label: `<confidence> correct </confidence>` or `<confidence> incorrect </confidence>`. The evaluation is only based on the consolidated context.  

Note that correct consolidation and confidence do not necessarily indicate the correct answer.

Question: 
{query}

Correct Answer: 
{answer}

Input Context: 
{input}

Model Response: 
{response}

Evaluation:

Our experimental results show that the accuracy for knowledge consolidation is 98.2%, and for confidence assignment, it is 95.0%. These results demonstrate the effectiveness of the proposed framework in the intermediate stages. It also indicates that the current prediction errors are mainly due to the knowledge gaps instead of propagation of error from each step in our framework.

W1. I strongly recommend that the authors include experiments using more open-source models.

We further conducted experiments using two open-source LLMs of different scales. The results remain consistent across both models. Specifically, for the more capable LLM, AstuteRAG significantly outperforms the baselines. For the relatively smaller model, AstuteRAG still demonstrates superior performance compared to the baselines on most datasets.

W2.1. the design of adaptive generation is not effectively reflected in the experiments

The results below illustrate the model's performance when varying the maximum number of passages generated. The design of adaptive generation has been effectively reflected, as with the default setting (m=1), the model is already able to dynamically change the number of generated passages.

W2.2. the authors should focus on comparing the total number of API tokens used

We present the average number of tokens used per instance for Claude on our benchmark. Given that inference cost scales with the number of tokens, Astute RAG incurs only a marginal cost increase of less than 5% while delivering a substantial relative improvement of over 11% compared to the RAG baseline.

W3. similar ideas have appeared in some prior works (e..g, [1])

We have acknowledged the contribution of [1] in the related work section (line 484). The key differences are as follows:

• We provide an in-depth analysis linking imperfect retrieval, knowledge conflicts, and RAG failures.
• Our study specifically focuses on the imperfect context setting, where the majority of passages may be incorrect.
• Our method is training-free and applicable to black-box LLMs, offering broader usability and adaptability.

[1] Merging Generated and Retrieved Knowledge for Open-Domain QA

Dear Reviewer axAR,

As the rebuttal deadline approaches, please kindly check the papers' discussion threads and respond to the authors' rebuttals. If you haven't had a chance to respond yet, I’d greatly appreciate your input soon. Your insights are invaluable to the authors and the review process.

Thank you for your effort and support!

Best regards,

Area chair

W1 (cont) the authors do not provide sufficient experimental evidence regarding the performance of current models in these intermediate stages

To address your concern, we have evaluated the intermediate stages of knowledge consolidation and confidence assignment using LLM-as-a-judge with the following instruction:

**Task:** You are provided with the following:  
1. A question.  
2. The correct answer.  
3. The input context.  
4. The model's response, which contains:  
   - Consolidated context.  
   - Confidence scores for candidate answers.  

Your task is to:  
- Evaluate the **quality of the consolidated context** in the model's response and provide a label: `<consolidation> correct </consolidation>` or `<consolidation> incorrect </consolidation>`. This evaluation is only about whether the consolidation is correct given the input context.  
- Evaluate the **accuracy of the confidence score** (whether it aligns with the confidence of the supporting context) and provide a label: `<confidence> correct </confidence>` or `<confidence> incorrect </confidence>`. The evaluation is only based on the consolidated context.  

Note that correct consolidation and confidence do not necessarily indicate the correct answer.

Question: 
{query}

Correct Answer: 
{answer}

Input Context: 
{input}

Model Response: 
{response}

Evaluation:

Our experimental results show that the accuracy for knowledge consolidation is 98.2%, and for confidence assignment, it is 95.0%. These results demonstrate the effectiveness of the proposed framework in the intermediate stages. It also indicates that the current prediction errors are mainly due to the knowledge gaps instead of propagation of error from each step in our framework.

W3. similar ideas have appeared in some prior works (e..g, [1])

In the previous response, we provided a comparison based on research focus. Here, we delve into the differences in terms of technical novelty.

Methodologically, our work provides the following novel insights/benefits:

• Adaptive Generation: The internal knowledge of LLMs is not equally accessible to all queries. Leveraging adaptive generation improves overall performance. In contrast, [1] generates a fixed number of passages.
• Iterative Source-Aware Consolidation: Explicitly and iteratively instructing the model to consolidate internal and external knowledge in a source-aware manner enhances results. In contrast, [1] relies on a finetuned passage matcher and does not explicitly or iteratively refine the context.
• Train-Free and Efficient: Our method is train-free, applicable to black-box models, and achieves an 11% relative performance improvement with only a 5% increase in token cost, demonstrating significantly higher efficiency compared to baselines. In contrast, [1] need training and is not applicable to black-box models.

We have made this clear in the updated paper pdf (lines 520–526).

Dear reviewer axAR,

We have updated the manuscript to address all the key points raised, including:

• Inclusion of more open-source models: Results on Mistral-Large (128B) and Mistral-Nemo (12B) are provided in Table 4.
• Analysis of intermediate processes: Lines 443-451 present evaluations of the accuracy of intermediate processes.
• Effect of adaptive generation: Lines 437-442 and Table 5 show a performance comparison with varying maximum numbers of generated passages.
• Comparison of API tokens: Lines 452-455 and Table 6 demonstrate that AstuteRAG achieves an 11% relative performance improvement with less than a 5% increase in token cost.
• Novelty comparison with [1]: Lines 520-526 introduce [1] and clearly discuss the differences.

We believe that all the issues raised by the reviewer have been thoroughly addressed. If further revisions are required, we would greatly appreciate it if the reviewer could specify the requested changes.

Q1.1. Could you elaborate on your choice and implementation of Google Search as the retrieval method?

The details have been provided in the appendix B (lines 775-779). We will make them clearer in the updated version.

Q1.2. Were there specific criteria for determining which snippets were included in the 10 selected passages?

The snippets are returned by the Google Search API. We extract the paragraph including the snippet for each of top 10 retrieved results.

Q1.3. What were the specific reasons for choosing Google Search over other commercial search engines (e.g., Bing)?

We chose Google Search as it is one of the most widely used search engine, making it an ideal testbed for our evaluation.

Q1.4. How did you handle sponsored or advertised content in the search results?

The API does not return sponsored or advertised content [1].

[1] https://support.google.com/adsense/thread/14297155/is-the-custom-search-api-response-contain-ads?hl=en

Q2.1. How might the results differ with other retrieval approaches?

We emphasize that for quantitative analysis, the critical factor is not the choice of retriever itself but the retrieval quality it delivers. Retrieval quality, partially determined by the retrieval approach, directly impacts RAG performance. Analyzing performance based on retrieval quality provides a more rigorous evaluation. To demonstrate the method's generalizability across varying retrieval qualities, we report performance across different retrieval precision levels, as shown in Figure 5.

Q2.2. How sensitive is the method to different prompt templates?

During development, we did not observe significant performance changes for slightly different expressions.

Q2.3. What is the impact of different parameter settings (e.g., number of iterations, maximum generated passages) on performance and stability?

We have presented the results across varying numbers of iterations (t) in Tables 1 and 2. For reference, we include a subset of these results below.

We also conduct experiments to evaluate the impact of the maximum number of generated passages. The corresponding results are included below for reference.

W1. Recent RAG studies has shown significant differences between distinct arrangement strategies (e.g. Alessio et al. 2024)

We want to clarify that these research directions are orthogonal to the focus of our work. RAG systems consist of multiple components, and users can always integrate complementary techniques (e.g., inference algorithms and ordering strategies) to further enhance performance. As stated in line 79, our work specifically focuses on combining internal and external knowledge during LLM inference, assuming the external context is already provided. We adopt the current default settings because descending order has been identified as the best practice by prior work, including Alessio et al., 2024.

Nevertheless, we agree with the reviewer that additional experiments on more ordering strategies will provide further insights. Per the reviewer’s request, we evaluate different ordering strategies introduced by Alessio et al., 2024, on RAG and our AstuteRAG. We find that the improvement with AstuteRAG is significantly larger than the gap between different ordering strategies.

W2/Q3. The focus on short-form QA tasks limits understanding of the method's broader applicability

We conduct additional experiments on the long-form QA dataset, ASQA, following the setting in [1,2]. The results demonstrate that Astute RAG consistently achieves significant improvements in this new task, reinforcing its effectiveness across diverse scenarios.

[1] Wei, Zhepei, Wei-Lin Chen, and Yu Meng. "InstructRAG: Instructing Retrieval-Augmented Generation with Explicit Denoising." arXiv preprint arXiv:2406.13629 (2024).

[2] Asai, Akari, et al. "Self-rag: Learning to retrieve, generate, and critique through self-reflection." arXiv preprint arXiv:2310.11511 (2023).

W3. Could you discuss strategies for adapting Astute RAG to resource-constrained environments?

We assume that the LLM possesses standard instruction-following capabilities, which is a common assumption when leveraging LLMs for QA tasks. It is unlikely that one would depend on answers from a weak model incapable of comprehending the question. To address concerns about the resource requirements of LLMs, we further conducted experiments using smaller, open-source LLMs, demonstrating the generalizability of Astute RAG to more resource-constrained environments.

Dear Reviewer TZiG,

As the rebuttal deadline approaches, please kindly check the papers' discussion threads and respond to the authors' rebuttals. If you haven't had a chance to respond yet, I’d greatly appreciate your input soon. Your insights are invaluable to the authors and the review process.

Thank you for your effort and support!

Best regards,

Area chair

We have provided detailed responses to address the concerns and questions raised by each reviewer. Below is a concise summary of the key updates and clarifications:

• Reviewer TZiG: The main requests were to add results on different ordering strategies, long-form QA, and smaller LLMs. These have been addressed in Table 7, Table 2, and Table 4, respectively. Additional clarification questions have been thoroughly answered in a Q&A format.

• Reviewer axAR: The primary requests included adding open-source models, analyzing adaptive generation, examining intermediate steps, detailing the cost of API tokens, and comparing with prior work. We have addressed these in Table 4, Table 5, lines 443–450, Table 6, and Section 5.

• Reviewer 4FQF: The main requests focused on the cost of API tokens, analysis of intermediate steps, and long-form QA. These are now covered in Table 6, lines 443–450, and Table 2. Additional questions about the number of retrieved documents, refusal rates, and the role of passage ranking have also been clarified with supplementary results.

• Reviewer VS5X: The primary concern regarding baselines and comparison with prior work has been thoroughly clarified in the responses.