How Much Can RAG Help the Reasoning of LLM?

W5: This paper lacks an experimental setting to evaluate whether the model can reason effectively using only noisy documents in the context.

Answer: Thanks for your suggestion, we have conducted further experiments when the context contains only noisy documents. Below we show that how does the model performs without any documents, and we show that how will the model perform when 3 additional noise documents are added. query only means that no documents are included, and ordered RAG means with noise documents and query is placed after documents while reverse RAG means that query is placed both ahead and after the documents.

We can observe that the noise documents greatly impacts the performance of LLM, but place the query ahead can greatly help.

W6: While the paper emphasizes noise filtering, it lacks discussion of related noise filtering works, such as Wang et al.'s "Learning to filter context for retrieval-augmented generation" or the re-rank-based filter method by Glass et al. in "Re2G: Retrieve, rerank, generate."

Answer: Thanks for your suggestion, we have added more related work in Appendix E of the revision

W7: In line 740, "k" appears to be a typo and should actually be "n".

Answer: Thanks for noticing that, we have fixed that in the revision

Q1: In your modeling, you describe the role of RAG as reducing the depth of reasoning required for a given question probabilistically. Can it cover the conditions under which RAG mitigates hallucinations or corrects reasoning errors(where it may increase reasoning depth but help guide the model to the correct solution path)?

Answer: In our opinion, given a task, the reasoning tree to solve the task is fixed (ignore those situations that multiple solution can be applied). Then if the model tends to hallucinate without the document information, it means that the model may fail to recognize the reasoning tree or get some incorrect middle results due to knowledge or capacity limitation, and RAG can help to provide more knowledge and reduce the reasoning depth of the original task. In our paper, we mainly consider that how much reasoning depth can be reduced and how much would cost for filtering noisy documents.

Considering the help brought by extra knowledge, in our opinion, it mainly helps by letting the model directly know the answer of some middle results by information extraction instead of the original inference process, so it will not help the model to correct its wrong reasoning path, instead it directly leads the model to the right path. For example, a query "When is Alan Mathison Turing born?", and the model may hallucinate and give answer 1915, then we give the document information "Alan Mathison Turing father of artificial intelligence born in 1912. When is Alan Mathison Turing born?", then the model tends to answer "1912" rather than "I think it should be 1915, but the document says that it is 1912, so the answer is 1912". Therefore, RAG can help mitigates hallucinations, but not by correcting the wrong one, instead it directly lead to the correct reasoning path.

Q2: All three models have been trained on chat data and are aligned, but the explanation given in lines 482-485 seems rather tenuous—could you provide a stronger justification?

Answer: Thanks for your suggestion. The data used for finetuning Vicuna is mainly collected from ShareGPT it contains 70K user-shared multi-turn ChatGPT conversations, in this dataset, previous conversation data are quite long and functions similar to the document in RAG, they both provide information for current query. And when finetuning Vicuna, the previous conversation is placed before the current query (document before the query), so the model might overfit to the situation and when place the query ahead of documents, the model would perform bad. For Llama 3 and Mistral, they may be trained on various data so place the query ahead of document would not lead to overfit or the previous chat data is not that similar to the documents of RAG.

Thanks for your reply and further suggestions. Please find below our comments on the raised issues and questions:

Q1: Regarding W3, I would like to understand the differences in baseline settings in Table 2, specifically, what are the implementations corresponding to vanilla, LoRA, prompt, and LoRA+prompt.

Answer: The vanilla means that we do not finetune the model and directly conduct inference and measure the performance. LoRA/prompt means we conduct LoRA finetuning/prompt tuning then test the performance. LoRA+prompt means that we combine LoRA finetuning and prompt tuning, which means we conduct LoRA finetuning with some extra virtual prompt tokens added. The test set contains 1000 samples that are different from those in the training set.

Q2: For W5, I would like to see the performance of different baseline methods when evaluated on documents containing only noise, thereby demonstrating DPrompt's robustness.

Answer: Thanks for your suggestion, below we show how does the methods perform when evaluated on documents containing only noise. Those models are trained on datasets containing both gold and noise documents.

The result shows that our method performs well when only noise documents are contained.

Q3: Regarding Q1, are you claiming that your paper's fundamental assumption is that RAG only reduces reasoning depth without correcting reasoning direction? Isn't this too strong an assumption? Or do you consider the chain process of RAG correcting reasoning direction as part of the original ideal reasoning path?

Answer: We believe that RAG helps correct the reasoning direction, it leads the LLM to the correct reasoning tree, and RAG can help to reduce the reasoning depth of the correct reasoning tree. But in our opinion, if the LLM reasons in a wrong way without the document information, and then we add the document to the prompt, it will not follow the original wrong reasoning path, instead it directly leads to the correct reasoning path whose depth is reduced by RAG. For example, a query "When is Alan Mathison Turing born?", the reasoning path might be "Alan Mathison Turing->The Great Scientist of Artificial Intelligence->born in 1915". Then we give the document information "Alan Mathison Turing, the father of artificial intelligence was born in 1912", then the reasoning path might be "Alan Mathison Turing->born in 1915" rather than "Alan Mathison Turing->The Great Scientist of Artificial Intelligence->born in 1915->document analysis->born in 1912". In nutshell, we believe that RAG does correct the reasoning direction, but it does not correct it when the LLM has reasoned the wrong answer, instead it directly leads to the correct reasoning path (no process like "Alan Mathison Turing->The Great Scientist of Artificial Intelligence->born in 1915->document analysis->born in 1912", instead it directly leads to "Alan Mathison Turing->born in 1915"). We are not sure that we correctly understand your concern, please let us know if there exists some misunderstanding. Also we are happy to know it if you have a better idea about what is the reasoning process like with additional document information.

Q4: For Q5, my intention was to suggest that such benchmarks designed for noise could better demonstrate your method's robustness. Even though it doesn't require multi-hop reasoning, it can still be expressed as a reasoning graph with a small number of nodes.

Answer Thanks for your suggestion, it seems that adding more experiments on RGB is helpful, we will add it in the future version.

Q1: Why is the connection between different nodes in adjacent layers assumed to occur with probability q when defining the reasoning tree?

Answer: We do not fix the probability of connection between adjacent layers, instead q is a random variable depending on the specific task assigned to the Large Language Model (LLM). For different tasks, the value of q also changes, so is the probability of valuable retrieval p. In our paper, we show that to reduce the reasoning depth of LLM, a sparse connection between adjacent layers or high probability of retrieval is required, showing that in normal cases, RAG can hardly help the reasoning of LLM, but when the task is simple (sparse connection) or we can effectively retrieve useful documents, RAG can help the reasoning of LLM.

Q2: If a deterministic representation is used, causing f(t), as expressed in Line 212, to no longer increase with respect to t, is the subsequent analysis still valid? (A simple scenario is when a node has three child nodes, with the first two having two child nodes each, and the last one having five. In this case, the erase rate does not exhibit a monotonic relationship.)

Answer: Our paper primarily examines the expected rate of node erasure. The fact that $f(t)$ monotonically increases with $t$ suggests that the expected erasure rate should also increase but it may not increase at ease case. So, as illustrated in your scenario, the erased nodes could be smaller with more upper layer nodes erased. Nevertheless, this discrepancy does not impact our subsequent analysis. If $f(t)$ does not increase with $t$, it becomes less likely that RAG will enhance the reasoning capabilities of LLMs. In cases where the probability of erasing a node decreases compared to its upper layer, the fission reaction described in Theorem 2.4 is less likely to occur, resulting in diminished support for reasoning provided by RAG. Furthermore, when exploring how RAG can enhance LLM reasoning, we aim to demonstrate that RAG offers overall assistance across various tasks, which is why we calculate the expectation rather than focusing on specific cases.

Q3: Di He et al. proved that CoT can enhance computational power through the “thought” portion, enabling the resolution of arbitrary DP problems. Additionally, other studies have shown that adding meaningless characters during reasoning can also increase computational capacity, thereby enhancing the model’s reasoning ability. Considering these findings simultaneously, whether they would impact the original conclusions.

Answer: Actually our work considers different question compared to Di He et al, they show that LLM with CoT can solve arbitrary DP problems by generating step-by-step reasoning or explanations rather than providing direct answers, formalized as $x_1 = f(x),\ x_2 = f(x, x_1), \ldots, y = f(x, x_1, \ldots, x_k)$. This process allows CoT to effectively expand reasoning depth by execute $f$ multiple times, potentially reaching infinite depth with sufficient CoT steps. However this is achieved by generating a long sequence, so CoT actually helps by increasing the inference cost and achieve better reasoning ability. And adding meaningless characters may be helpful because larger hidden space (num token $\times$ token dim) or changed attention pattern. But in our paper, we mainly considers that how much can the information contained in the documents decrease the reasoning depth of the original problem, making the original problem easier to solve. Therefore, CoT tries to increase the reasoning ability of LLM by enlarge the inference cost, but RAG helps by reduce the reasoning depth of the original problem enabling the LLM to accomplish tasks with higher reasoning requirements. And our papers shows that the help might be limited, and the effort to extract those information and filter out irrelavant documents could be difficult, the help RAG brought may be less than the effort needed for noise filtering.

Dear reviewer,

We would like to know that is there any concern we do not solve, or do you have further suggestions to improve the paper. If not, could you please raise the score.

Best regards

Thanks so much for the response. However,.

1. I don't find the argument convincing as it appears to be a simplified assumption.
2. The long form experiments such as with Bio (https://arxiv.org/abs/2305.14251) and ALCE-ASQA (https://arxiv.org/abs/2305.14627)

Thanks,