Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoning

We sincerely appreciate your thorough review and your positive remarks regarding the conciseness, effectiveness, and delivery of our work. We deeply appreciate your constructive critique regarding our limitations, as it truly helped us strengthen this research.

W1: The main idea appears to be an incremental improvement over Search-R1 ...

Thank you for your incisive feedback regarding our paper's contribution. You've accurately identified the two core foundations of our work: (1) search-during-reasoning methods and (2) document refinement in RAG. We believe our proposed AutoRefine introduces distinct technical contributions, differentiating it from current approaches in three key aspects: (1) training technique, (2) data requirements, and (3) model behaviors, as summarized in Table 1:

Table 1: Technical Comparison Between Our Work and Previous Methods.

As you suggested, we elaborate on the unique challenges and benefits of applying RL-driven refinement within a search-during-reasoning framework.

[Unique Challenges] To illustrate the challenges in developing refinement abilities via RL, we conducted new experiments providing empirical evidence, as shown in Table 2.

• Simple paradigm modification is helpful but sub-optimal. Simply changing the reasoning paradigm to "search-and-refine-during-think" does improve overall performance.
• Retrieval reward on retrieved documents shows limited improvement. Checking whether the answer appears in retrieved documents is also not the ultimate solution, which is noticed by previous research[7]. We hypothesize this is because rewards on documents may shortcut the model to draft more search queries for broader knowledge coverage, rather than more accurate queries and more efficient refinement.
• Linear reward may over-emphasize intermediate behaviors. An intuitive way is to add $R_{ref}$ with $R_{ans}$ linearly as $R_{overall}=R_{ans}+R_{ref}$. However, we believe that the extra abilities (format-following, refinement) should be foundational yet incidental, as the primary aim is to achieve a correct answer. As a result, non-linear retrieval rewards surpasses linear rewards on both docs/refines.

Table 2: Comparison Between Different Rewards Used in AutoRefine.

[Unique Benefits]

• No annotation requirements. AutoRefine leverages the intrinsic knowledge refinement ability of LLMs without preliminary search trajectory annotation.
• Complexity in search-during-think reasoning. We conducted additional experiments comparing AutoRefine with using an external refiner model with Search-R1, as shown in Table 3 addressing your Q3. While external refiner achieve good performance on single-hop QA, it causes performance drops on multi-hop ones. Besides summarization, AutoRefine learns to recognize missing information and plan subsequent search steps through RL (Table 5 in our paper). This ability contributes to the superior multi-hop performance.

W2: Could supervised fine-tuning (SFT) or instruction tuning suffice for learning the refinement step?

We appreciate the reviewer's astute observation regarding the unclear justification in our introduction. We agree that our initial statement, claiming "SFT often fails to generalize retrieval behaviors beyond the training distribution", is unsubstantiated and overly opinionated without empirical evidence.

Upon your suggestion, we propose the following revisions to clarify the necessity of RL for training retrieval-augmented LLMs:

• Revised Justification for RL. We will modify the statement to highlight the practical challenges of SFT. Specifically, below is our revised statement:

"While SFT can be effective for training large models for search, it sometimes necessitates the construction of high-quality search paths, which incurs additional effort and resource overheads. To address this, recent studies draw inspiration from RL-based post-training and explore RL for retrieval-augmented reasoning, achieving excellent results by only evaluating final answer correctness without the need for pre-collected reasoning paths."

• Addition of Contextual Citations. We will explicitly acknowledge the effectiveness of SFT by adding the following citations [4,8,9,10].

Thank you once again for identifying this crucial flaw in our introduction. We will thoroughly review the remainder of our paper to identify and rectify any similar issues, ensuring our claims are rigorously supported and well-justified. We believe these revisions will significantly solidify our work and enhance its contribution to the research community.

W3: The part of title “Autonomous Retrieval-Augmented Reasoning of LLMs” is somewhat misleading.

We sincerely thank the reviewer for their insightful feedback regarding our original title. We agree that it (1) did not fully capture our core innovations, the refinement module and retrieval reward, and (2) could indeed imply a broader scope than intended.

Acting on this valuable suggestion, we've decided to update our paper's title to:

Search and Refine During Think: Facilitating On-the-Fly Knowledge Distillation for Improved Retrieval-Augmented Reasoning

We believe this new title more accurately reflects the essence of our work by highlighting its key focus (retrieval-augmented reasoning), key novelty (on-the-fly knowledge distillation methodology), and key results (improved performance).

Q1: Are these two scenarios treated equivalently during training?

Yes. In our method, the retrieval reward specifically reflects the correctness of the refinement process itself. We believe that truly effective search-augmented reasoning requires not only retrieving correct information, but also effective use of the information.

As shown in Figure 5(a) in our paper, the search success rate begins relatively high compared to refine and answer, and ends up nearly 80%. From this phenomenon, we hypothesize that search behaviors are relatively easy to learn. Thus, we primarily shape the refine behaviors during RL training.

Q2: Can the authors provide empirical evidence or prior work that supports this claim, especially in the context of retrieval-augmented generation?

Please see our response to W2.

Q3: Is RL truly necessary for learning the refinement step?

Thank you for raising the excellent point regarding the necessity of RL for the refinement step and the comparison against more direct, supervised approaches. Upon your advice, we conduct additional experiments under the intuitive setting: using a fine-tuned external refiner [11] to summarize retrieved documents in Search-R1, and the results are presented in the table below:

Table 3: Comparison Between SFT-based Refiner and RL-based Refinement.

From these results, we observed that external refiners improves performance on several general QA benchmarks, but leads to a performance decline on multi-hop QA ones.

[Role of RL] These observations lead us to the following explanation regarding the distinct roles of external refiners and our RL-trained refinement module:

• Single-Hop Scenarios: In single-hop QA, refinement learned via RL appear to function similarly to external refiners.
• Multi-Hop Scenarios (RL's Advantage): Under multi-hop settings, the LLM learns to introspect missing information (Table 5 in the paper). This phenomenon explains why RL-driven refinement outperforms simply applying external refiners in multi-hop QA.

[1] Learning to filter context for retrieval-augmented generation

[2] Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions

[3] InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales

[4] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

[5] Search-o1: Agentic search-enhanced large reasoning models

[6] Search-r1: Training llms to reason and leverage search engines with reinforcement learning

[7] An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents

[8] InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales

[9] Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering

[10] Multi-Hop Paragraph Retrieval for Open-Domain Question Answering

[11] BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Dear Reviewer KVnw,

Thank you again for your constructive suggestions. We have carefully addressed your concerns by:

• systematically comparing previous methods and ours
• elaborating the unique challenges and benefits of this work
• modifying the misleading statement & title
• demonstrating the effectiveness of RL-driven refinement with new experiments

We look forward to engaging in further discussion with you and would greatly appreciate your positive feedback on our rebuttal.

Best regards,

The Authors

We're grateful for your insightful review and appreciate your recognition of our work's comprehensive evaluation and outstanding efficacy. We have addressed your major concerns in detail below.

Weakness 1: More Explanation over Methodological Contributions. It seems that the two methodological contributions of this paper ("Refine" and "Process Rewards for Search") each help on different benchmarks. This suggests that potentially these two methods are actually more targeted. I was hoping to see more discussion of this.

Response: We appreciate your insightful suggestion regarding the distinct impacts of our methodological contributions.

We agree that our two main contributions, "Knowledge Refinement Step" and "Processing Rewards for Search" (retrieval reward), exhibit different performance characteristics across benchmarks. To better answer the question, we extend our ablation study with a new control group that combines Search-R1 with an external fine-tuned summarization model (facebook/bart-large-cnn [1]), as shown in Table 1. In the new setting, the external model summarizes retrieved documents and appends the summarization at the end of the retrieval.

Table 1: Extended Ablation Study.

We observe the following:

• External Document Summarization Boosts Single-hop Performance. Search-R1's single-hop QA performance (TriviaQA, PopQA) is approximately the same as AutoRefine when equipped with an external summarization module. This indicates that mere summarization is sufficient in such simple scenarios. On the contrary, the multi-hop QA performance slightly drops.
• Refinement Step Boosts Multi-hop Performance. Incorporating refinement steps in the reasoning paths significantly improves the model's performance on multi-hop benchmarks, while keeping the single-hop performance unchanged. What promotes such solid multi-hop performance? Taking a closer look at the case studies (Table 5 in our paper), we find that the model evolves the introspection behavior - it spontaneously reflects missing information in the retrieved documents and
• Retrieval Reward Yields Comprehensive Improvements. However, merely adding a refinement step does not fully develop refinement behavior. When trained with the retrieval reward, we observe a comprehensive improvement across both single- and multi-hop benchmarks. Notably, the model exhibits similar performance on single-hop benchmarks as vanilla Search-R1 with an external summarization module, which suggests that the performance gain stems from the enhanced summarization capability of the LLM.

We hope our fine-grained explanation with additional experiments can address your concerns.

Weakness 2: Possible Risk of EM Retrieval Reward. The retriever reward uses a binary reward based on term matching of the answer with the ground truth. This seems to effectively be training the model to prefer exact match queries, which may not be globally optimal. I'd like to see more discussion of the possible risks of this approach.

Response:

Thanks for pointing out the potential risk in the strict cover EM retrieval reward. After careful consideration, we hypothesize that the strict reward may harm the models' performance when the answer is complex.

We add new experiments, comparing our current retrieval reward against two more flexible designs:

1. Token-level recall: Calculates how many tokens in the ground-truth answer appear in the refinement,
2. Word-level recall: Calculates how many words in the ground-truth answer appear in the refinement;

under different benchmark settings:

• Full dataset: Test on all questions in the benchmark.
• Complex answers: Test on questions with an answer >5 words.

The model is evaluated with EM, and the results are summarized in Table 2.

Table 2: Comparison between original and finer-grained retrieval reward design.

The results confirm that though the cover EM reward works well on most QA cases, it could bring potential risks such as performance degradation when the target answer is complex.

Weakness 3: Performance at Different Model Scales. As mentioned by the authors in the limitation section, the proposed method is only demonstrated with 3B models, which are a bit smaller than standard for readers in a RAG system these days (8B has become the norm in academic work). It's likely that 7B or 14B models will benefit less from carefully-designed inductive biases such as the ones proposed in this paper. In any case, this does not invalidate the value of the paper.

Response: We appreciate you raising the point about model scale and its potential impact on our method's effectiveness. We agree that larger models (e.g., 7B or 14B) might inherently benefit less from explicitly designed inductive biases.

We conduct additional experiments to demonstrate AutoRefine's effectiveness at the 7B level, and the results are shown in Table 3. The baseline result (Search-R1) is reproduced using their officially released checkpoint.

Table 3: Performance Comparison at Different Model Scales

From the Table, we observe that AutoRefine still yields positive results even at the 7B parameter level, demonstrating an overall performance improvement of approximately 5%.

This gain is indeed slightly less than the significant 9% improvement observed with 3B models, which aligns with your hypothesis that the benefits from explicit refinement steps might relatively diminish as model scale increases.

Question 1: Potential Risk in Strict Retrieval Reward. The retriever reward uses a binary reward based on term matching of the answer with the ground truth. This seems to effectively be training the model to prefer exact match queries, which may not be globally optimal. Is this a legitimate concern?

• Please see our response to your Weakness 2.

[1] Lewis, Mike, et al. "BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension." Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020.

Dear Reviewer UA8w,

Thank you again for your constructive comments and positive opinions. We have carefully addressed your concerns by:

• making more explanation of methodological contributions with new empirical evidence
• exploring new reward designs to see the practicability of binary retrieval reward
• successfully expand our method to a larger model scale

We look forward to further discussion with you and would greatly appreciate your positive feedback on our rebuttal.

Best regards,

The Authors

We thank the reviewer for recognizing the significant performance gain achieved in this work. We provide point-to-point responses to your primary concerns below.

W1: The paper appears to be an incremental development over Search-R1[1] and Search-o1[2]. It is established in the recent works, such as Search-o1[2] (Figure 2, "Get concise information and continue coherent reasoning.") and FILCO[3], that evidence compression plays critical role to avoid noise generate from imperfect retrieval.

We appreciate the reviewer's comments on the contribution of our work, but we beg to refute the opinion that our paper is merely an incremental development.

Our work addresses distinct challenges and demonstrates unique advantages:

• [Unique Challenges We Tackle] We conduct new experiments (see Table 2 addressing Weakness 1 of reviewer KVnw) to demonstrate the challenges in developing refinement abilities via RL in Search-R1. These challenges include:
  • Sub-optimality of simple paradigm modifications.
  • Limited improvement from direct retrieval rewards on retrieved documents, echoing findings in previous research [1].
  • Model benefits from non-linear retrieval reward.
• [Key Behaviors and Unique Benefits] We conduct additional experiments (please see Table 3 addressing Weakness 1 of reviewer KVnw) to demonstrate the advantage of our method beyond simply applying summarization modules in Search-R1.
  • No Extra Annotation Requirements, unlike previous SFT-based knowledge distillation.
  • Recognization of missing information and adaptively plan subsequent search steps through RL (Table 5 in the paper).

We hope our detailed analysis of challenges and unique benefits clearly illustrates the novel contributions of our work.

W2.1: Though the focus of the paper is on evidence compression, no strong baseline is chosen related to it, such as FILCO[3] and FaviComp[4]. Moreover, the Related Work section completely overlooks all evidence compression literature.

Response: We appreciate the reviewer's suggestion for comparison with previous evidence compression methods.

According to the reviewer's suggestion, we have conducted an additional experiment comparing evidence compression methods with AutoRefine comparing with (1) FaviComp [2] and (2) Search-R1 with an external summarization module [3]. The results are shown in Table 1.

Table 1: Comparison with compression approaches.

The experimental results show:

• External summarization helps on single-hop QA benchmarks (TriviaQA and PopQA);
• AutoRefine surpasses the external summarizer in multi-hop QA setting, which highlights the unique advances summarized in the previous response addressing weakness 1.

W2.2: Moreover, important details of the baselines are missing, such as which model size and algorithm (For Search-R1, PPO or GRPO?) was chosen for comparison.

Response: Thank the reviewer for pointing out the unclear implementation details of baseline methods.

• We use GRPO as the default algorithm for RL-based baselines.
• We use Qwen2.5-3B-Base as the default model for RL-based methods, as depicted in Sec 3.1.

We hope the above clarification can address your concerns.

W3: Model scale and scalability: The paper does not discuss model scale’s impact on generalization or scalability. The choice of GRPO over PPO for Search-R1 is unclear, as PPO is a stronger baseline for this task in Search-R1.

Response: We thank the reviewer for pointing out the importance of the model's scale and RL algorithm choice.

[Impact of Model's Scale] We conduct an additional experiment to see the impact of the model's size, please check Table 3 in our response to Reviewer UA8w. AutoRefine surpasses previous methods by ~9% on 3B-level model and ~5% on 7B-level model.

[Choice of RL Algorithm] We beg to refute the claim that “PPO is a stronger baseline for this task”, because:

• As shown in Table 3 of Search-R1 [4], the GRPO algorithm has comparable performance as PPO on Qwen-3B models.
• According to the discussion in Sec 5.1 of Search-R1 [4], "both methods GRPO and PPO) achieve similar final train reward and performance”.

W5: Limited applicability: The soft-reward mechanism is restricted to short-form generations, with no exploration of its utility for long-form tasks (e.g., Self-RAG). This limits the method’s real-world relevance. It is also unclear what is the latency of the method over baselines for use experience.

Response: We thank the reviewer for this comment.

[Long Form QA Effectiveness] We really appreciate the importance of long-form generation tasks, but we focuses on multi-hop QA ability which is also of high practicality. We believe that RAG models don't have to excel at every QA type to be considered contributory. For example, Self-RAG [5] focuses on improving the long-form QA without considering multi-hop, but it is still an excellent work with a high influence.

[Service Latency] The additional computation of AutoRefine is merely the generation of <refine> content, which causes little and acceptable latency overhead. To support our claim, we showcase additional results about speed test between models in Table 3.

Table 3: Speed Test of Different Methods.

W5: Statistical rigor: Error bars or standard deviations are omitted, such as in Table 1 and Figure 4, despite the checklist mentioning them to validate result significance. This weakens the reliability of the reported improvements.

Response: Thanks for pointing out the importance of statistical significance. We add a new statistical analysis by running the experiments under different random seeds, and the results are shown in Table 4. The p-value column indicates the T-test between each baseline and AutoRefine.

Table 4: Statistical Analysis.

The results suggest the improvement achieved by AutoRefine is significant over previous search-during-think methods.

Q1: Reward function ablation: The reward function’s design (e.g., 0.1 as partial reward, no partial reward for correct final answers) is not analyzed. This omission limits understanding of the method’s effectiveness and trade-offs.

Response: Thanks for your comment on the lack of explanation in our reward design. Different reward designs will indeed influence AutoRefine's performance. We conduct additional experiments to reveal this influence. Please refer to our Table 2 in response to Reviewer KVnw.

The results indicate:

• Retrieval reward on retrieved documents shows limited improvement. We find that directly checking whether the documents contain the answer only gives a limited performance gain. Previous researchers have also made similar conclusions [1].
• Linear reward may over-emphasize intermediate behaviors. Adding $R_{ref}$ with $R_{ans}$ linearly as $R_{overall}=R_{ans}+R_{ref}$ is also sub-optimal. In this work, we use non-linear retrieval rewards that prioritize final answer correctness beyond merely correct knowledge refinement.

Q2: Hyperparameter justification: The paper lacks explicit rationale for key hyperparameters (e.g., choice of E5 as retriever, 3 documents fetched during training). If informed by prior work, this should be clearly stated to demonstrate scientific grounding.

Q3: Training configuration ambiguity: The choice of maximum search calls (5) and whether training settings align with Search-R1 are unclear. Section B.1 and Table 3 do not provide sufficient details to verify reproducibility or consistency with prior work.

Thank you for the opinion. We keep the retrieval setup the same as our baseline Search-R1. We will acknowledge this inspiration in future revision.

The choice of maximum search calls is empirically set, because:

• The average search calls of our method remain less than 2.5, so no need for too many calls.
• There’s no clear evidence that this parameter will significantly influence the result.

All methods compared in Table 1 have the same experiment settings, as mentioned in Section 3.1 of our paper.

We have included most of the important parameters in Section 3.1, Appendix B, and Table 3. we suggest referring to our attached code for more details.

Q4: Data split specification: The dev split of Hotpot-QA (distractor vs. fullwiki) is not explicitly stated, creating ambiguity about the evaluation’s scope and fairness.

We use the dev split of HotpotQA, which does not contain distractor/fullwiki variants (Table 4 in paper).

[1] An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents

[2] Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation

[3] BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

[4] Search-r1: Training llms to reason and leverage search engines with reinforcement learning

[5] Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

Dear Reviewer 9S2E,

Thank you again for your constructive suggestions. We have carefully addressed your concerns by:

• elaborating on distinct challenges and unique advantages in this research comparing with Search-R1
• including new baseline methods and clarifying baseline/hyperparam settings
• demonstrating the acceptable latency overhead and scalability of our method
• adding statistical analysis
• disambiguating data split and training configurations

We look forward to further discussion with you and would greatly appreciate your positive feedback on our rebuttal.

Best regards,

The Authors

Dear Reviewer 9S2E,

Regarding your feedback on the evidence compression baseline, we are pleased to share the new experimental results we obtained during the discussion period.

W2.1: Though the focus of the paper is on evidence compression, no strong baseline is chosen related to it, such as FILCO[3] and FaviComp[4]. Moreover, the Related Work section completely overlooks all evidence compression literature.

In our rebuttal, we compared FaviComp and Search-R1 + BART as baselines. However, we realized that the BART model might be considered small and somewhat outdated, so we supplemented our experiments using the Qwen2.5 model as an external refiner for Search-R1. Specifically, we prompted Qwen2.5-3B-Instruct to perform evidence compression in two settings:

1. Ask the model to summarize the documents
2. Ask the model to summarize the documents, and make plans for the next search step

The results are presented in the table below.

Table: Performance Comparison with different external refiners

From these new results, we find that:

• As expected, the newer and larger Qwen2.5-3B-Instruct serves as a better summarization model than BART (a 0.011 overall improvement).
• Autorefine demonstrates superior performance compared with the evidence compression baselines.

We hope these results better address your concerns.

Besides, we want to kindly remind you that the discussion period is coming to a close. We look forward to hearing from you.

Best regards,

The Authors

Dear Reviewer 9S2E,

Thank you for your questions regarding our model's scalability, service latency, statistical rigor, and missing of baseline methods. To address your concerns, we've conducted additional comparisons at the 7B level, a latency test, statistical analysis, and added new baseline methods. We hope our response has resolved your main concerns and we would appreciate it if you would consider increasing our score. If you still have some concerns, we would be very happy to have the opportunity to continue our discussion with you.

Best regards,

The Authors

We sincerely appreciate the reviewer's acknowledgement of the soundness, strong performance and comprehensive analysis. Below are point-to-point responses addressing your major concerns.

Weakness 1: Similarity to the Search-R1 paper. Search-R1 could adopt these two tweaks with minimal changes. I’m not fully convinced that it is worth a NeurIPS publication.

Response: Thank you for your constructive feedback. We understand your concern regarding the similarity to Search-R1 and the novelty of our contribution.

While AutoRefine builds upon the foundational search-during-reasoning paradigm established by Search-R1, we respectfully disagree that our method constitutes "two tweaks with minimal changes." Here we would like to clarify the key challenges we tackled and the key advances we achieved in our paper:

[Key Challenges] To demonstrate the key challenges in developing knowledge refinement steps in search-during-think methods, we conduct additional experiments comparing different supervision reward signals, as shown in Table 1. We empirically show that:

• Simple paradigm modification (e.g., "search-and-refine-during-think") is sub-optimal. As observed in Table 1, merely adding a refinement step does improve the overall performance. Nevertheless, we uncover that the model's refinement ability can be further enhanced with proper retrieval-related rewards.
• Direct retrieval reward on retrieved documents showcases limited improvement. One simple way to add a retrieval-related signal is to check the existence of the final answer in the retrieved documents. Nevertheless, as shown in Table 1, we find that this improvement is also not the ultimate solution. Previous researchers have made similar conclusions: direct retrieval reward can not always improve the performance of search-during-thinking models [1].
• Linear rewards may over-emphasize intermediate behaviors. Our findings clearly indicate that a linear combination of answer and refinement rewards ($R_{overall}$​=$R_{ans}$​+$R_{ref}$​) is inferior to our proposed non-linear reward design, which prioritizes the final answer correctness while still fostering robust refinement capabilities. This intricate balance in the reward function is a core innovation of AutoRefine, directly contributing to its superior performance across various QA benchmarks.

Table 1: Comparison Between Different Rewards Used in AutoRefine.

[Key Advances] Building on our solutions to these challenges, AutoRefine offers key advances that enhance performance and capabilities, such as:

• Summarization ability with no data annotation. AutoRefine successfully activates the intrinsic knowledge refinement ability embedded within LLMs, resulting in significant performance gain across single- and multi-hop benchmarks.
• Introspection of missing knowledge. In multi-hop search-during-think methods, mere summarization may not be sufficient. The model also needs to recognize missing information and adaptively plan subsequent search steps. We observed that AutoRefine successfully developed this capability in the refinement steps through RL (Table 5 in our paper), which significantly contributes to our superior performance on multi-hop benchmarks.

Upon your opinions, we have acknowledged that our delivery in the paper does not fully reveal the challenges in developing the spontaneous refinement ability and unique advances of our method. We hope our explanation has successfully addressed your concerns regarding the novelty and contribution of this work.

Weakness 2: More experimental settings exploring more diverse experimental settings could make the paper much stronger. For example, if the ground-truth is given as a long and complex answer, how do we compute the retrieval reward, and how does that affect the final performance?

Response: Thank you for this suggestion. We entirely agree that exploring more diverse experimental settings, especially concerning the nature of the ground-truth answers and their impact on retrieval reward computation, is crucial for strengthening our paper. Besides the experiment added in Table 1, we have conducted additional experiments that explore how a more fine-grained retrieval reward would affect the model's performance on long and complex answers.

[Adapting Retrieval Rewards for Complex Answers] Except for the existing cover-exact match used in our paper, we've explored two additional reward designs:

1. Token-level recall: Calculates how many tokens in the ground-truth answer appear in the refined documents,
2. Word-level recall: Calculates how many words in the ground-truth answer appear in the refined documents;

under different benchmark settings:

• Full dataset: Test on all questions in the benchmark.
• Complex answers: Test on questions with an answer >5 words.

The results are shown in Table 2.

Table 2: Comparison between original and finer-grained retrieval reward design.

The results presented in Table 2 confirm that a more fine-grained retrieval reward can greatly improve the model's performance when answers have complex forms, while keeping the overall performance approximately the same.

We hope these additional results could help strengthen our experiment design.

[1] Jin, Bowen, et al. "An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents." arXiv preprint arXiv:2505.15117 (2025).

Dear Reviewer jRtt,

Thank you again for your constructive opinions. We have carefully addressed your concerns by:

• elaborating on the key challenges we tackled and the key benefits we achieved in this research
• exploring more diverse experimental settings on long and complex QA

We look forward to further discussion with you and would greatly appreciate your positive feedback on our rebuttal.

Best regards,

The Authors