WebDancer: Towards Autonomous Information Seeking Agency

We sincerely thank you for your insightful comments and constructive suggestions, which are highly valuable for improving our work. Below, we provide point-by-point responses to the raised concerns.

Q1. Novelty

Our core contribution lies in proposing a reusable and end-to-end framework for reproducing a Deep Research agent. This framework consists of four stages: (1) browsing-based data construction, (2) trajectory sampling, (3) supervised fine-tuning for effective cold-start capabilities, and (4) reinforcement learning to enhance generalization.

Throughout this pipeline, we investigate several fundamental questions, such as how to construct high-quality QA data, how to represent intermediate thoughts, how to design data formats for SFT and RL under deep research scenarios, and how to enable multi-step reasoning. To the best of our knowledge, this is the first work that attempts to systematically reproduce Deep Research behavior and provide actionable implementation strategies.

While distillation from advanced LLMs has become a common practice, we would like to clarify that effective distillation is neither trivial nor devoid of research value. Our work systematically explores several under-explored but important aspects of this process, including which models to distill from, how to construct and filter high-quality distillation data, and how to best utilize this data for different stages (SFT and RL) of training.

Rather than treating LLM distillation as a plug-and-play component, we view it as a central research challenge and contribute actionable solutions and insights to make the distillation process effective and scalable in the context of deep research agents.

Q2. Task Generalization

Thank you for your careful reading and thoughtful comments. We would like to clarify that our current work focuses on multi-step single-turn QA tasks. As shown in Appendix F (case study), solving each query often requires the agent to perform multiple steps of information retrieval, verification, and reasoning. And the agent interacts with the user in a single-turn setting, meaning the user provides only one initial query and does not participate in the intermediate reasoning steps.

We will revise and clarify this point in the main text to avoid misunderstanding.

As for multi-turn interactions and human-in-the-loop dialogue, we agree that they are important directions for building more interactive research agents, and we plan to explore them in future work.

Regarding open-ended task structures, our current focus on short-answer QA tasks is due to their well-defined evaluation protocols and relatively reliable reward computation. In contrast, open-ended and long-form tasks present challenges in both RFT data construction and RL training, mainly due to the lack of robust evaluation metrics.

Interestingly, we observe that our model, trained on short QA tasks, demonstrates strong generalization to long-form settings in terms of information-seeking behavior. This suggests that the model has learned effective strategies for decomposing complex queries and locating relevant evidence, which naturally transfers to long-form tasks. We will include a case study to illustrate this generalization behavior in the revised version.

We appreciate your suggestion and will consider these broader task settings in future work.

Q3: Agent Action Space

We agree that the restricted action space deserves a more explicit discussion. In our current design, the agent operates with "search" and "visit" actions, which are considered fundamental primitives in the information-seeking process[1,2]. In principle, these two actions are sufficient to access and retrieve any information available on the web.

Moreover, our framework is designed to be modular and extensible. It supports seamless integration with both browser modeling (e.g., scrolling, form filling) and Python sandbox environments, enabling more complex interactions when needed. Given the challenges of sample efficiency in RL settings, we chose to focus on "search" and "visit" as a strong starting point. These tools already demonstrate substantial capabilities across our benchmark tasks.

We will add further discussion of this design choice and its trade-offs in the revised version.

Q4. Data Quality Filtering

Thank you for pointing this out. We acknowledge that our current phrasing may be ambiguous. Specifically, the sentence “we first apply rules to filter out trajectories with more than two actions, ensuring that there are no hallucinations and no severe repetitions” might be misinterpreted as suggesting that the number of actions alone is used to detect hallucinations.

To clarify: the filtering pipeline operates on top of trajectories with more than two actions. We then apply additional filtering steps to ensure quality. In particular:

• Hallucination detection is based on inspecting the thought steps to check whether the agent generates hallucinated tool outputs (i.e., fabricating a tool return and reasoning based on it, without actually invoking the tool).
• Severe repetition is detected using n-gram overlap heuristics, ensuring that repetitive patterns across thoughts and actions are filtered out.

We will revise the manuscript to make this process clearer and avoid confusion. Additionally, we commit to open-sourcing the full data filtering pipeline, so that the community can inspect, reproduce, and build upon our approach.

If you have any further questions or suggestions, we would be happy to address them. We kindly ask you to consider our responses when reassessing the paper, and we appreciate your constructive feedback.

[1]. OAgents: An Empirical Study of Building Effective Agents

[2]. WebThinker: Empowering Large Reasoning Models with Deep Research Capability

Thank you very much for your recognition of our work and for the constructive suggestions. We truly appreciate your thoughtful feedback and will address each of your comments in detail below.

Q1: Resource-extensive

We acknowledge that our method is resource-intensive during both training and inference. However, our goal is to build a Deep Research agent capable of solving high-value problems that often approach or exceed human-level complexity. In such settings, the trade-off between computational cost and reasoning capability is justified.

Notably, even for advanced research agents like those developed by OpenAI, solving a single problem may require 30 minutes to an hour of computation—a timeframe comparable to what human experts would spend on the same task. We believe that investing computational resources to automate such high-effort, high-reward tasks is both meaningful and impactful for augmenting human productivity.

We will clarify this motivation in the revised version.

Q2: : Reward Reliability

In our early-stage experiments, we extensively compared several commonly used reward designs in the search-agent setting, including recall, F1, and model-based rewards. Our findings motivated the use of LLM-as-Judge.

Specifically, both recall- and F1-based rewards suffered from reward hacking:

• For recall, the model often learned to include large numbers of candidate answers in order to boost recall, leading to verbose and unreliable outputs.

• For F1, the model tended to output only partial answers that scored well but lacked completeness and readability.

As an alternative, we adopted a model-based reward using LLM-as-Judge. Our prompts are adapted from established benchmarks (e.g., HLE[3], BrowseComp[4]). To evaluate the robustness of this judge, we tested two strong LLMs: Qwen2.5-72B (open-source) and GPT-4o (closed-source). The results demonstrated high consistency between the two.

Furthermore, we manually audited 100 samples judged by Qwen2.5-72B and found only one judge error, suggesting strong reliability in practice. While a full quantitative evaluation of judge accuracy remains an interesting direction for future work, our empirical evidence indicates that the LLM-as-Judge is significantly more aligned with the QA task and more robust than standard metric-based rewards in this setting.

We will include a detailed discussion of this aspect in the revised version.

Q3: GRPO Performance

In our early experiments, we also evaluated GRPO on the Qwen-32B backbone and observed that it achieved around 35 points on GAIA, which was lower than DAPO under the same conditions. The performance gap may stem from DAPO’s more effective dynamic sampling of agent trajectories, which better aligns with our task setting.

Given the computational cost and efficiency considerations of RL training, we chose to focus on DAPO as the primary RL algorithm in this work.

Q4: Short vs. Long CoT

Thank you for the insightful question. We agree that simply being "longer" does not necessarily imply better reasoning. In our work, the Long-CoT trajectories are not just extended versions of the Short-CoTs; rather, they demonstrate richer reasoning behaviors, including:

• Step-by-step decomposition
• Hypothesis testing
• Handling information gaps
• Iterative reflection and self-correction

As illustrated in Appendix F, Long-CoTs often break down complex queries into subgoals, assess intermediate evidence, and revise earlier assumptions based on new findings. In contrast, Short-CoTs, while still logical, tend to follow a more direct and surface-level path with limited internal deliberation.

For example, in response to a question about the presence of a nonnative species (clownfish) in the US before 2020, a Short-CoT may look like this (with tool outputs omitted for brevity):

To find out where this species was found as a nonnative species in the United States before 2020, I will need to search for information from the USGS or related sources that track invasive species. I will start by searching for the clownfish and its status as an invasive species in the US.

The search results provide several resources related to the clownfish and its status as an invasive species. The most promising sources seem to be from the USGS database, which specifically tracks nonindigenous aquatic species. I should visit the relevant USGS webpage to gather detailed information about clownfish occurrences in the United States as a nonnative species.

The Wikipedia page did not provide any additional information regarding clownfish being found as nonnative species in the US before 2020.

While this Short-CoT is coherent and goal-oriented, it lacks deeper exploration, such as verifying conflicting sources or revisiting assumptions when evidence is missing.

We will include a side-by-side comparison between Short-CoT and Long-CoT examples in the revised version to better illustrate these distinctions.

Once again, we sincerely thank you for your recognition of our work and your thoughtful feedback. If you have any further questions or suggestions, we would be more than happy to address them.

Thank you for your follow-up. We’re glad our rebuttal addressed your main concerns. We appreciate your thoughtful review and recognition.

We sincerely thank you for your insightful comments and constructive suggestions, which are highly valuable for improving our work. Below, we provide point-by-point responses to the raised concerns.

Q1. Dataset Open-Sourcing

We are committed to open-sourcing all code used for data generation and curation. Furthermore, we will release the full CrawlQA and E2HQA datasets on open-access platforms such as HuggingFace and ModelScope. By doing so, we aim to contribute to and support the broader open-source research community.

Q2. Quality of CrawlQA

In CrawlQA, our construction strategy follows the construction methodology introduced in WebWalkerQA[1], which demonstrates that information buried deep within websites is generally not easily accessible via search engines. However, we acknowledge that some queries may still have shortcuts due to search engine indexing. To address this, we adopt a filtering step inspired by BrowseComp[2]: we employ a basic agent built on Qwen-72B with search capabilities to execute. Queries that can be resolved in a single-step search are removed. This ensures that the retained QA pairs require meaningful multi-step navigation and better reflect realistic web exploration challenges. We will include further details about this filtering process in the revised version.

Q3. Space Arrangement and Writing

Our core contribution lies in presenting a systematic, end-to-end, and actionable reproduction of a Deep Research agent. Thanks for your advice. Given the complexity of the full pipeline, we will allocate substantial space to ensure clarity and completeness. That said, we will revise the space arrangement to better balance novelty and background content, and refine the writing to be more concise and focused.

Q4. Insights and Novelty

Our core contribution lies in proposing a reusable and end-to-end framework for reproducing a Deep Research agent. This framework consists of four stages: (1) browsing-based data construction, (2) trajectory sampling, (3) supervised fine-tuning for effective cold-start capabilities, and (4) reinforcement learning to enhance generalization.

Throughout this pipeline, we investigate several fundamental questions, such as how to construct high-quality QA data, how to represent intermediate thoughts, how to design data formats for SFT and RL under deep research scenarios, and how to enable multi-step reasoning. To the best of our knowledge, this is the first work that attempts to systematically reproduce Deep Research behavior and provide actionable implementation strategies.

Q5. CoT Styles and Filtering Efficiency

For each query, we can collect both a short CoT and a long CoT version of the reasoning trajectory. Importantly, we train the two styles separately to avoid interference: as shown in Table 3, the instruct model is trained with short CoTs, while the reasoning model, like QwQ, uses long CoTs.

We also conducted transfer experiments across styles, and the results support the effectiveness of our current setup. Additionally, we experimented with combining both CoT styles during training, but as you anticipated, this led to performance degradation due to conflicting reasoning patterns between short and long CoTs. This confirms that treating them separately is a more effective strategy.

Regarding the filtering efficiency, for queries used in RFT training, we apply multiple layers of filtering, including format validity control during rejection sampling, correctness verification of final answers, quality assessment of thought steps, and constraints on the number of actions taken.

Our QA construction pipeline is designed to be scalable: as long as high-quality RFT-style trajectories can be generated, the corresponding QA examples are meaningful and useful. While high-quality QA data with detailed thoughts is crucial for RFT, other QA instances without such strict requirements can still be effectively used in the RL stage.

We will supplement and clarify this part in the revised version.

Q6. Task Generalization

Our current work focuses on short-answer QA tasks, primarily because they offer well-defined evaluation protocols and allow reward computation to be more reliable. In contrast, long-form QA tasks pose significant challenges in both RFT data construction and RL training due to difficulties in evaluation.

Interestingly, we observe that our model, trained on short QA tasks, demonstrates strong generalization to long-form settings in terms of information-seeking behavior. This suggests that the model has learned effective strategies for decomposing complex queries and locating relevant evidence, which naturally transfers to long-form tasks. We will include a case study to illustrate this generalization behavior in the revised version.

However, long-form QA involves not only information retrieval but also generation quality, which remains an open challenge. We plan to explore this direction in future work.

If you have any further questions or suggestions, we would be happy to address them. We kindly ask you to consider our responses when reassessing the paper, and we appreciate your constructive feedback.

[1]. WebWalker: Benchmarking LLMs in Web Traversal

[2]. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents

Dear Reviewer Q7WQ,

We would like to kindly remind you that the discussion period is approaching its end, with approximately 24 hours remaining. We truly appreciate the thoughtful feedback you have already provided and wanted to check if you might have any follow-up thoughts or questions regarding our rebuttal.

We are more than happy to further clarify any remaining concerns or provide additional details to support a productive discussion. Your continued input would be greatly valued as part of this collaborative review process.

Thank you again for your time!

We sincerely thank you for your insightful comments and constructive suggestions, which are highly valuable for improving our work. Below, we provide point-by-point responses to the raised concerns.

Q1: Exposition Improvement

Search-based web agent tasks present a set of unique challenges that differentiate them from traditional web-based QA or single-turn retrieval tasks. Specifically:

1. Lack of High-Quality, High-Difficulty Training Data: Unlike simpler retrieval tasks, search-agent benchmarks such as Gaia and BrowserComp involve complex multi-hop reasoning and multi-step interaction. However, there is a significant scarcity of training data that reflects this level of difficulty. The few existing training datasets are typically not well-aligned with the test distributions of these benchmarks. These benchmarks feature long-horizon tasks, which require capabilities beyond what standard SFT data can teach.

2. Limited Support for Multi-Step Search and Deep Research: Current agent models often struggle with multi-step retrieval and iterative refinement. This limitation hampers their ability to perform deep research, i.e., cases where information must be verified, cross-checked, or gathered from multiple heterogeneous sources.

3. Lack of Systematic Guidance for Deep Research Behavior: Before our work, there has been no systematic methodology for training or evaluating agents on “deep research” tasks. Most existing approaches focus on shallow interactions or fail to generalize to tasks requiring flexible search strategies, adaptation, and reflection.

Our work aims to address these gaps by providing both a practical framework and empirical insights into training agents capable of sophisticated web-based research behaviors.

We will enhance the Introduction and Related Works sections with a clearer and more comprehensive introduction to the unique characteristics and challenges of web agent tasks, to better support readers who may not be familiar with this area. Additionally, while we have already included a case study in Appendix F, we appreciate the suggestion to make the practical relevance more prominent. We will incorporate a more illustrative and motivating case study directly in the Introduction to help guide the reader and highlight the real-world significance of our setting.

Q2: Backbone Generalization

Due to computational constraints, we primarily conducted experiments using the Qwen2.5 series as backbone models. We selected Qwen2.5 for its competitive performance and open availability, which allowed for a thorough and reproducible evaluation of our proposed pipeline. Importantly, we have validated our approach across multiple sizes of Qwen2.5 (e.g., 7B, 32B), consistently observing performance improvements, which supports the generality of our method across model scales. In future work, we plan to extend our study to other backbone families, such as LLaMA and Mixtral, to further verify the robustness and generalizability of our approach across diverse architectures.

Q3: Agent Action Space

We agree that the restricted action space deserves a more explicit discussion. In our current design, the agent operates with "search" and "visit" actions, which are considered fundamental primitives in the information-seeking process[1,2]. In principle, these two actions are sufficient to access and retrieve any information available on the web.

Moreover, our framework is designed to be modular and extensible. It supports seamless integration with both browser modeling (e.g., scrolling, form filling) and Python sandbox environments, enabling more complex interactions when needed. Given the challenges of sample efficiency in RL settings, we chose to focus on "search" and "visit" as a strong starting point. These tools already demonstrate substantial capabilities across our benchmark tasks.

We will add further discussion of this design choice and its trade-offs in the revised version.

Q4: Reward Reliability

In our early-stage experiments, we extensively compared several commonly used reward designs in the search-agent setting, including recall, F1, and model-based rewards. Our findings motivated the use of LLM-as-Judge.

Specifically, both recall- and F1-based rewards suffered from reward hacking:

• For recall, the model often learned to include large numbers of candidate answers in order to boost recall, leading to verbose and unreliable outputs.

• For F1, the model tended to output only partial answers that scored well but lacked completeness and readability.

As an alternative, we adopted a model-based reward using LLM-as-Judge. Our prompts are adapted from established benchmarks (e.g., HLE[3], BrowseComp[4]). To evaluate the robustness of this judge, we tested two strong LLMs: Qwen2.5-72B (open-source) and GPT-4o (closed-source). The results demonstrated high consistency between the two.

Furthermore, we manually audited 100 samples judged by Qwen2.5-72B and found only one judge error, suggesting strong reliability in practice. While a full quantitative evaluation of judge accuracy remains an interesting direction for future work, our empirical evidence indicates that the LLM-as-Judge is significantly more aligned with the QA task and more robust than standard metric-based rewards in this setting.

We will include a detailed discussion of this aspect in the revised version.

Q5: Interaction Patterns and Agent Behavior

Current agent models lack robust multi-step reasoning and iterative retrieval, limiting their effectiveness in deep research tasks requiring cross-verification and multi-source synthesis. As shown in Appendix F, we have conducted a case-based analysis of agent behaviors in the GAIS task. We agree that a more detailed analysis of interaction patterns is valuable. Our current findings indicate that:

• In the SFT stage, the model already learns effective decomposition and high-level planning patterns, such as structured sequences like "First ... Then ... Finally ...".

• In the RL stage, the agent further acquires more advanced behaviors such as cross-validation, reflection, and adaptive replanning. For the case in Appendix F, in a complex task involving ZIP code lookup, the agent first fails to find the result via the USGS database, but then adapts by issuing a second targeted search, ultimately succeeding, demonstrating robust handling of uncertainty.

Regarding failure modes and current limitations, we acknowledge that longer interaction turns introduce long-context challenges in our proposed WebDancer, which can affect coherence and memory. As part of future work, we are exploring Long-CoT to Short-CoT condensation and summarization techniques to mitigate these issues.

We will expand this discussion and include additional case studies in the revised version to further clarify the evolution of interaction strategies and the impact of training on agent sophistication.

We sincerely appreciate your valuable suggestions, which are highly helpful for improving our work. We will incorporate all of them in the revised version. Please feel free to raise any additional concerns.

[1]. OAgents: An Empirical Study of Building Effective Agents

[2]. WebThinker: Empowering Large Reasoning Models with Deep Research Capability

[3]. Humanity's Last Exam

[4]. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents