AAAR-1.0: Assessing AI’s Potential to Assist Research

We sincerely appreciate your detailed review and comments. Below, we provide our comprehensive responses to your questions.

---

Q1. Why LLMs fail on EQINFER task & provide the failure patterns discussion.

EQINFER leverages a challenging binary inference setting, where LLMs are forced to examine each option separately rather than relying on superficial shortcuts from multiple-choice QA (please refer to the Q1 of Reviewer qTbh for more detailed discussion). According to our observations, most LLMs tend to predict any given equation as correct (a common error pattern for most LLMs) without performing in-depth reasoning over the paper context. Thus, we assume that the inferior performances of most LLMs result from their limited long-context reasoning capacity.

To verify our assumption, we ran the OpenAI o1 model with varying levels of reasoning effort.

The o1’s performance consistently improves as we increase the reasoning effort. Although this is a simple empirical verification, we believe it highlights the importance of reasoning capacity for this task, particularly for open-source LLMs.

---

Q2. Heavy reliance on automatic metrics for evaluation.

Thanks for your suggestions. We proposed task-specific automatic metrics for all the tasks in AAAR to ensure that the public can easily and efficiently reproduce the experimental results reported in our paper.

At the same time, we agree that human judgment is also valuable. As shown in Tables 3 and 4, we conducted a small-scale human evaluation for the model-generated experiment ideas, where we found that despite a few unexpected evaluation results (i.e., false negatives), the results from the automatic metrics generally align well with human judgments, especially when compared with conventional generation metrics like ROUGE. This suggests the reliability of the proposed automatic metrics.

---

Q3. Discussion on computational efficiency.

Apologies for missing this detail. When running the open-source LLMs on our local machine, we used vllm to accelerate computational efficiency. Given 4 NVIDIA A100 GPUs for LLM inference, the largest model we utilized in our experiment, Qwen-72B, took approximately 1.25 hours, 0.4 hours, and 1.75 hours for EQINFER, EXPDESIGN, and WEAKNESS, respectively. All the running hyperparameters, such as the maximum model length, can be found in our paper.

In our next version, we will include more details about the computational costs of various LLMs on AAAR.

---

Q4. Would AAAR generalize to other fields?

Yes. Our proposed data collection method can ideally be generalized to other disciplines, with AAAR serving as a representative benchmark in the AI/ML field. However, the main constraint is still the requirement for domain experts, as recruiting a reliable and large annotation team is extremely expensive for this kind of research benchmark (see the Q2 discussion of Reviewer qTbh).

---

Q5. Could incorporating multi-turn dialogue improve the performance of LLMs in research critique tasks?

We assume so. To our knowledge, multi-turn dialogue context can be seen as a structured reasoning path, potentially benefiting LLMs in complex tasks (based on our experience). Meanwhile, some studies have explored dialogue-based collaboration between humans and LLMs or among different LLMs, showing that intra-perspective message sharing can enhance performance in reasoning-intensive tasks [1][2].

---

References:

[1]. Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration. (arxiv 2025)

[2]. Chain of Agents: Large Language Models Collaborating on Long-Context Tasks. (NeurIPS 2024)

Your comments are very much appreciated! We took your comments carefully and tried to address them one by one.

---

Q1. The critical references that were missed (i.e., CycleResearch and DeepReview).

Thanks for highlighting these important concurrent works. Specifically, we ran CycleResearch on our benchmark during the rebuttal.

The table above presents the WEAKNESS results, where the 70B CycleReview model (based on LLaMA-3.1) achieves an S-F1 score nearly comparable to GPT-4o, highlighting the benefits of post-training for open-source LLMs on this task. However, CycleResearch's ITF-IDF score (the proposed diversity metric) remains similar to LLaMA-3.1 due to a lack of specificity—a common error pattern of LLMs discussed in our paper.

The table above presents the EXPDESIGN results. In our view, CycleResearch may not be a suitable baseline for this task, as it is a policy model designed for whole-paper writing rather than specializing in experiment design. Consequently, applying it to our task requires modifying its original system prompt and task objective, which could make the comparison unfair.

We will provide further observations on CycleResearch in our next manuscript.

---

Q2. The definition of “assisting research”.

In terms of 'AI for research', our benchmark differs from existing works in two key aspects.

• i) The scope of 'research': Since research activities are broad and diverse, we focus on domain-specific, expertise-demanding, and reasoning-intensive tasks that highlight the irreplaceability of researchers. For example, writing experimental code is a reasoning-light task, often done by students, whereas determining 'what experiments are necessary' to support a paper’s primary claim is an expertise-demanding task, typically decided by senior advisors. The latter is clearly more challenging and demonstrates that AI models cannot easily replace senior researchers.
• ii) 'Assisting researchers' rather than 'replacing researchers': For high-level research tasks, our benchmark primarily serves an educational purpose — LLMs assist junior researchers by offering imperfect yet insightful ideas rather than governing the entire research process [1]. Relying on LLMs to oversee research and replace human effort compromises academic integrity. For example, we can use LLMs to suggest weaknesses as feedback to help us refine our own manuscripts, rather than directly using model-generated comments for peer review.

---

Q3. Reason for choosing the three tasks and their connection with the real-world scenario.

As addressed in Q2, our benchmark focuses on expertise-demanding research tasks that highlight the irreplaceability of researchers. Though more tasks could be included, we prioritize those that are widely underestimated in existing works. For example, while writing a paper review is well-studied, identifying a paper’s weaknesses is significantly more challenging than writing a paper summary/strengths [2][3].

EQINFER relates to scenarios where LLMs assist in double-checking the correctness of our own equation writing. EXPDESIGN mirrors a PhD student seeking a professor’s advice before writing experimental code. WEAKNESS represents using LLM feedback to refine our entire research project. Each corresponds to a real-world scenario and strictly aligns with our 'assisting researcher rather than replacing researcher' perspective.

---

Q4. The limitation of the benchmark.

Thanks for your suggestion. We agree that discussing limitations would benefit readers and future research. In fact, we initially included a limitation discussion section in our manuscript but removed it to comply with the ICML submission format. Due to the rebuttal word limit, we have provided that section in this link and will aim to reintegrate it in future versions.

---

Q5. Details about random baseline and how we ensure the data collection is not biased.

For the PaperWeakness task (Table 5), we reported only human performance and did not include a 'random baseline'. Please refer to the Q2 of Reviewer qTbh for more details on how we ensured unbiased data collection.

---

References:

[1]. Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration. (arxiv 2025)

[2]. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. (NEJM AI 2024)

[3]. LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing. (EMNLP 2024)

Thanks for your efforts in reviewing our manuscript! We're glad you found our dataset interesting, the proposed evaluation metrics reasonable, and the experimental results useful. Below, we address your concerns in detail.

---

Q1. Problem for setting the 'ground truth' for EXPDESIGN and WEAKNESS.

This is a great point. While research is inherently 'open-ended,' human annotations still meet the regular acceptance standards of research — not a gold standard, but at least a reasonable one. Therefore, despite its limitations, using human annotations as ground truth remains a practical approach for benchmark construction, as evidenced by recent works that continue to adopt this methodology [1][2].

In this work, to mitigate potential bias from setting human annotations as the ground truth, we establish the evaluation framework that integrates both automatic metrics and human assessment. For example, in EXPDESIGN:

• First, we use an automatic metric to compute the En-F1 score (Table 2) and identify 'negative' predictions.
• Then, to quantify false negative judgments, experts manually assess the true correctness of these ‘negative’ predictions (Table 3).

To our knowledge, combining automatic metrics with further human assessment represents the 'best' current practice for evaluating research outputs.

---

Q2. Analysis for the false positive / false negative cases w.r.t. the 'ground truth'.

Thanks for your helpful suggestion. We have indeed made some analyses in our paper. Specifically, as shown in Table 3, for those model-predicted experiment ideas that do not match the human ground truth, we asked the human experts to evaluate manually.

We found a few model-generated experiment ideas that deviate from the ground truth but are deemed reasonable in manual checks (implying some false negatives). Meanwhile, our manual examination confirms that our rigorous peer review process ensures the ground truth's correctness and objectivity (indicating no notable false positives).

We hope these results can inspire future work on fairly evaluating research outputs without reliance on ground truth while maintaining efficiency and reproducibility.

---

Q3. Concerns on data contamination.

This is an important point to discuss. First, we believe that EQINFER and EXPDESIGN are less likely to be affected by data contamination, as the ground truth outputs for both tasks have been reformulated or rewritten from the original source. Output rewriting or distribution perturbation is a widely adopted method to mitigate data contamination [3][4].

However, we acknowledge the potential data leakage issue in the WEAKNESS task, as all outputs are directly taken from OpenReview (though we believe this is a common challenge faced by most current benchmark datasets [5]). We maintain that our experimental results still offer valuable insights and can serve as an upper bound for certain LLMs, especially if they were pretrained on papers from OpenReview.

Notably, this work introduced AAAR-1.0. Our ongoing efforts involve collecting confidential data that no LLM has touched; we will include that as blind test sets in AAAR-2.0, our next version.

---

Q4. Concerns on unifying input length.

Since not all LLMs support long-context reasoning and different models have varying maximum context sizes, we standardized the input length for a fair comparison. We agree that it is crucial to ensure all input information is provided to the LLMs.

To address this, we conducted extensive context length analyses in our manuscript (pls refer to Appendix D1 --- 'Input Context Scaling Investigation'). For example, Figure 4 in the Appendix demonstrates that not all long-context LLMs benefit from the full input information.

---

References:

[1]. ResearchTown: Simulator of Human Research Community. (arxiv 2024)

[2]. LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing. (EMNLP 2024)

[3]. ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery. (ICLR 2025)

[4]. ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code. (ICLR 2025)

[5]. CycleResearcher: Improving Automated Research via Automated Review. (ICLR 2025)

Many thanks for your detailed and comprehensive review. We're pleased you found our evaluation criterion reasonable and the literature topic important. As shown below, we address your main concerns one by one:

---

Q1: Unclear evaluation setup for the Equation Inference.

We apologize for the confusion. Although EQINFER was originally designed for a MCQA setting (which explains the 1:3 positive-to-negative distribution), we ultimately adopted a binary inference setup, as shown in Figure 1 / Footnote 1. This change was made because of two reasons:

• i) We found that the MCQA setup contains shortcuts --- LLMs may focus on superficial character differences among the options rather than reasoning through the paper context. In contrast, binary inference requires LLMs to evaluate the correctness of each option separately, eliminating any straightforward shortcuts.
• ii) In real-world equation reviewing during paper submissions, the task also involves determining whether the author-written equation is valid based on the context. We are making our 'AI assists researcher' setting more realistic.

If you're interested, you can find the MCQA setup performances at this link. As shown, LLMs generally achieve higher scores due to the MCQA shortcut. We apologize again for the misleading MCQA prompt template in the appendix and will update it in our next manuscript.

---

Q2. The reliability of the employed annotators.

Thank you for pointing out this critical argument. We will include the following recruitment details in our next version:

• Recruiting: we posted an online recruiting form with strict qualification requirements: i) more than 4 years of AI research experience; ii) more than 5 publications, with at least one first-authored publication in leading AI venues; iii) reviewed 10+ peer publications. The form was then shared through social media.
• Annotator Profile: over 10 annotators were selected, all from renowned academic institutions with strong ML/AI research backgrounds (not just five students), including some professors.
• Domains: we aimed to include annotators with diverse AI expertise, including NLP, CV, and ML theory.
• Reward: given the challenge of collecting expertise-level data, we offer a high payment of \$70 per hour to all annotators. Additionally, they receive a \\$5 bonus for each low-quality sample identified or valid response made during peer discussions.

We will follow your suggestion to tone down the term 'senior researcher' in the next version. It is important to emphasize that we have made every effort to ensure all annotators have a strong background and are fully engaged in rigorous data collection and examination, which was also acknowledged by Reviewer WuHG.

---

Q3. Performance of adding few-shot examples.

This is a good suggestion. Here, we provide the performance of adopting the few-shot examples.

The above table illustrates the results of EQINFER, we found that adopting few-shot examples only improves the F1 score of smaller open-source LLMs. We believe the examples mainly serve as format guidance for classification in this case, while the reasoning ability of LLMs plays a more critical role in EQINFER (see our Q1 discussion with Reviewer xpyH).

The above table illustrates the results of WEAKNESS, we found that few-shot examples even negatively impact the ITF-IDF scores of various LLMs. Our observation suggests that adding examples restricts LLMs' creativity in generating novel weaknesses. For instance, if we include 'lack of novelty' as an example weakness, LLMs tend to repeat this weakness across different papers.

For EXPDESIGN, due to the list format generation, this is the only task where we observed significant and consistent performance improvement after adopting few-shot examples (which we’ve already used in our paper, as seen in Figure 12).

---

Q4. Missing references.

Thanks for suggesting highly relevant references. We will include a discussion on them in our next version. Additionally, we have conducted further baseline experiments to make our evaluation more comprehensive (please see our Q1 discussion with Reviewer aPKE).

---