Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation

Thank you for your thoughtful review. We address your concerns below:

---

Information on the custom-created datasets

"...The descriptions, and construction method should be clearly stated in the main text. However, I can not find this...."

"Could you provide comprehensive information on the four custom-created datasets, including data collection methods, annotation processes, and validation protocols?"

In our current paper, key information about four custom-created datasets is already presented in Section 4.1 and Figure 2 in the main text. The construction methods are detailed in Lines 220-230 and illustrated in Figure 2. More detailed information is presented in Appendix A. We will enhance clarity in our revised paper and guide readers to the corresponding sections.

“Additionally, will these datasets be made publicly available to facilitate reproducibility and further research?”

Yes, the datasets and code will be made public upon publication.

Clarification of “Adapting While Learning”

"... However, I can not understanding why World Knowledge Learning and Tool Usage Adaption can be summarized as adapting while learning. I seems to adapting while inference?"

"Adapting While Learning" refers to adaptation occurring during the knowledge-learning process. The "learning" refers to the World Knowledge Learning phase where the model learns scientific knowledge through tool interactions.

Both adapting (tool usage adaptation) and learning (world knowledge learning) are built into the loss term of the fine-tuning process (Eq 6), not at inference time. If one were to create "adapt while inference," they would need additional procedures (e.g., Monte Carlo Tree Search) during inference, which is not the case in our framework.

Novelty of our work and relationship with previous literature

"The world knowledge learning and tool usage adaptation are not novel. The first one is very similar to SFT."

There appears to be a misunderstanding. Our innovation is NOT about SFT technology, but in adapting it to enable a new LLM-based learning paradigm for scientific problem-solving: learning knowledge from scientific simulators and learning to decide when to seek help from tools depending on problem complexity. Similar to how human scientists approach a problem, our training paradigm enables LLMs to use scientific simulation tools adaptively. We demonstrated strong empirical performance in several scientific domains including climate science, epidemiology, etc.

“The tool usage learning method is also largely borrowed from 'Learning to Use Tools via Cooperative and Interactive Agents'”

Our work is fundamentally different from theirs. The work of Shi et al. (2024) primarily focuses on constructing datasets to enhance tool usage ability, i.e., they only focus on tool calling, which can lead to over-reliance on tool calls (see our results in Table 2). This potential over-reliance is actually a motivation for our development of this work.

Definition of Tool Usage Accuracy

"The metric for tool usage accuracy is not clearly defined."

"How do you define and measure 'tool usage accuracy'? ..."

Tool usage accuracy is clearly defined in Section 4.3 (Lines 321 left col. - 303 right col.) in the main text and we provide potential 5 additional tool usage metrics in Appendix E. This metric assesses the model's ability to choose correctly between internal reasoning and tool usage. We can simplify our writing to better guide readers in our revised version.

Easy/Hard Problem Classification

"The criteria for classifying problems as 'easy' or 'hard' are not thoroughly explained."

"What specific criteria or thresholds do you use to classify problems as 'easy' or 'hard' within the TUA component? How consistent are these classifications across different domains and datasets?"

We describe this classification in Lines 186-193 and provide additional details for open-ended questions in Lines 248-254. Questions are classified as "easy" or "hard" based on the model's direct-answer accuracy—if a model answers correctly without tools, it's "easy"; otherwise, "hard." This classification is dynamic across datasets and models, but the rule for dividing is consistent. We will simplify our writing for a more concise and clear information delivery in the revised paper.

References Not Discussed

Thank you so much for these suggested references. We will discuss them in the "LLM Tool Usage" paragraph in Section 2 of the revised paper. We kindly note that Gorilla is already included (Line 20, Line 100, right col).

---

Thank you again for your time and effort. We hope our response addresses your concerns. If the issues have been resolved, we’d appreciate your consideration in the evaluation. Please feel free to share any additional feedback.

Thank you for your invaluable feedback. We address your suggestions below:

---

Variations in loss functions

"May be they could have considered more variations in loss functions and also showed some result with some experiments why is the proposed loss function better than the existing one and what are the other possible ones."

We thank you for this suggestion. Our work focuses on defining a new paradigm for LLMs to learn to solve scientific questions adaptively; as long as that target can be fulfilled, the loss term definition (also, similarly, the specific finetuning frame) is orthogonal to our work. We kindly find this ablation is not strongly related to our work.

Increasing Problem Partitions

"Also increase the partition to not just easy/hard but to more sections."

This is a very insightful observation and a great suggestion for future work. In this work, we are the first of our kind to encourage LLMs to choose smartly when to use tools, hence the setting is simplistic and binary.

But surely, in realistic scientific or engineering cases, we can have different granularity of tools for different requirements of a given problem.

As noted in Lines 420–424 (right col.), we have already acknowledged this limitation and discussed the potential of extending the framework to support more fine-grained partitions in future work. We appreciate the reviewer highlighting this direction and will further discuss it.

Functionality of Prompt Tuning

"Also it would be interesting to see prompt tuning/prompt injection as a comparison where the initial query is considered and based on the tool usage prompt is updated/injected according to the tool usage."

We appreciate this suggestion. In our current paper, Appendix B already shows prompts instructing models (without finetuning) to decide tool usage based on question difficulty.

To further investigate the functionality of prompt tuning (PT) on these tasks, we added additional experiments using few-shot prompting with direct-answer and tool-usage examples. We show its performance compared with the base model and our method on answer accuracy and tool usage accuracy.

Answer Accuracy

Tool Usage Accuracy

The experimental results indicate that few-shot prompting does not consistently improve the model's performance and may even degrade it in most cases. We will include the above results as a baseline in both Table 1 and Table 2 in our revised paper.

---

Thank you again for your time and effort. Our paper benefits from your suggestions. We hope our response addresses your concerns. We welcome any further comments and are happy to address them.

Thank you for your thoughtful review. We address your concerns below:

---

Novelty and Methodological Depth

"The claim of novelty and methodological depth ... existing fine-tuning technologies."

We are not claiming novelty in fine-tuning technologies. Rather, our innovation primarily lies in defining a novel learning paradigm for scientific problem-solving with LLMs, i.e., learning knowledge from scientific simulators and switching to tools intelligently based on problem complexity, which fits human expert behavior in scientific domains. We are the first to propose this paradigm, and our empirical experiments demonstrate that our paradigm can strike a great balance between answer accuracy and tool calling cost.

Performance Consistency on Public Datasets

"...claims about performance advantages are inconsistently supported...", "...in Table 1...GPT-4o performs best on public datasets."

The performance difference on public datasets is largely due to model size, as the strongest baselines such as GPT-4o are significantly larger than our base model (8B). To support this, we additionally include 2 larger open-source models Qwen2.5-{14B/32B}-Instruct as base models for our method and conduct experiments on MATH and SciBench, with results presented below, where TUA means Tool Usage Accuracy:

It can be observed that our framework applied to a 32B model can outperform GPT-4o on MATH and achieve comparable performance on SciBench. We will append these results in our revised paper.

Dataset Splitting

"...the easy/hard split should vary across models, but it is unclear whether the authors use a fixed or dynamic split..."

Yes, our easy/hard splitting is relative to each model's problem-solving ability, i.e., dynamic. We described it in Lines 186-193 and provided additional details for open-ended questions in Lines 248-254.

"...splitting datasets...may lead to inconsistent splits for different models... lacks theoretical guarantees."

The intuition behind this design decision is that different models, by construction, have different levels of problem-solving ability; hence, a fixed split is actually counter-intuitive. If one were to define a fixed split, it would inherently introduce biases using the reference for such a split, regardless of whether that reference is a human expert or an LLM.

As for theoretical guarantees, may you clarify what kinds of theoretical guarantees you are referring to? In Section 3.3, we have already stated that our methods ensure models learn the correct decision patterns through supervised training objectives. Our experiments empirically verify this, which demonstrates that the model successfully internalizes these patterns and applies them consistently. Extra theoretical analysis is rare and trivial in NLP application works.

Robustness to Format and Dataset Shifts

"...its reliability diminishes when the dataset changes or when the question format shifts (e.g., from multiple-choice to direct QA)."

Our datasets already include numerical problems (Lines 231-241) and open-ended questions (Lines 241-247, Figure 3b), which are both direct QAs. Experimental results on these questions are consistent with those on multi-choice questions. It demonstrates our method's robustness to problem format shifts.

Experiment Settings

"...which setting ($P\_n$, $P\_i$, $P\_f$) is used for other models."

In Table1, Llama3.1-70B-Instruct, GPT4o, GPT4o-mini, Claude3.5-Sonnet are vanilla models, i.e., they were evaluated without tool assistance ($P\_n$). We will clarify this in the final version.

Relevance of Tool Usage Metric for Lightweight Tools

"Not all scientific tools are resource-intensive (e.g., Python scripts)..."

We agree that not all scientific tools are resource-intensive, but many of them are, especially in realistic scientific and engineering applications.

These applications, such as climate modeling, and epidemic prediction, would require much more significant computing time than LLM inference (e.g., days for simulation vs. seconds for LLM inference), due to inherent complexity (such as high spatial resolution). Our work targets those use cases and serves as a starting point to apply LLM automation.

Apart from those cases, we also incorporated commonly used benchmarks such as MATH and SciBench, and Python is the most commonly used tool for them. The strong performance of our method shows that our pipeline is applicable to any tools, regardless of their cost.

---

Thank you again for your time and effort. We hope our response resolves your concerns and would be grateful for your consideration in the evaluation. We welcome any further comments.

Thank you for your detailed review and constructive suggestions. We address your concerns below.

---

Validation on Larger Models

...limited to models other than Llama and large-scale models(10B+)

We additionally included Qwen2.5-14B-Instruct and trained it with our method. We conducted experiments on 2 custom and 2 public datasets. We present models’ answer accuracy under $P\_n$ and $P\_i$, Tool Usage Accuracy (TUA), and Tool Use Ratio (TUR), which corresponds to Table1, 2, 13 in the paper respectively.

Results show consistent gains in answer and tool usage accuracy, with reduced tool use. It shows our method also benefits larger models.

Problem Difficulty Hyperparameters

...a sensitivity analysis of hyperparameters (especially the problem difficulty threshold)...

We conducted a sensitivity analysis on the number of samples (k = 1, 3, 5) used to assess LLMs' answer accuracy and partition questions by difficulty. We trained different models with these thresholds on MATH and SciBench datasets, with the results presented below.

Answer Accuracy (pass@k)

Tool Usage Accuracy

Results show that our method remains stable across different hyperparameters about the problem difficulty threshold.

Computational Cost Reduction

...a quantitative analysis of computational cost reduction...

Currently, Tables 4 and 13 have already presented the tool use ratio of different models and showed that our method can reduce unnecessary tool usage thus reducing computational cost.

Additionally, we conducted an analysis on open-ended problem-solving which allows multiple tool usage. The numbers in the table represent the average times of tool usage across all questions. It further demonstrates that our method brings computational cost reduction.

Error Analysis

...error analysis and case patterns...

We add a new section on error analysis. We categorize errors into: 1. Problems unsolvable even with tool usage (agent limitation); 2. Problems solvable with tools but answered incorrectly. The latter category is further divided into: a. Calculation mistakes, b. Reasoning errors, and c. Knowledge gaps.

Following standard practices in benchmark papers, we provide GPT-4o with the errors and their corresponding correct solutions and ask it to annotate their error types. The error type distributions are read below.

Base model

Our trained model

After training, the proportion of Reasoning Errors and Knowledge Gaps decreased and the proportion of Agent Limitation increased. This suggests that: 1. Our method enables the model to learn scientific reasoning and domain-specific knowledge; 2. The remaining errors after training are mainly concentrated on questions that cannot be correctly solved even with tool usage. This analysis points out the key challenges that future work should address.

Potential Knowledge Transfer

Cross-domain generalization isn't the primary focus of this work. We have discussed it in our future work section (Line 416-420, right col.). We will further highlight it.

Application Descriptions

We have provided examples of application cases and usage scenarios in Figure 2 and Appendix A (Pages 12-18). We’ll clarify application relevance in revisions.

Human Evaluation for Open-ended Questions

For these questions, our evaluation is based on the simulation result given the LLMs' proposals, which is intrinsically stable. Manual reviews are out of the scope of this work, but we will include some discussion.

---

Thank you again for your time and effort. We hope our response has addressed your concerns. If the issues have been resolved, we’d appreciate it if you could reflect it in your evaluation. We welcome any further comments.