Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger

We thank the reviewer for the constructive comments and we have taken the precious advice and made improvements in the revised paper.

Weakness-1: No clear definition and examples for the self-defined concept.
A. See the response in A1.

Weakness-2: MeCo is outperformed by fine-tuning. Although MeCo is fine-tuning-free, but does not compare the overhead of fine-tuning and MeCo.
A. MeCo and fine-tuning are orthogonal approaches, with MeCo capable of being applied to both raw and fine-tuned models, providing additional benefits to the fine-tuning approach. Intuitively, MeCo can be seen as a plugin that enhances adaptive tool use and RAG accuracy by leveraging the model's internal awareness. Importantly, MeCo integrates seamlessly with LLMs, adding negligible latency. In contrast, fine-tuning faces two primary challenges:

1. Data dependence and scaling effect: The performance of fine-tuned models heavily depends on the availability and quality of the fine-tuning dataset, which is often costly and not always accessible. This requirement can limit the applicability of fine-tuning, especially in environments where such datasets are scarce or incomplete. Besides, the marginal utility of fine-tuning decreases as the size of the fine-tuning data increases, leading to exponentially increasing costs for slightly better performance.
2. Limited transferability: Fine-tuned models often underperform in "out-of-distribution" testing scenarios. For instance, in Table 2, the fine-tuned Llama-3-8b showed degraded performance on MeCa-Tool Tasks 2 and 3, which involve "neural" queries that are absent in the fine-tuning data. In contrast, MeCo avoids these limitations, offering consistent performance across diverse scenarios.

We provide the training overheads of both approaches in our experiment. Fine-tuning Llama-3-8b/Mistral-7b required ~4 GPU hours with 4,000 fine-tuning examples on A100 GPUs. In contrast, training the meta-cognition probes and fitting the thresholds for MeCo took ~20 minutes per task. Therefore, MeCo's overhead is significantly lighter than fine-tuning.

MeCo's generality, low overhead, and ability to complement fine-tuning, make it a practical and scalable solution for real-world applications.

Weakness-3: Insufficient experiments. 1) small backbone models. 2) CoT is not clearly explained.
A. See the response in A3.

Q1. Please give a more detailed definition of Meta-cognition and put it in a more prominent position. It would be best if you could provide examples and quantitative indicators.
A1. We thank the reviewer for this valuable suggestion. In response, we have included a definition of meta-cognition, along with an illustrative example, at the beginning of Section 3 in the revised paper. We kindly invite the reviewer to refer to the revised paper for these details.

Q2. Please provide experiments on larger models (e.g. 70B level)
A2. Thank you for the suggestion. We have included Llama-3-70b-Instruct as another backbone model in our experiments. Please refer to Table 1 in the revised paper for the results.

Q3. Please provide the COT prompt used in the experiment and the method of how to get "Yes" or "No" from LLM's output.
A3. Thank you for raising this question. We have added the CoT prompting details in Appendix C.1 of the revised paper. The model is instructed to think step-by-step, reasoning whether it needs external tools to address the user query, and finally concludes with "Yes" or "No." We detect the final "Yes/No" answer by searching for an exact match in the last 150 words of the response. The matching signals for "Yes" and "No" are as follows:

"Yes": 1) I(i)t is necessary to use; 2) I(i)t is essential to use; 3) will need to use; 4) external tool is necessary; 5) I need to use the external; 6) etc.

"No": 1) I(i)t is not necessary to use; 2) I(i)t is not essential to use; 3) will not need to use; 4) the external tool is not necessary; 5) do not need to rely on; 6) etc.

Through close human examination, we found that CoT suffers from reasoning inconsistency, a challenge also reported in the literature $1,2,3$ . Specifically, LLMs sometimes generate the correct answer following an invalid reasoning path or produce a wrong answer after a correct reasoning process, leading to inconsistency between the derived answer and the reasoning process. In contrast, the "Yes/No-Explanation" prompting strategy does not suffer from this reasoning inconsistency in our experiments, thereby achieving better performance compared to CoT.

$1$ Wei et al. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS 2022.

$2$ Lyu et al., Faithful Chain-of-Thought Reasoning. The 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics 2023.

$3$ Zhao et al., A Survey of Large Language Models. arXiv preprint arXiv:2303.18223.

We sincerely appreciate your valuable feedback and constructive engagement throughout the review process. Below, we address your questions in detail and welcome any further concerns you may have.

---

Q1. The improvement in fine-tuned models is more critical. The improvement of MeCo is relatively limited.
A1. We appreciate the reviewer’s perspective that improving fine-tuned models is crucial for real-world applications. We would like to emphasize that MeCo consistently enhances the performance of fine-tuned models, as demonstrated in the tables below. Notably, as a fine-tuning-free, easily integrable, and plug-and-play method, MeCo achieves considerable improvements without introducing substantial latency or training costs. This balance of effectiveness and efficiency highlights MeCo's practical advantages.

Additionally, as widely acknowledged in the research community, the marginal benefits of fine-tuning diminish as the size of the fine-tuning dataset increases. This diminishing return further underscores the value of MeCo: it complements fine-tuned models by delivering additional gains without incurring the high costs typically associated with further fine-tuning.

Thus, MeCo represents a practical and efficient solution for improving tool-use capabilities in both pre-trained and fine-tuned LLMs.

Table 1. Results on Metatool.

Table 2. Results on MeCa-Tool.

---

Q2. The CoT prompt used in the experiment can be further improved, which will lead to a certain degree of accuracy decline. Of course, I also admit that the effect of CoT is related to the model capability.
A2. While we agree that the CoT prompt could be refined, CoT suffers from two significant limitations that prevent it from being an ideal solution for adaptive tool use:

1. Reasoning inconsistency: As mentioned in our previous response, CoT often produces inconsistencies between the reasoning process and the final answer $1, 2, 3$ . Even with carefully designed prompts, CoT has also been observed to negatively affect tool use performance $4$ .
2. High latency and inference costs: In our experiments, CoT prompting takes significantly more time (3–4 times longer) than the Yes/No+Explanation prompting strategy. This increased latency and computational cost make CoT less suitable for real-world applications where efficiency is paramount.

These drawbacks highlight the practical challenges of CoT, which go beyond prompt optimization.

$1$ Wei et al. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS 2022.

$2$ Lyu et al., Faithful Chain-of-Thought Reasoning. The 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics 2023.

$3$ Zhao et al., A Survey of Large Language Models. arXiv preprint arXiv:2303.18223.

$4$ ToolACE: Winning the Points of LLM Function Calling. arXiv preprint arXiv:2409.00920.

We sincerely appreciate the reviewer’s careful and thorough review. We have addressed the raised issues in the revised paper and kindly request the reviewer to consider our responses below:

1. Overview Figure: The overview figure is straightforward, and we have discussed the pipeline and logic in the introduction and approach sections. Specifically, the determination of thresholds is described in lines 91, 208, and 381 of the revised paper.

2. Fine-Tuning Mention: Fine-tuning is not the core contribution of our paper and is thus not highlighted in the abstract. We implement MeCo on top of fine-tuned models because LLMs are typically fine-tuned to improve performance on downstream tasks, such as adaptive tool use in our paper. This aligns with standard evaluation practices in the literature.

3. Figure Visualization: Thank you for the suggestion. We have revised the figure for better visualization.

4. MeCa Benchmark Refinement: Post-submission, we continued refining the MeCa benchmark (details provided in Section 4 and Appendix A), increasing its scale, and incorporating more complex and realistic queries to better mimic real-world scenarios. The evaluation results on the updated MeCa benchmark are presented in Table 2 of the revised paper.

5. Larger Backbone Model: We have included Llama-3-70b-Instruct as another backbone model and presented the results in the revision. While time constraints prevented us from replicating all experiments on MeCa with Llama-3-70b, we plan to complete this comprehensive evaluation in the next revision.

6. 1) Thank you for identifying the errors; we have fixed them in the revision. 2) The performance margin decreased after fine-tuning because the model's awareness of tool use significantly improves after fine-tuning, reducing the additional benefit brought by MeCo. However, we want to emphasize that MeCo still provides consistent and notable improvements to the fine-tuned models across two benchmarks and various testing settings. Additionally, MeCo is a fine-tuning-free approach that is easily integrable with any LLMs, offering substantial benefits without incurring high costs.

7. Thresholds Derivation: We have discussed the derivation of thresholds in the third paragraph of Section 6.1.

8. Ablation Study: We believe that an ablation study is not needed for the evaluation of MeCo.

We sincerely appreciate the reviewer’s careful and thorough inspection. We have addressed the issues, oversights, and references in the revised paper and would like to provide clarifications on the following two points:

1. PCA explanation and reference. We have cited PCA in the revision. PCA is an unsupervised technique primarily used for dimensionality reduction. It identifies the principal components that capture the maximum variance in the data, thereby enabling the learning of principal reading vectors. This method helps in simplifying the data while retaining its most important features.

2. Figure 2 explanation and reference. We’ve referenced Figure 2 in the revised text. Figure 2 provides a visualization of the pipeline for training the meta-cognition probe and other types of probes. Sections 2 and 3 thoroughly describe the procedures involved in the training process.

Q1. Figure 1: Why not use a tool, if the initial decision is to not use a tool, but it also beyond the ability of the LLM?
A1. Sharp observation; this is indeed a typo in our paper. We have corrected it in the revised version. Please refer to the updated pdf.

Q2. section 3.2: The discussion about the probe selection seems rather fuzzy, since detailed results are not shown. How does the selection process generalize to other models?
A2. The selection process generalizes to all three backbone models used in our paper. While we omitted detailed results in the manuscript for conciseness, we conducted extensive experimentation to systematically evaluate various probes and determine which yielded the best final performance. Our findings consistently showed that probes with higher intermediate classification accuracy contribute to better final decision accuracy. For instance, a probe with an intermediate classification accuracy of 99% typically improves final decision accuracy by 1-2% compared to probes with 95-98% accuracy.

Q3. section 5, baselines: Where do P(Yes | Prompt) and P(No | Prompt) come from? Token probabilities of the respective model?
A3. Correct. We used the logits of the first token in the response from the respective model.

Q4. Will the source code be publicly released?
A4. Yes, we will release the source code soon. We aim to further refine and enrich the MeCa benchmark by incorporating more complex scenarios as suggested by some reviewers before making it publicly available.

We thank the reviewer for the positive support and constructive comments.

Weakness-1. Simplified benchmark.
A. Our updated results in Table 2 of the revised paper should address this concern. Post-submission, we continued refining the MeCa benchmark (details provided in Section 4 and Appendix A), increasing its scale and incorporating more complex and realistic queries to better mimic real-world scenarios. MeCa now includes 7000 new queries specifically designed to assess adaptive tool usage across six tasks and incorporates various scenarios and interaction lengths, such as multi-turn dialogues.

Weakness-2. Underexploration of meta-cognition, where it could fail and lead to sub-optimal decisions.
A. As shown in Figure 5, a clear gap between the meta-cognition scores of correct and incorrect responses allows our decision-making strategy to effectively differentiate correct from incorrect Yes/No decisions. While we acknowledge that MeCo may occasionally misclassify a correct response due to inaccurate meta-cognition feedback, particularly for ambiguous queries, this potential error is outweighed by the significant benefits of MeCo, as demonstrated in our empirical results.

Weakness-3. Lack of comparison to other adaptive algorithms.
A. See the response in A4.

Weakness-4. Scalability concerns and lack of experiments in real-world queries?
A. We have conducted experiments with the Llama-3-70b model and validated the effectiveness of MeCo on the updated MeCa benchmark, which incorporates more complex and realistic queries. Please refer to the global response and the revised paper for the updated results.

Q1. How does MeCo handle real-world scenarios with complex or ambiguous questions? Were any experiments conducted in more open-ended, unstructured environments, and if so, what were the results?
A1. We have refined the MeCa benchmark to incorporate more complex and realistic queries (over 7000 queries). Please refer to the global response and the revised paper for the updated results.

Q2. Can the meta-cognition scoring approach manage ambiguous queries that may require partial or iterative tool engagement? If not, how does the system handle such edge cases?
A2. Partial or iterative tool use scenarios are effectively managed within our evaluation framework, as they essentially involve sequences or combinations of single tool use decisions. All queries are processed using the decision-making strategy illustrated in Figure 1. In our experiments, we treat all queries equally based on their meta-cognition scores without applying specific decision rules for ambiguous cases.

We acknowledge that boundary cases may require more sophisticated handling. For such edge cases, a hybrid decision strategy (e.g., combining MeCo with rule-based methods) might improve the user experience. While the development of such hybrid strategies is beyond the scope of this paper, it represents a promising direction for future research. Our contributions, particularly the adaptive tool use framework and meta-cognition system, provide a strong foundation for future advancements in this area.

Q3. Have the authors tested MeCo’s scalability in more diverse and high-stakes environments where model latency or tool usage frequency could impact outcomes significantly?
A3: Currently, no established benchmarks specifically target high-stakes environments. MeCa represents the most comprehensive benchmark currently available. While we recognize the importance of such high-stakes environments, the development and standardization of relevant benchmarks are ongoing. Future work will aim to assess MeCo in these more demanding settings as appropriate benchmarks become available.

Q4. How does MeCo compare to other adaptive approaches like reinforcement learning-based or rule-based systems? Were any such methods considered for direct comparison, especially for efficiency or accuracy?
A4. Existing frameworks in the literature indiscriminately rely on external tools without adaptiveness. To our knowledge, there are no reinforcement learning-based or rule-based approaches designed for adaptive tool use in this context. Therefore, such comparisons were not included in our experiments

Q5. Could the authors clarify any limitations they see in the MeCa benchmark? Are there aspects of meta-cognitive performance that the benchmark doesn’t capture, and are there plans to address them?
A5. Post-submission, we have improved the MeCa benchmark and will keep refining it with more complex and realistic scenarios. A few potential tasks are:

1. Tool Costs and Risks: Incorporating varying costs and risks associated with different tools, as highlighted by Reviewer-AwNJ.
2. Multi-round and Multi-tool Interactions: Adding scenarios with multi-round interactions with dependencies between tool invocations. This will significantly increase the benchmark’s complexity and relevance to real-world scenarios.

We appreciate the reviewer's insight and will work towards integrating these considerations in future iterations of our research.

Weakness-1. Technical Contribution is low.
A. Our contributions are substantial and extend beyond mere technical contributions.

First, we introduced the adaptive tool use framework, a concept that has been largely overlooked in both academic and industry communities. This framework addresses a critical gap by focusing on the strategic timing of tool usage, a crucial aspect in real-world applications. Moreover, we are the first to integrate adaptive tool use and adaptive RAG within a unified framework, and provide a unified strategy based on our proposed MeCo approach.

Second, we empirically demonstrate that MeCo significantly enhances the model's awareness in adaptive tool use and RAG with minimal training required. Meanwhile, our analysis shows that MeCo provides stronger signals and higher accuracy, capturing a model's internal cognition more effectively than existing metrics like honesty and confidence. Moreover, we advanced the methodology for utilizing detection scores and trained probes (as described in Section 3.2).

Third, we introduced a new benchmark, MeCa, which greatly expands the existing Metatool benchmark (which contains 1040 queries). MeCa includes 7,000 new queries specifically designed to assess tool usage across six main tasks and incorporates various scenarios and interaction lengths, such as multi-turn dialogues. Moreover, we added 300 queries related to adaptive RAG, allowing MeCa to evaluate the effectiveness of both adaptive tool usage and RAG within a unified framework.

Weakness-2. The decision mechanism is simplistic and not principled. Different tools are viewed equally.
A. We have validated the effectiveness of the simple decision mechanism employed by MeCo through extensive empirical results across multiple benchmarks and various evaluation settings. Additionally, we conducted experiments with three backbone models: Llama-3-8b, Llama-3-70b, and Mistral-7b, demonstrating the generality of MeCo across various LLMs. Instead of being a shortcoming, we believe the simplicity and generality of MeCo should be seen as an advantage, allowing for easy and fast integration with any LLMs.

The notion of different tools having different costs or risks is indeed interesting, and we will incorporate this perspective in further developing the MeCa benchmark. As the pioneering paper introducing the concept of adaptive tool use, we initially treated various tools equally in the evaluation to establish a baseline. It is worth noting that this approach is consistent with most existing literature on LLM tool use, where tools are generally treated equally. We believe that addressing tool differentiation is a valuable direction for future work, to be considered once the adaptive tool use framework has gained wider acceptance.

Q1. How does detecting "meta-cognition" differ technically from detecting these other concepts? The paper shows strong empirical results - is this because meta-cognition is particularly well-suited to RepE compared to other concepts? Are there any such insights in the results?
A1. Detecting "meta-cognition" differs from detecting honesty or confidence in two key aspects:

1. Training data: Unlike honesty and confidence probes, which rely on factual statements independent of user queries, meta-cognition requires task-specific data that reflects the model’s self-awareness of its capability to address queries. This is especially important for tasks involving external tool use. Our training data consists of user queries related to tool use, accompanied by the corresponding decisions and explanations. Detailed data construction and contrastive instructions are provided in Section 3.1 and Appendix D.3.
2. Methodology: We have refined the methodology for utilizing meta-cognition scores. Rather than using the common mean-pooling approach, we focus on scores with the highest intermediate classification accuracy (as described in Section 3.2, "Reducing n to 1"). This provides a more accurate reflection of the model's self-awareness, improving the precision of meta-cognition score utilization and further enhancing the effectiveness of our method.

Our empirical results show that meta-cognition consistently outperforms concepts like honesty and confidence in detecting a model’s internal awareness of its capabilities and limitations. As shown in Figure 3, meta-cognition scores yield higher intermediate classification accuracy, indicating meta-cognition more effectively captures the cognitive processes for decisions on adaptive tool use and RAG. Additionally, Figure 5 highlights a clear gap between the meta-cognition scores for correct and incorrect responses, which our decision-making strategy exploits to discern correct from incorrect Yes/No decisions. For further analysis on why meta-cognition outperforms, see Appendices C.3 and C.4.

Dear Reviewer AwNJ,

In the above responses, we have tried our best to answer your questions and resolve your confusion. As the rebuttal deadline is approaching, we are very willing to have a more in-depth discussion with you, and we welcome you to give us more suggestions. If you have additional suggestions, please let us know, and we will try to respond as quickly as possible.