ToolTalk: Evaluating Tool Usage in a Conversational Setting

Thank you for your review and constructive feedback. See our response below:

The scope covered by the proposed benchmark is too narrow, compared to recent work on benchmarking the tool-using ability. Although I understand the authors may want to have a focus contribution to this specialized direction (a.k.a, conversation), the difference with other work is not that clear.

As you pointed out, we focus on contributing to a specialized direction, conversation. We chose to do so since we are not trying to replace existing benchmarks for tool-using LLMs, but rather fill a gap apparent in existing work. Specifically, existing benchmarks fail to measure performance on complex tasks that require multiple utterances from a human to specify such as in a conversation. While this gap may seem narrow, it is of particular importance since we can expect most humans to interact with tool-using LLMs in this way, and indeed many already do such as ChatGPT, Anthropic's Claude, or Microsoft's Bing Chat.

For example, a lot of work recently tried to evaluate the tool-using ability to solve real-world tasks. See [1] for an example. In this case, real-world tasks can always happen during the conversation.

While it is true real-world tasks often do occur during a conversation, a single real-world task does not necessarily encapsulate the entire goal of a user for the conversation. For example, [1] makes use of some popular mathematical reasoning datasets. We can imagine a scenario where a user wishes to answer a single math question in the conversation, in which case existing mathematical reasoning benchmarks might be sufficient. But we can also imagine scenarios where a user wishes to answer several, potentially dependent, math questions like a student, or even combine domains such as a user trying to balance dieting and a grocery budget. In this case, measuring performance on each individual instance outside of a conversation is less meaningful as failure to succeed at one or more instances would mean failure to accomplish the user’s end goal. Furthermore, specifying these user tasks in a conversation allows us to create complex tasks without having potentially long and unwieldy task descriptions. We find existing benchmarks do not capture these nuances of specifying tasks in a conversation resulting in the creation of ToolTalk.

There are also several differences when comparing ToolTalk to [1] in particular. For one, [1] improves the realism of existing benchmarks by simulating a human in the loop when an incorrect answer is provided. However, the benchmarks re-used still specify their tasks in a single utterance, and thus the “dialogue” provided by [1] is not used to specify additional tasks in a conversation instead allowing for the LLM to “retry” at a single task. Thus, the concept and usage of dialogue is completely different between ToolTalk and [1].

The evaluation is too limited. This is a fundamental drawback for this kind of benchmark paper. Basically, only two models, namely GPT-3.5-turbo and GPT-4, are considered in the experiments. It would be more interesting to see the results of some open-sourced models, like Vicuna, and Alpaca.

Thank you for your feedback, we were unable to evaluate more models due to time constraints. We are doing so for future revisions to our paper.

The overall presentation needs to be improved. For example, the two algorithm blocks in Page-5 do not provide too much useful information.

Thank you for your feedback, what did you find confusing and additional information would be useful?

References:

[1] MINT: Evaluating LLMs in Multi-Turn Interaction with Tools and Language Feedback. Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, Heng Ji.

I feel that this review does not fairly assess the contribution of the authors. While, indeed, the proposed dataset is small and focused on a specific problem, such datasets are crucial for developing truly capable digital assistants and there is a conspicuous dearth of such data even among a flurry of benchmarks assessing LLM tool use capability. Meanwhile, citing [1] as an example of more comprehensive benchmark is rather unfair and misguided given its focus on arithmetic reasoning, code generation and embodied agents, tasks orthogonal to the settings considered in the present work. Even more, the contribution of [1] is focused on curating existing datasets as opposed to contributing new data.

I also found points 2 and 3 overly critical. Additional evaluations on open source models are trivially achievable by the community. Having crisp and clear pseudo-code describing inference and evaluation procedures has definitely improved the reading experience and the clarity of the paper, for me.

I hope the authors are not dissuaded by the negative tone of this review and they will focus on scaling their work which definitely provide significant benefit to research groups focused on grounded dialogue.

Thank you for your review and constructive feedback. See our response below:

The number of involved tools is small, which may lead to biased evaluation for the LLMs measurement.

While the number of tools is small compared to recent works, we believe it is sufficient to show the shortcomings of GPT-4 and GPT-3.5. For example, both models tend to over-trigger, calling tools before all relevant information from the user has been supplied. This issue generally occurs for all tools though it is more consequential for action tools. While the occurrence of this problem might vary for any new, unseen tools, it seems safe to theorize that over-triggering is an issue not specific to any particular tool.

We also generally chose the tools to reflect conversations we might expect for personal productivity, a topic that is likely to receive a lot of use for tool-augmented LLMs. So even in the unlikely event many of the issues we found in our analysis are only specific to these kinds of tools, we can expect many people to still experience these issues when conversing with a tool-augmented LLM.

Ultimately, the current number of tools seems sufficient to demonstrate that evaluating tool-augmented LLMs in a conversational setting is a new task with unique characteristics that even some of the most state-of-the-art LLMs struggle to perform well on. However, adding additional tools is a clear target for future versions of ToolTalk,

There is a lack of experiments on open-sourced LLMs, and how to enhance the ability of LLMs on their proposed benchmark is missing.

Thanks for your feedback. We plan on evaluating more LLMs for future revisions.

Thank you for your review and constructive feedback. See our response below:

There is some lack of clarity about the creation of the ground-truth dialogs. It seems that it was written by humans, but how? Was there any validation of the data? Are the GPT4-generated scenarios biased in any particular way?

We will add additional details about ToolTalk’s creation in the revision. To clarify now, all conversations were written by the authors using the scenarios as general guidelines. Step by step, our process involves:

1. Reading the scenario, discarding it if it is unrealistic, does not contain the required tools, does not involve a minimum of 3 tool calls, or is too similar to an already existing conversation.
2. Creating dialogue to match the scenario along with tool calls and arguments.
3. Update tool databases as appropriate to contain information needed for the newly created conversation.
4. Validate ground truth tool calls using an oracle predictor to ensure that ground truth tool calls and accompanying database changes are achievable.
5. Validate dialogue to ensure ground truth tool calls and arguments can be realistically inferred from the proceeding conversation without ambiguity. We do so by evaluating GPT4 on the conversation and check that failures are not due to bad dialogue.

It's not clear whether the user is always requesting that specific tools are invoked (e.g., "search my email", "check the weather") or whether there are situations wherein the assistant is expected to infer tool usage from a passive statement (e.g., "I forgot who emailed me about X" --> search_email(X), or "John asked me to do X by tomorrow" --> add_reminder(X, tomorrow)). Additional examples would help

We plan on adding additional example visualizations in the revision. To clarify now, the assistant is generally expected to infer what tool to use. The creation of ToolTalk was done under the assumption that the user does not necessarily know the exact tools enabled by any particular plugin, and thus will not say things like “Use AddReminder to create a reminder to do X by tomorrow”.

Will the proposed methodology generalize to a tool such a search engine or general-purpose retrieval? I imagine the tool correctness metric would need to change.

Yes, the methodology should extend to search. We already implement some basic search tools such as searching for emails or messages. These tools have a tool correctness metric that simply checks that all relevant emails or messages have been found regardless of the input to the tool. Given a static search database, a similar tool correctness metric could be implemented for search where we check for a few key search results and maybe additionally check that the search query is relatively similar to the ground truth.

The evaluation of GPT4/GPT3.5 does not provide much value, except to show that the benchmark is challenging and that tool documentation is important. To improve this, you could consider evaluating with alternate prompts (e.g., ones that induce better/worse tool usage) in order to demonstrate that the proposed benchmark can effectively discern between tool-usage capabilities.

Thank you for your feedback, we plan on evaluating additional LLMs using ToolTalk in future revisions. We will also consider varying the prompts used as well.