DIVERSITY OF THOUGHT IMPROVES REASONING ABILITIES OF LARGE LANGUAGE MODELS

[part 1] We thank the reviewer for their comments. We now respond to any concerns the reviewer may have. Should the reviewer have any clarifications, we are happy to further engage. If the responses are satisfactory, we kindly request the reviewer to increase their score towards acceptance.

The novelty of the work is very limited. There exists work in which diverse prompts are used to generate diverse reasoning paths: This paper should be cited due to its similarity.

Response: We thank the reviewer for sharing this paper, but would like to strongly stress that the methodology of this paper is very different from that of ours. In the paper by Li et al., the reasoning problem’s prompt is formed by providing multiple, yet different examples depending on the test sample (refer to figure 10 in their paper), and using a step-wise verifier to correct errors made. This requires them to have a base of examples from which they can sample few-shot examples for each prompt, a critical limitation as noted in Appendix B.1 of their paper. We do not.

The “diversity” in their prompts are reasoning paths based on self-consistency for examples from the base of examples (i.e., a hold out set); these reasoning paths are not truly diverse as they all use the same approach (as highlighted in response to reviewer PAGk) and only serve as “more examples” for the model or at best as linguistic diversity triggers (but not reasoning diversity). In contrast, our work (i) involves truly diverse (refer to response to reviewer PAGk) and “related” approaches to solve the reasoning problem (refer to Figure 2 in our work), whilst (ii) limiting the number of examples in the prompt (we use 5, Li et al. need close to 20) and (iii) do not have a step-wise verifier (which further increases the cost).

It is not completely clear [how/why] DIV-SE works well on Blockworlds and AQUA-RAT, but less so on GSM8K and Commonsense QA.

Response: We would like to stress that the minor improvements are in scenarios where the models are already very performant in the tasks; regardless - the performance using our method is consistently high (with an average improvement of 8 percentage points) and often state-of-the-art (as in the Blocksworld case, with a 29.69 percentage point improvement). All this, with a fraction of the cost of contemporary approaches. We request the reviewer to look at the table below (in the summary link) to learn more about performance gains, and Figure 1 + 3 in the paper to learn more about the cost vs. performance trade-offs.

The reasons for little improvements for some models and datasets is because of task saturation (particularly in the case of GSM8K and GPT-4). This can be for a variety of reasons, such as these tasks not needing substantial reasoning capabilities as the models become bigger, or dataset contamination. However, for tasks such as Blocksworld, reasoning is more important and this is where our method shines compared to prior approaches.

Summary of the results can be found here:: https://ibb.co/68PF2Dr

IDIV is likely to have significant context length constraints.

Response: The prompts proposed in the work by Li et al. are significantly longer (with an average of 6050 tokens per prompt, based on their training data provided in https://github.com/microsoft/CodeT/tree/main/DIVERSE) compared to IDIV-SE (which has an average of 523 tokens per prompt) for the GSM8K task – almost a 12x gap. Additionally, the context length of models is increasing (as shown in most recent model releases), and we believe this should not be an issue. Additionally, we have taken measures to mitigate this potential issue. Specifically, we limit our prompts to a maximum of 3-shots, as opposed to (a) 5-7 shots typically used in standard few shot prompts (Wang et al., 2023), or (b) 20 shots as used by the work of Li et al. (https://arxiv.org/pdf/2206.02336.pdf). Please note the inference cost reported in Figure 1 and 3 accounts for both input and output token costs.

It’s not clear whether adding a persona increases the diversity of output. I suspect the persona does not add much to the method. This requires an ablation.

Response: As part of our preliminary analysis, we observed that the persona was very important in generating enhanced performance. The work of Salewski et al. that we cite highlights the importance of personas in in-context learning situations.

[part 1] We thank the reviewer for finding our work clever, novel, and emphasizing the importance of diversity in prompting. We now respond to any concerns the reviewer may have. Should the reviewer have any clarifications, we are happy to further engage. If the responses are satisfactory, we kindly request the reviewer to increase their score towards acceptance.

The method requires a held out set where different personas and approaches can be tested to pick useful ones. This makes the proposed DIV-SE method somewhat limited in practical applications.

Response: We agree with the observation made by the reviewer that in practice, our proposal requires a hold out set for evaluating the choice of approaches and personas. However, this set turns out to be very small, often with 100 samples (though we believe fewer, e.g., 50 might also suffice). Prompt engineering is typically a one-time effort, and the resulting prompt could potentially serve thousands or even millions of user queries

In addition, all prompt engineering practices in general (and not only our proposed approach) do require a hold out set, but the main issue with most practices in the status quo is that they require a significant amount of trial and error (https://docs.anthropic.com/claude/docs/optimizing-your-prompt).

Additionally, the approaches discovered by our DIVERSE-PROMPTING strategy are general and re-usable across reasoning problems of the same domain, adding to the practicality of our approach. Thus, the cost of evaluating the approaches on a hold out set is further amortised. As an example, for the GSM8K task, some of the LLM-generated approaches include: using visualisations, working backwards, using direct calculation, and method of elimination; these can be reused for other arithmetic reasoning problems.

Results seem mixed: Blocksworld clearly benefits from the diversity in prompting. It is clear on this benchmark that with IDIV-SE more can be achieved in less cost. But on other benchmarks this is not necessarily true e.g. GSM8K (Fig 3 & Fig 4).

Response: We would like to stress that the minor improvements are in scenarios where the models are already very performant in the tasks; regardless - the performance using our method is consistently high (with an average improvement of 8 percentage points) and often state-of-the-art (as in the Blocksworld case, with a 29.69 percentage point improvement). All this, with a fraction of the cost of contemporary approaches. We request the reviewer to look at the table (in the summary link below) to learn more about performance gains, and Figure 1 + 3 in the paper to learn more about the cost vs. performance trade-offs.

The reasons for little improvements for some models and datasets is because of task saturation (particularly in the case of GSM8K and GPT-4). This can be for a variety of reasons, such as these tasks not needing substantial reasoning capabilities as the models become bigger, or dataset contamination. However, for tasks such as Blocksworld, reasoning is more important and this is where our method shines compared to prior approaches.

Summary of the results can be found here: https://ibb.co/68PF2Dr

No qualitaitve/quantitative assessment of the synthetically generated demonstrations (using LLMs). I suspect some human annotation to improve the quality of these demonstrations can improve eventual model performance.

Response: In generating the demonstrations for a specific approach, we do not solely rely on the LLM to create the demonstrations from scratch. Instead, we utilize the same chain-of-thought demonstrations from Wei et al. (2022) as a base, and instruct GPT-4 to perform a “style transfer” of the solution using the specific approach.

To ensure the quality of these demonstrations, we conducted a manual evaluation as well. Given that the task primarily involves a style transfer of the demonstrations, we found no errors in the style transferred demonstrations i.e., correctness was preserved.

We also agree with the reviewer that more curation of the “augmented examples” can boost performance. However, we would like to point out that despite the lack of such curation, our approaches demonstrate SOTA performance on tasks that require complex reasoning (e.g., Blocksworld).

We thank the reviewer for their comments, and for finding our approach novel, and one that doesn’t require extensive prompt engineering while being applicable to many tasks. We now respond to any concerns the reviewer may have. Should the reviewer have any clarifications, we are happy to further engage. If the responses are satisfactory, we kindly request the reviewer to increase their score towards acceptance.

The methodology part of this paper is not well written. Figure 2 is readable, but Section 2 lacks a clear high-level structure. What are the goals for extracting approaches & personas, augmented demonstrations in the proposed method?

Response: We apologise for the lack of clarity in our presentation and will strive to improve this aspect, based on your suggestions and questions. What we wished to convey was the following structure:

1. Prompts in our proposal comprise of a combination of approaches (that provide technical insight) and personas (to bootstrap the model and align it)
2. We use the prompting template shown in Figure 6 to request the LLM to generate the names of various approaches. We then construct a word cloud from these approaches using a hold-out set and select the top five. For personas, we directly ask the model to provide a list of individuals suitable for task resolution.
3. To generate the few-shot examples, once the approaches are selected (based on frequency of occurrence), we first start with a chain-of-thought demonstration used in Wei et al. (2022), and query GPT-4 to style transfer this using the selected approach, to create an “augmented demonstration”. To ensure the quality of these demonstrations, we conduct a manual evaluation. Given that the task primarily involves a style transfer of the demonstrations, we found no errors. Thus, we have a prompt with (a) a persona, (b) a set of diverse approaches, and (c) corresponding augmented samples.

We agree with the observation made by the reviewer that in practice, our proposal requires a hold out set for evaluating the choice of approaches and personas. However, this set turns out to be very small, with 100 samples (though we believe 50 might often suffice). Additionally, most prompting strategies that are deployed require such a hold out set, and this is not a limitation to our work alone (https://docs.anthropic.com/claude/docs/optimizing-your-prompt).

We also thank the reviewer for other formatting changes suggested to improve the clarity of the paper; we will actively work on making these changes.

Merging multiple inference calls into one was first invented and discussed in Batch Prompting ([1] Cheng, et al. Batch Prompting: Efficient Inference with Large Language Model APIs. EMNLP 2023.). Please cite it and tone down your contribution on this technique.

Response: We thank the reviewer for the pointer to the paper, and will cite it. We would, however, like to highlight the difference between our approach and Batch Prompting. In our approach, we batch multiple “reasoning paths” within the same prompt. Batch Prompting does not (a) specifically address reasoning problems, and (b) primarily involves grouping the response to “multiple queries together” whereas our approach involves soliciting the response to “one query” using multiple reasoning paths within the prompt.

Section 2.2. “It independently formulates new approaches” -> Is it a hallucination or a feature? It looks like a hallucination to me. If this is important for achieving good performance, can you provide an ablation study based on whether to allow new approaches or not?

Response: Since there is no well-defined manner to restrict the model to only following the approaches we propose, we believe it would be hard to perform the ablation. However, from our evaluation of the approaches created (or hallucinated as suggested by the reviewer), the methodology proposed and calculations performed are often accurate. This begs the question as to understanding why these approaches must be restricted. Could the reviewer shed some additional clarity on their thoughts?