Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing

We are grateful for the reviewer's detailed observations, which have prompted us to provide clearer explanations. Here are our responses to the concerns raised:

Regarding Loss Functions for Each Module: Not enough details are provided about loss functions of each module.

Response: We apologize for any confusion caused by our initial explanation. The update process for each module in our system is primarily driven by prompting a large language model (LLM), such as GPT-4. The specific prompts used for the different LLM roles within the Evoke framework are detailed in Appendix B. Additionally, the exact implementation process for these updates is clearly outlined in Algorithm 1 of our paper.

Explanation and Reproducibility of Modules: Not enough explanation of the modules; making it difficult to reproduce for other researchers.

Response: We understand the importance of reproducibility in research. To this end, we have provided a comprehensive description in Algorithm 1. Moreover, the prompts used for each module's role are included in Appendix B. We are committed to enhancing the reproducibility of our work and will make the code and data publicly available alongside the publication of the camera-ready version of our paper.

Loss Functions for Each Module: What are the loss functions for each module?

Response: As previously mentioned, our method, Evoke, does not incorporate traditional explicit loss functions due to its reliance on prompting an LLM. However, it's noteworthy that recent studies have begun to explore the connection between in-context learning in LLMs and traditional gradient learning methods [1]. Additionally, there's emerging research that explores understanding chain-of-thought prompting from a gradient perspective [2]. In Evoke, the instruction formulation for LLM-Author could be interpreted as a synergistic integration of in-context learning and chain-of-thought prompting. This suggests the possibility of an implicit, or 'proxy', loss function guiding the update process, aligning with these recent insights into LLM behaviors.

Targets for the LLM-Selector Module: What are the targets for example for first difficulty ranking module?

Response: The primary objective of the LLM-Selector module is to assess the difficulty level of a given task, considering the instruction, input, and correct answer. This assessment is then used to identify and select the most challenging ('Hard') examples. We focus on hard examples because LLMs, due to their extensive training on diverse datasets, are typically proficient in solving easier tasks. Incorporating easy examples into our prompt editing process would not significantly enhance its learning process. This selection strategy is thoroughly analyzed in the Experiment section of our paper, where we compare the outcomes of selecting 'Hard,' 'Easy,' and 'Random' examples. We welcome any further inquiries on this aspect of our research.

References:

[1] Dai, Damai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. "Why can gpt learn in-context? language models secretly perform gradient descent as meta optimizers." arXiv preprint arXiv:2212.10559 (2022).

[2] Wu, Skyler, Eric Meng Shen, Charumathi Badrinath, Jiaqi Ma, and Himabindu Lakkaraju. "Analyzing chain-of-thought prompting in Large language models via gradient-based feature Attributions." arXiv preprint arXiv:2307.13339 (2023).

We are grateful for your insightful questions, which highlight important aspects of our study. Let us provide further clarification:

Regarding the Use of a Reviewer for Scoring: The choice of using a reviewer to estimate scores instead of directly relying on end-task scores is an interesting one? Is it for computational reasons/costs to query GPT4? An elaboration would help (I may have misunderstood this as aspect). I had to re-read the sections describing the author, reviewer, selector multiple times along with the algo block to be sure of whats going on. This section could benefit from some more details in an image and/or writing.

Response: We appreciate your observation about our use of the LLM-Reviewer. Our decision was driven by both efficiency and effectiveness:

• Correlation with End-Task Accuracy:

One significant challenge in evaluating auto-prompt tasks, such as those in Instruction Induction, Big Bench, and the OpenLLM Leaderboard, lies in the limited size of the datasets. Typically, these datasets contain fewer than 1,000 entries, a stark contrast to the tens or hundreds of thousands of data points commonly used in traditional machine learning. This small sample size inevitably leads to high variance and instability in performance metrics when using a standard train-test split. As a result, the derived accuracy from such a small dataset may not reliably reflect the true task performance.

In contrast, LLMs like the one we employ in LLM-Reviewer are trained on vast and diverse data sources, encompassing a wide range of tasks. This extensive training equips LLMs with a robust understanding and capability to assess task performance more reliably. Our approach leverages this strength of LLMs. As shown in Figure 9, our analysis reveals a significant positive correlation between the evaluations made by LLM-Reviewer and the actual end-task accuracy. This correlation not only validates our methodology but also underscores the reliability of LLM-Reviewer as a proxy for task performance assessment.

Moreover, as evidenced in Table 1 below, the LLM-Reviewer outperforms the Author-Only approach in accuracy and stability. This further emphasizes the advantage of integrating LLM-Reviewer in the evaluation process, particularly when dealing with tasks characterized by limited data availability. By utilizing LLM-Reviewer, we are able to mitigate the challenges posed by small datasets and provide a more accurate and stable measure of task performance.

• Computational Efficiency: Opting for the LLM-Reviewer's score rather than direct task accuracy for prompt selection notably enhances computational efficiency. This method reduces the frequency of querying the inference LLM (e.g., GPT-4), thus saving computational resources.

In the camera-ready version of the paper, we will use the additional one page to expand on these motivations and provide additional details for clearer understanding. We also plan to include a comparative analysis showing the performance enhancement when incorporating the reviewer, as outlined in the table below:

Table 1: Test Accuracy (%) Change on Author-Reviewer vs. Author-only

We are deeply appreciative of the reviewer's insightful comments and would like to clarify some aspects of our work in light of the observations made:

On Similarities with DeepMind's Work: The Evoke method shares significant similarities with "Large Language Models as Optimizers" from DeepMind. While there are variations in the detailed processes, the core contributions and methodologies closely resemble each other. Since the DeepMind paper was released before the ICLR deadline [Sep 7], it is imperative to include a discussion of this relationship in the paper.

Response: We thank the reviewer for highlighting the need to differentiate our Evoke method from [1]. While we are working on similar problems, significant distinctions exist:

• Methodological Distinction: In our approach, the Large Language Model (LLM) assumes a novel and dual functionality, acting both as an author and a reviewer. This is a key differentiator from [1], where the LLM's role is confined to that of an author. The implications of this dual role are profound and multifaceted.

  As an author, the LLM generates content, similar to [1]. However, as a reviewer, it also critically evaluates and refines the generated content. This reviewer aspect adds an additional layer of quality control and perspective, enabling a more nuanced and balanced output.

  To further understand this distinction, it's instructive to examine the meta prompt used in our method. Our meta prompt for LLM-Author guides the LLM in content creation and the meta prompt for LLM-Reviewer frames its approach to reviewing and refining that content. This dual engagement in the creative and evaluative processes is fundamental to our approach and distinguishes it significantly from the model used in [1], leading to potentially more robust and well-rounded results.

• Efficiency Focus: Evoke emphasizes efficiency, optimizing prompts in no more than five steps. This is a stark contrast to the hundreds of steps in [1].

• Generated Instruction: The comparative analysis between the Evoke system and the method described in [1] reveals a strategic difference in instruction generation, tailored to the complexity and familiarity of the tasks. For instance, in tasks like logical fallacy detection, where terms such as 'formal' and 'informal logical fallacies' might be unfamiliar to a general audience, Evoke prompt incorporates detailed definitions that can ensure clarity and comprehension. This is in stark contrast to the more generic approach observed in [1], where generated prompts resemble a task description. Conversely, for more universally understood tasks like movie recommendations, Evoke smartly refrains from over-elaborating, recognizing the common knowledge base of its users. This adaptive strategy in instruction crafting contributes to better task performance.

Regarding the timing of the referenced paper [1], we acknowledge its contemporaneous nature, as per the reviewer guideline: "We consider papers contemporaneous if they are published (available in online proceedings) within the last four months". And the suggested paper [1] was published less than one month from ICLR deadline. However, we will ensure to discuss its relationship and the nuances differentiating our work in the revised manuscript.

Improving Evoke's Methodological Clarity: Some of the detailed tables and figures could be relocated to the appendix to allow ample space for a more comprehensive description of the experimental settings and the Evoke method itself. A more detailed explanation of the Evoke process would enhance the paper's clarity.

Response: We agree with the suggestion to relocate some detailed tables and figures to the appendix. This will create space for a more thorough description of the experimental settings and the Evoke method. In the camera-ready version, we will include additional explanatory text to distinguish our main algorithm from [1], ensuring greater clarity for readers.

Rationale Behind Dataset Selection and LLM Usage: The rationale behind selecting the Instruction Induction and Big Bench Instruction Induction datasets is not clearly articulated. Some of these tasks may be practical, while others could be considered more akin to toy tasks. Additionally, the use of the LLM as merely GPT-4, a black-box large language model, raises the question of whether the Evoke method is equally effective with other large language models, such as Llama 2 or ChatGPT.

Response:

• Dataset Choice: The selection of the Instruction Induction and Big Bench Instruction Induction datasets was strategic, aligning with current practices in auto-prompt work [1,2,3]. These datasets offer a broad spectrum of task difficulties, reflecting real-world LLM applications. We chose them for their relevance and to facilitate comparisons with existing and future studies in auto-prompting.
• LLM Choice: We selected GPT-4 for its state-of-the-art capabilities, as noted in [4]. However, it can also be extended to smaller scale, open source LLMs. To demonstrate this, we conducted additional experiments with Mistral-7B and Llama2-70B for inference, showcasing significant improvements across various tasks. The detailed results are presented in Table 1 of our response.

Table 1: Test Accuracy (%) Comparison on APE and Evoke for Mistral-7B

Due to limited time and computational resources, we have not finished all experiments on Llama2-70B, we'd like to firstly present available result below.

Table 2: Test Accuracy Comparison (%) on APE and Evoke for Llama2-70B

On Comparative Fairness with Baseline Methods: Comparing Evoke to other baseline methods may be perceived as somewhat unfair, given that Evoke leverages supervised information from the training dataset. This distinction should be acknowledged when discussing the comparative results.

Response: We appreciate the concern regarding the fairness of comparisons. It's important to note that other baseline methods also leverage supervised information from training datasets. For human prompt engineers, viewing sample data is necessary to understand the given task, and during prompt iterations, failed cases of a certain prompt are also taken into account. For APE, it uses instruction induction to initialize the prompt, along with sample data. All approaches assume the availability of a given dataset with input-output pairs; where Evoke differs is in its more effective usage of this set.

Clarifying Evoke's Distinction from DeepMind's Method: A key concern revolves around the distinctions between Evoke and "Large Language Models as Optimizers" by DeepMind. Clarifying these differences is pivotal for a comprehensive understanding of both approaches.

Response: As previously mentioned, our method's efficiency and dual role of the LLM in Evoke are key differentiators from [1]. These aspects are crucial for understanding the unique contributions of our work.

We sincerely thank the reviewer for recognizing the novelty and effectiveness of our method. We appreciate the opportunity to clarify and expand upon the questions raised:

Concerning Iteration Dependence: The efficacy of the approach may heavily depend on the number of iterations and the quality of feedback loops, which could vary significantly based on the complexity of the task and the initial prompt quality.

Response: We appreciate your concern about the iteration-dependent nature of our approach. To clarify, we have standardized the number of iterations (T) to 5 across all tasks in our experiments. We have initially tried T up to 10, and setting T=5 was informed by preliminary experiments showing converging improvements in task accuracy at this iteration count, irrespective of task complexity. Furthermore, we experimented with varying initial prompts, including task descriptions and empty strings, to assess the sensitivity of our Evoke algorithm to initial prompt quality. Our findings, detailed in Table 1 of our response, demonstrate that Evoke's performance is robust to variations in initial prompt quality, consistently outperforming baseline methods.

On Complexity and Overhead: The Evoke model might introduce additional complexity and computational overhead since we need to do the iterative selection process T times.

Response: We acknowledge the reviewer's point about potential complexity and computational overhead. To contextualize this, we compared Evoke's process with the conventional human prompt engineering workflow, which typically involves several iterative steps of testing, analyzing, and refining prompts. Interestingly, Evoke operates more efficiently in terms of computation time while achieving comparable or superior accuracy. This efficiency gain is particularly notable given that Evoke's iterative process, even set at its most computationally intensive mode, matches the computational demands of a single iteration in the human-driven process. Furthermore, by limiting T to 5 in our experiments, we achieved a balanced trade-off between performance optimization and computational resource usage.

Additionally, in all experiments, we set T to 5, resulting a strong optimized prompt.

Regarding Evaluation Phase and Prompt Concatenation: Are all the prompts generated by Evoke concatenated together during the evaluation phase?

Response:

• During Training (Prompt Editing) Evaluation Phase: In the training phase of our method, each prompt generated by Evoke undergoes an independent evaluation process. We have adopted this strategy deliberately to mitigate any potential interactions or biases that could arise from prompt concatenation. This independent assessment of each prompt allows us to accurately gauge the effectiveness of every individual prompt generated, ensuring that our method's learning process is informed by clear, unbiased feedback.

• During Inference Evaluation Phase: For the evaluation phase during inference, our approach is different. Here, we utilize only the final optimized prompt generated by Evoke for calculating accuracy on the test set. This decision is based on our objective to evaluate the practical applicability of the final prompt in real-world scenarios. By focusing on the final prompt, we are able to assess the ultimate effectiveness of the Evoke algorithm in producing a single, highly effective prompt for practical use.

Impact of Initial Prompt Quality: How do the quality of initial prompts affect the effectiveness of the Evoke method throughout iterations?

Response: The influence of initial prompt quality on the efficacy of Evoke was an important aspect of our investigation. To thoroughly examine this, we conducted additional experiments using two alternative initial prompts: a task description and an empty string. As illustrated in Table 1, Evoke's performance showed relative insensitivity to the quality of the initial prompt. These results, further underscore the robustness of our approach.

Table 1: Change in Task Accuracy Using Different Initial Prompts

We hope these responses address the reviewer's concerns adequately and demonstrate the robustness and efficiency of our Evoke method. We remain open to further discussions and insights that can refine our approach.