Are Human-generated Demonstrations Necessary for In-context Learning?

We greatly appreciate the constructive feedback and insightful suggestions provided by the reviewer. Our responses to the raised concerns are as follows:

Q1: The statistically consistent conclusion on the correctness of demonstrations

A1: Sorry for the confusion. We acknowledge the reviewer's concern about the correctness of demonstrations. However, previous study shows that in-context learning does not rely on the input-label mapping in the demonstrations to perform the task (Min, Sewon, et al.). The correctness of individual demonstrations is not the sole determinant of the overall quality, especially in the context of language models' ability to generalize from these inputs.

Thus, regarding the phenomenon that correct answer rate increases even though the generated demonstration is incorrect, a reasonable explanation is that strict correctness may not be the primary influencing factor in ICL's performance. Instead, language models might utilize other cues and patterns within demonstrations to arrive at correct conclusions.

Thus, the correctness of SEC's demonstration isn't a key factor of SEC's success.

Moreover, while our study involved 20 correct and 20 incorrect predictions, each case incorporated 5 demonstrations, resulting in an effective analysis of 200 demonstrations. This sample size, we believe, offers a reasonable overview of potential error types within demonstrations. Nevertheless, we recognize the importance of expanding this analysis and plan to include a larger dataset in future work to enhance the robustness of our conclusions.

Q2: Generality of SEC using multiple language models

A2: We concur with the reviewer on the importance of investigating SEC's applicability to multiple models. To this end, we intend to extend our experiments to include both open-source models and the newer GPT-4 architecture.

[1] Min, Sewon, et al. "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?." Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.

The author's reply and additional experiments seem reasonable to me. Therefore, I raise my score.

We thank the reviewer for their insightful comments and constructive feedback. We address each concern below:

Q1: SEC’s performance on less capable language models.



A1: We appreciate your concern about SEC’s applicability to smaller, less capable models. As reported in Section 4, SEC indeed shows a performance drop when applied to smaller models, which can struggle with instruction following and generating high-quality few-shot examples. This is a critical aspect of our ongoing research, and we plan to extend our evaluations to a wider range of models, including open-source ones. These additional experiments will provide deeper insights into SEC's scalability and effectiveness across different model sizes.

However, it is important to highlight the growing trend towards larger and more capable language models in the field. As language models continue to scale up in size and sophistication, we anticipate that the importance and relevance of methods like SEC will increase correspondingly. The ability of SEC to autonomously generate demonstrations is likely to become more valuable as models become more adept at complex reasoning and generation tasks. Therefore, while we acknowledge the current limitations with smaller models, we believe that the scaling trend in language models will make SEC an increasingly crucial method in the future.

Q2: Comparisons with "Automatic chain of thought prompting in large language models (Zhang et al., 2022)"

A2: We recognize the importance of contextualizing SEC within the landscape of related work, such as Auto-CoT by Zhang et al. Our initial decision to exclude Auto-CoT from our baseline comparisons stemmed from its intensive querying and clustering requirements to generate demonstrations. This process can be particularly challenging, and even infeasible, for users dealing with single-question scenarios, which are common in practical applications.



However, we appreciate the reviewer's suggestion to include a comparison with Auto-CoT. In our revised manuscript, we plan to incorporate a direct comparison between SEC and Auto-CoT on some classical benchmarks.

Q3: Combining ICL and SEC

A3: The suggestion to combine ICL with human-crafted demonstrations and SEC’s model-generated questions is indeed intriguing. While we have not explored this combination, we recognize its potential for synergistically enhancing performance. We will initiate experiments combining these methods in the coming days.

However, it's important to note that the primary contribution of this paper remains the introduction of SEC as a strong, novel and flexible zero-shot prompting method.

Q4: Presentation and Figures



A4: We thank the reviewer for pointing out the typographical errors in Figure 1 and the readability issue in Table 2. We will correct these in our revised version to enhance clarity and presentation quality.

Once again, we thank you for your valuable feedback and look forward to further improving our work.

We have carried out the experiments on Auto-CoT. CoT-SEC's performance is comparable to Auto-CoT, even without access to the full test dataset and additional clustering.

The results in the table will later be added into the revised version of our paper.

We are thankful for the valuable insights and constructive feedback provided by the reviewer. We address each concern below:

Sorry for the confusion. First we want to clarify that SEC isn’t a variant of in-context learning. Inherently, SEC is a zero-shot method which only relies on test input.

Q1: Comparison with Automated Demonstration Selection Approaches

A1: We apologize for any confusion. We acknowledge the reviewer's point on comparing SEC with automated demonstration selection and curation approaches. However, it's crucial to emphasize that SEC inherently functions as a zero-shot method, relying solely on test input, which sets it apart from ICL. Comparing SEC directly with automated demonstration selection approaches may not be entirely fair, as the latter often requires a substantial pool of training examples for selection. Our intention in comparing SEC with ICL was primarily illustrative, highlighting SEC's superiority over the zero-shot baseline.

SEC substantially outperforms zero-shot baseline and reaches comparable performance with ICL methods which proof it’s efficacy. To provide a more comprehensive comparison, we are currently adding experiments with Zero-shot CoT. Preliminary results indicate that on benchmarks like GSM8k and ARC, Zero-shot CoT's performance (73.39 on GSM8k and 84.04 on ARC) is outperformed by CoT-SEC (77.0 on GSM8k and 86.9 on ARC).

For the five paper you kindly provided, (a) focuses on exemplar selection from the view of explanations, (b) focuses on active example selection, (c) focuses on applying ICL to offensive content paraphrasing, (d) focuses on CEIL, an ICL example selection method, and (e) focuses addressing the order sensitivity in ICL. These approaches, indeed, improve upon vanilla ICL using training datasets. However, our approach with SEC offers a novel perspective by achieving comparable ICL performance without relying on training data. This difference in approach underscores SEC's flexibility and ease of use, presenting an alternative direction in the realm of language model prompting.

Q2: Bias of SEC

A2: We recognize the importance of addressing potential biases in LLM-generated demonstrations. While SEC demonstrates comparable performance to ICL in common benchmarks, we understand the need for further exploration, especially in tasks rare in training data. We plan to add such tasks in our revised version.

Additionally, it's crucial to note that human-crafted prompts are not immune to biases. As highlighted by Ma et al. (2023), human biases can inadvertently influence the crafting of prompts. Hence, both human-crafted and LLM-generated content require careful consideration regarding bias.

Q3: Applicability to Open-Source Models

Our initial study focused on closed-source GPT-based models due to their prevalent use and performance. However, we agree with the reviewer that investigating SEC's applicability to open-source models is essential. We plan to conduct additional experiments with open-source models to validate SEC's effectiveness across a wider range of language model architectures.

Q4: Quality of Demonstrations and Performance with Increased Demonstrations

A4: Sorry for the confusion. We acknowledge that SEC's performance may not always surpass that of human-crafted prompts. The example in Figure 20 of our paper illustrates this point.

Regarding the observed decrease in performance with an increased number of demonstrations, we hypothesize the existence of an optimal number of demonstrations for SEC, akin to what Reynolds and McDonell (2021) observed for human-crafted prompts. The tailored nature of SEC's demonstrations might necessitate a smaller number of examples compared to human-crafted ones for optimal performance. Therefore, the decline in performance after a certain number of shots should not be misconstrued as an indication of poor demonstration quality.

Also, we only observe this decrease in performance with an increased number of demonstrations on HumanEval dataset while on other dataset (e.g. GSM8k), there doesn't exist such a decrease.

Besides, we do include a qualitative analysis on the correctness demonstrations in Section 4 and Appendix B of our paper.

[1] Ma, Huan, et al. "Fairness-guided Few-shot Prompting for Large Language Models." arXiv preprint arXiv:2303.13217 (2023).

[2] Reynolds, Laria, and Kyle McDonell. "Prompt programming for large language models: Beyond the few-shot paradigm." Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021.

We would like to thank the reviewer for the encouraging comments and address the concerns below.

Q1: On the Significance of SEC as Another Form of CoT

A1: Your observation that SEC may seem akin to an extension of the chain-of-thought (CoT) prompting is insightful. However, SEC distinguishes itself by enabling the model to perform analogical reasoning, which is a leap beyond the capabilities of traditional CoT methods. SEC is fundamentally a zero-shot prompting pipeline. To provide a more comprehensive comparison, we are currently adding experiments with Zero-shot CoT. Preliminary results indicate that on benchmarks like GSM8k and ARC, Zero-shot CoT's performance (73.39 on GSM8k and 84.04 on ARC) is outperformed by CoT-SEC (77.0 on GSM8k and 86.9 on ARC).

These findings reinforce our belief that SEC's performance cannot be easily replicated by existing CoT methods in a Zero-shot scenario. SEC's novelty, therefore, lies in its unique ability to autonomously generate meaningful demonstrations, which sets it apart from existing CoT approaches.

Q2: The Limitations of SEC

A2: We apologize for any confusion regarding the limitations of SEC. We recognize the potential for performance degradation with incorrect self-generated demonstrations. Our analysis indicates that large language models, particularly those with capacities at or above GPT-3.5, tend to be robust against minor errors in self-generated prompts for common tasks, maintaining performance comparable to that with human-crafted prompts.

Our experiments across various reasoning tasks show that SEC generally achieves promising results. However, we recognize that in rare or abstract reasoning tasks, which have limited representation in training data, the model might encounter challenges. We plan to conduct further experiments in these domains to comprehensively understand SEC's limitations. Notably, in the domain of MATH, where LLMs typically struggle, CoT-SEC has outperformed CoT-ICL, reaffirming our observation about the robustness of larger models against minor prompt errors.

In Section 4, we have evaluated SEC’s performance on smaller models and observed that its effectiveness diminishes with smaller model capacities. We are committed to extending our research with additional experiments using open-source models to further explore SEC’s applicability and limitations across different model scales.

There are some updates to the response.

A1: We have added zero-shot CoT as a part of our baseline methods to all our main experiments on GPT3.5, GPT4 and Llama2 34B. SEC consistently outperforms zero-shot CoT.

The distinctiveness of SEC lies in its ability to autonomously generate meaningful demonstrations, differentiating it significantly from traditional CoT methodologies. 



A2: On the experiments of 200 3-digit base-5 addition problems, we do observe slight performance degradation of SEC compared to ICL. Nevertheless, SEC still outperforms it’s Zero-shot baselines in corresponding scenarios (Vanilla SEC to Zero-shot and CoT-SEC to Zero-shot CoT).

Contrary to what might be assumed, the incorrect demonstrations are not the primary cause of this performance drop, as supported by the research of (Min, Sewon, et al.). We hypothesize that the performance degradation may stem from the model's unfamiliarity with the task, leading it to generate question-answer pairs that deviate from the expected distribution. For instance, the model might produce answers containing digits like 6, 7, 8, 9, which are outside the scope of base-5 calculations.

References

[1] Min, Sewon, et al. "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?." Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.