Supervised Knowledge Makes Large Language Models Better In-context Learners

We sincerely thank you for your insightful comments and valuable suggestions on our manuscript. Your expertise and detailed review have been instrumental in enhancing the quality and clarity of our work. For your convenience, we have marked all the related revisions in blue within the manuscript, ensuring easy identification of the changes made. We welcome any further inquiries or suggestions you might have, as your input is invaluable in refining our research.

Question 1:

The supervised model is trained on the in-domain corpus and then used to enhance the inference of LLMs. Have you tried fine-tuning the LLMs with the in-domain corpus? Although it has a much higher cost, I wonder whether there is a much larger performance improvement after SFT.

Answer 1:

We appreciate your suggestion regarding instruction-tuning the LLMs with the in-domain corpus. Existing literature indicates that while tuned LLMs demonstrate high performance on many reasoning datasets, they still face challenges with out-of-distribution tests. This insight, derived from recent studies such as Liu et al. (2023), informed our decision to focus on enhancing LLM inference with supervised models trained on in-domain data, a less explored approach with potential benefits in handling OOD scenarios.

[1] Liu, Hanmeng, et al. "Evaluating the logical reasoning ability of chatgpt and gpt-4." arXiv preprint arXiv:2304.03439 (2023).

Question 2:

I cannot capture the motivation of presenting Section 4.2. It seems that this section is not related to your core idea of the paper.

Answer 2:

Thank you for highlighting the need for clarity regarding Section 4.2. The purpose of this section is to investigate the limitations of in-context learning, as observed in Table 2, where LLMs underperform in this setting compared to our SuperContext approach with zero-shot demonstrations. We designed two interpretation tasks in Section 4.2 to delve into why in-context learning might be less effective, thereby providing a deeper understanding of the dynamics between LLMs and supervised learning models. This analysis is crucial for comprehending the full scope of our proposed methodology.

Question 3:

Apart from appending the model prediction to the prompt, you also include the model confidence score to the prompt. How much does the framework benefit from this design? If removing the model confidence score, how does the performance change?

Answer 3:

Your inquiry into the role of model confidence in our framework is useful. We have incorporated ablation studies in Table 2 to address this. The results show that including the confidence score of small models significantly boosts the performance of LLMs in various NLU tasks. For instance, the performance of ChatGPT improves from an average of 78.43 to 80.05 when confidence scores are included. Similarly, LLaMA2-7B's performance increases from 75.68 to 79.08, demonstrating the effectiveness of integrating model confidence in our approach.

Question 4:

How do you explain the extremely low scores in Table 3, i.e., 5.32, 6.08 and 3.72?

Answer 4:

The low scores in Table 3 primarily result from the models' challenges in generating correctly formatted responses and handling no-answer scenarios. We observed that the original Llama2-7B-chat model struggled with format adherence and hallucinations, especially in answering no-answer questions. This is reflected in the notably low score of 3.72. In other words, it cannot output I don't know when the question is unanswerable. However, when applying in-context learning with a mix of no-answer and has-answer instances, we noticed an improvement in handling no-answer questions, though this came at the cost of reduced accuracy in has-answer questions. These observations and their implications are further discussed in the revised manuscript following Table 3.

We sincerely thank you for your insightful comments and valuable suggestions on our manuscript. Your expertise and detailed review have been instrumental in enhancing the quality and clarity of our work. In response to your comments, we have carefully examined and addressed each issue you raised.

For your convenience, we have marked all the related revisions in blue within the manuscript, ensuring easy identification of the changes made. We welcome any further inquiries or suggestions you might have, as your input is invaluable in refining our research.

The most important concern about the reliability of the experimental results has been explained in Question 4.

Question 1:

The illustration of the proposed method in Figure 1 is not clear. It is unclear how the supervised knowledge participate in the whole procedure, while the capture states that it plays a core role.

Answer 1:

Thank you for your feedback. We acknowledge the need for clearer representation of supervised knowledge in our methodology. Accordingly, we have revised the caption of Figure 1 to better illustrate how supervised knowledge plays a pivotal role in our approach, clarifying its integration and significance in the overall process.

Question 2:

Equation (1) is misleading. On one hand, the left term contains "{x_i,y_i}{i!=j}" as part of the condition for LLM inference, while "i!=j" is not a complete expression for the range of "i". On the other hand, the right term contains "{x_i,y_i}{i \in S_j \subset [1,...,N]", i.e., the few-shot examples, as part of the condition, which may conflict with the statement in context: "where i \in [1,...,N]". I can only assume the author intended to mean "{x_i,y_i}_{i!=j, i \in [1,...,N]}" in the left term instead of the right term. However, the following context called this condition "the concatenation of the task description", which leads to a puzzle again.

Answer 2:

We appreciate your insightful observations regarding Equation 1. You are correct in identifying a discrepancy. The intended meaning is that the term {x_i,y_i}_{i!=j, i \in [1,...,N]} applies to the left side of the equation. We have now rectified this in the manuscript to accurately reflect the experimental setup and avoid any confusion. The revised text should now provide a clear and accurate depiction of our method.

Question 3:

a. "Specifically, our receipt r_i": receipt from what? in the form of what? b. "learning from which ... and which ... is important": who learn what? how does it learn? c. "as typical data points are OOD for the model": for the discriminative model or for the LLM?

Answer 3:

Thank you for highlighting these ambiguities. We have made the following revisions for clarity:

Question A: The term 'receipt r_i' refers to the output and confidence score generated by Supervised Language Models (SLMs), in form of the prompt. Question B: We have elaborated on how Large Language Models (LLMs) are trained to learn from in-context examples, providing detailed descriptions of the prompt design and learning process. Question C: The term 'Out-Of-Distribution (OOD)' applies to both discriminative models and LLMs in our context. The training set and in-context examples are both drawn from the in-domain dataset, while the training set is used to fine-tune the SLM and in-context examples are used as the prompt, which we now explain more comprehensively in Section 2.2.

Question 4:

Table 2 has two lines for the same baseline "ELECTRA-large" with different scores and no explanation for such difference. This raises serious questions about the reliability of the experimental results.

Answer 4:

We understand the importance of clarity in our experimental results. The discrepancy in Table 2 for ELECTRA-large arises due to differing test data sizes for ChatGPT and LLaMA2, as elaborated in Section 3.1. We have included a note in Table 2 explaining this variance: for LLaMA2-7B-chat, we used 37,438 instances, whereas, for ChatGPT, the number was 43,728. This clarification should address concerns regarding the reliability of our findings.

Thank you sincerely for your insightful comments and valuable suggestions on our manuscript. We have diligently reviewed and addressed each point you raised. To facilitate your review, we have highlighted all revisions in blue in the manuscript.

Question 1:

It seems that the approach in this paper is quite similar to what Hugging Face's GPT does, with the only difference being that it uses its own trained API instead of Hugging Face's SOTA small model API.

Answer 1:

We appreciate your observation. Our work, indeed, shares a conceptual similarity with Hugging Face's GPT in leveraging Language Model (LM) architectures. However, the key distinction lies in our approach's application and analysis under out-of-distribution (OOD) conditions, a less explored area in the existing literature. Our method uniquely combines Supervised Language Models (SLMs) and Large Language Models (LLMs) to assess and correct potential inaccuracies in SLMs' outputs. We've designed specialized tasks for interpretation and calibration laws to delve into this interaction, emphasizing the novelty of our approach. These distinctions and our experimental framework are now more clearly articulated in the revised sections of our manuscript, as shown in Related Work.

Question 2:

The paper lacks significant novelty, essentially enhancing the output of small models with larger models. However, it extensively validates the effectiveness of this approach.

Answer 2:

Thank you for your feedback. While recognizing HuggingGPT's contributions, our study pivots on a critical gap: the systematic evaluation of Large Language Models (LLMs) in Out-Of-Distribution (OOD) scenarios. In addition, a significant novelty of our work is investigating whether Supervised Language Models (SLMs) can mitigate LLMs' hallucination tendencies, an aspect scarcely addressed in current research. This inquiry not only extends the current understanding of model interactions but also proposes a novel perspective in improving LLM reliability.

Question 3:

The paper intentionally incorporates confidence when using the output of small models, but it lacks a detailed ablation study on the role of confidence. I am particularly interested in understanding the significance of confidence in this context.

Answer 3:

We're grateful for your interest in the role of model confidence. To address your query, we've included Table 2 in our revised manuscript, presenting a detailed ablation study. The data demonstrate that incorporating the confidence metric from small models significantly enhances LLM performance across various Natural Language Understanding (NLU) tasks. For instance, the average performance of ChatGPT improved from 78.43 to 80.05, and for LLaMA2-7B, it rose from 75.68 to 79.08.

These findings underscore the importance of confidence as a valuable component in our approach, offering a more nuanced understanding of model interactions.