In-Context Learning at Representation Level via Unlabeled Texts

Thank you for your response.

Comment 6: Could you share the prompts or give some analysis about the effect of the prompt use to get the presentation of the in-context examples?

Thanks for your careful reading of these details. The prompts used in the experiments are presented in the below Table. We will modify the paper to include the prompts.

Comment 1: The paper requires some additional comparation to some in-context vector methods like [1][2], which also create a hidden state offset by in-context example.

Thanks for your kind reminder. We have already considered the papers you mentioned when developing our paper. The reason for not comparing our method with those papers is that our setting differs from the papers you referenced. We focus on utilizing unlabeled texts for in-context learning. In real-world situations, gold standard demonstrations can be difficult to come by. However, The methods mentioned in the papers you cited rely on gold labels from training data. Therefore, this difference makes a direct comparison less meaningful in our context.

Comment 2: Required some additional ablation study to prove why use the unlabeled texts from test set rather than labeled. For example, if you use labeled data in your framework in Figure 2, how will the performance be?

Thank you for raising an interesting question. The motivation of using unlabeled texts stems from two factors. First, obtaining gold standard demonstrations in real-world scenarios can be challenging. Therefore, there is a need to develop a method for utilizing unlabeled texts for in-context learning. Second, as shown in Table 2 on page 3, we observe an increase in ICL performance when labels are removed. Taking into account these two factors, we have decided to incorporate unlabeled texts for in-context learning. However, our method is dedicated to unlabeled texts instead of input-label pairs. We appreciate your valuable feedback of using labeled data and will investigate the potential for input-label pairs in our method in the future.

Comment 3: The improvement from increasing numbers of the retrieved hidden states is limited.

Thank you for careful reading of these details. The limited improvement from increasing numbers of the retrieved hidden states is more likely to be caused by the composition of the test dataset. Our method depends on the similarity between the retrieved texts with the test input to reconstruct the input test vector. However, when constructing the test dataset, we typically want the test dataset to be diverse. Therefore, it is hard to find the relevant examples to the test input. The performance can not be improved much by increasing numbers of the retrieved hidden states. We appreciate your valuable feedback and will construct more diverse datasets to better utilizing larger number of the retrieved hidden states.

Comment 4: The link of the codes is invalid.

Thank you for your interest in our work. We are willing to make our code open-source, but the rules of OpenReview prohibit us from including any links that reveal personal identity in papers under review. Instead, links should contain the word "anonymous" to signify that we will release our code. The complete implementation of our paper will be released in the future.

Comment 5: Besides MRPC, the difference between labeled and unlabeled data for ICL is limited. Is there any more dataset can be used for prove the conclusion about un-labeled ICL in general-domain?

Thank you for carefully reviewing our experiments. There are two experiments supporting the claim that labels may not be necessary for in-context learning in the general domain. In addition to the traditional ICL experiments with and without labels, the comparison between our method without labels and traditional ICL also confirms this claim. As shown in Table 6 on page 9, the largest improvement of our method without labels compared to ICL with labels is observed in the general domain. The large improvement in general domain indicates labels may not be necessary for ICL in the general domain. Combining these two experiments, the claim is reasonable.

Comment 5: Some discrepancies in ICL performance of baselines are observed between this paper and the literature.

Thanks for your concern about the experimental results. The ICL performance in our paper is produced by the framework proposed in OpenICL: An Open-Source Framework for In-context Learning presented at ACL 2023. This framework provides standard implementation of ICL and has been widely adopted in other ICL papers [1][2][3], demonstrating its reliability. Also, the framework is widely recognized in the ICL community (542 stars on GitHub). Therefore, the results of our paper are reliable.

Regarding the differences with the referenced paper, we carefully check the implementation of the referenced paper. The following reasons may account for the performance difference.

1. In-context learning is quite unstable to the prompts. Our prompts can be quite different with those in that paper. For instance, for rte dataset, they used "{premise}\nquestion: {hypothesis}. true or false?\n" while we used "The two sentences (have | do not have) the relationship that if one is true, then the other is true.\nsentence1: {sentence1}\tsentence2: {sentence2}". Different prompts can result in quite different results.
2. They loaded the model with half precision to save the memory, while we used full precision.
3. They incorrectly implemented the padding_side in the tokenizer. When calculating the perplexity loss for the sequence, the padding_side should be right, not left. With padding_side set to left, language models are conditioned on the <pad> tokens when processing the entire sequence, while <pad> tokens are not seen by LLMs during pretraining. The padding_side should be left when language models are required to generate new tokens.

To summarize, the above three factors can greatly impact the final performance. Therefore, the performance discrepancies should be reasonable.

[1] ACL 2024 | Forward-Backward Reasoning in Large Language Models for Mathematical Verification by Weisen Jiang, Han Shi, Longhui Yu, Zhengying Liu, Yu Zhang, Zhenguo Li, James Kwok

[2] NAACL 2024 | Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis by Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li.

[3] ICLR 2024 | OS-Copilot: Towards Generalist Computer Agents with Self-Improvement by Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, Lingpeng Kong

Q1: Why does the proposed method achieve the same accuracy across four different LLMs on MRPC and CoLA datasets?

The MRPC and CoLA datasets are both from the general domain. It is probable that both datasets are pretrained by four models. Therefore, the pretrained knowledge of four models on the MRPC and CoLA datasets is likely to be similar.

Q2: Why there is a decrease in performance when upgrading from Llama2-7B to Llama2-13B on SST-2 and Phrase datasets using the proposed method?

The decrease in performance is due to the weaker abilities of Llama2-13B compared to Llama2-7B for the performance decrease is also observed in zero-shot when upgrading to Llama2-13B.

Comment 1: The first concern is the absence of empirical evidence to support the theoretical claims about the weak semantic relevance's impact on ICL performance.

The empirical evidence to support the claims about the weak semantic relevance's impact on ICL performance can be found in Table 5 on page 8. The traditional ICL takes in the concatenation of multiple demonstrations which are not necessarily relevant. We compare traditional ICL to our method that processes demonstration inputs and test input independently. As shown in Table 5, the traditional concatenation performance for GPT-Neo-2.7B and Llama2-7B is at least $7.94$ and $10.57$ worse than representing the inputs independently. The notable performance drop for two models indicates the negative impact of weak semantic relevance caused by the traditional concatenation, aligning with the theoretical claims about the weak semantic relevance's impact on ICL performance.

Comment 2: The proposed ICL paradigm simplifies the interaction between examples and the test input into a single step, potentially losing vital information that could be obtained in multi-layer interactions of conventional ICL.

Your concern about our method potentially losing vital information is reasonable. We took this into consideration when developing our method. The most common practice is to use the performance metric for assessing whether a method effectively retains vital information or not. The results in Table 6 demonstrate that our method outperforms traditional in-context learning for GPT-Neo-2.7B, Mistral-7B, Llama2-7B, and Llama2-13B in three well-known settings. Despite simplifying the interaction process between examples and test input into a single step, our method still brings significant improvements. This suggests that our method may uncover crucial insights that traditional ICL methods may miss. This finding underscores the importance of further exploration within the ICL community. We appreciate your valuable feedback and will investigate the potential for multi-layer interaction in our method in the future.

Comment 3: The proposed method is likely to increase computational and storage demands in computing independent representations of each example and reconstructing the representation of the test sample.

1. The computational demands for computing independent representations for each example and reconstructing the representation of the test sample are much lower than the traditional ICL. With our approach, the model only needs to process one example at a time instead of the $k$ concatenated examples, resulting in lower GPU memory usage. The computing latency caused by our method is also within an acceptable range.
2. While it is true that we require memory for storing representation vectors, advances in text retrieval have succeeded in reducing the memory required for vectors, such as through optimized vector databases. Therefore, when using the techniques in these vector databases, the memory will not be large.
3. When applying our method in real-world scenarios, such as determining whether the intent of a user is positive or negative, we find that our approach is faster than traditional ICL because we pre-compute the representation vectors ahead of time.

Comment 4: For example, the prompts used in Section 2 and the main experiments are not provided, and the methodology for selecting hyperparameters in Equation 11 is not introduced, and the references of the baselines used in the main experiment are missing.

Thanks for your careful reading of these details.

1. The prompts used in the experiments are presented in the below Table. We will modify the paper to include the prompts.
2. We will add the discussion of selecting hyperparameters in Equation 11 to our paper. The selection relies on the intuition and some simple experiments. We discover that it is important to assign larger weights to the representation of the original test input, rather than giving the same weights to them.
3. The references of the baselines used in the main experiment is in Section 2.1 Analysis Settings on page 2. From your comment, we realize it may be not friendly for readers to find the baselines. We will add the references of the baselines in every experiment section. Specifically, we compare our method with zero-shot inference and traditional ICL [1] with gold input-label pairs from the training set.

[1] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.

Comment 4: It would be helpful if the paper included an analysis of the proposed method’s efficiency. While accuracy is critical for evaluating performance, the method’s efficiency is also a key factor in understanding its practical implications.

Your reminder is helpful for readers to better understand our method. We will include a more detailed efficiency analysis in the paper.

1. Time Efficiency: When conducting per-dataset testing, our method is slightly slower than traditional ICL due to the retrieving of unlabeled examples. Therefore, we choose BM25 in this paper for its efficiency. However, when applying our method in real-world scenarios, such as determining whether the intent of a user is positive or negative, we find that our approach is faster than traditional ICL because we pre-compute the representation vectors ahead of time.
2. Computational Efficiency: The computational demands for computing independent representations for each example and reconstructing the representation of the test sample are much lower than the traditional ICL. With our approach, the model only needs to process one example at a time instead of the $k$ concatenated examples, resulting in lower GPU memory usage. The computing latency caused by our method is also within an acceptable range.
3. Memory Efficiency: While it is true that we require memory for storing representation vectors, advances in text retrieval have succeeded in reducing the memory required for vectors, such as through optimized vector databases. Therefore, when using the techniques in these vector databases, the memory will not be large.

Comment 5: I am curious why sentences from the test set are used to construct representations for the method’s computation.

Thank you for raising an interesting question. The motivation of using unlabeled texts from the test dataset stems from two factors. First, obtaining gold standard demonstrations in real-world scenarios can be challenging. Therefore, there is a need to develop a method for utilizing unlabeled texts for in-context learning. Second, as shown in Table 2 on page 3, we observe an increase in ICL performance when labels are removed. Taking into account these two factors, we have decided to incorporate unlabeled texts for in-context learning.

Comment 1: While assumptions like label appearance and weak semantic relevance are intriguing, I am somewhat doubtful, especially regarding weak semantic relevance. In pre-training, language models are exposed to a substantial variety of cases, some of which might closely resemble in-context learning scenarios. For instance, if multi-choice QA datasets were included in pre-training, the answer choices could introduce sequences that appear relatively unrelated. Thus, it may not be easy to guarantee that LMs are unfamiliar with inputs seen in in-context learning scenarios.

We appreciate your careful reading of this assumption. We also consider this assumption when developing this method. We provide the following evidence to support that LMs may be unfamiliar with inputs in in-context learning scenarios and need better methods to handle the weak semantic relevance between demonstrations.

1. While it is true that in-context learning style datasets like QA datasets may be included in pre-training, the proportion of ICL datasets is relatively small compared to the regular coherent texts. Therefore, LLMs are more likely to remember the regular coherent texts instead of ICL style datasets during pretraining.
2. If LLMs are already acquainted with the inputs in ICL scenarios, changing the order of the demonstrations should not have much impact on performance. In reality, however, performance is actually quite sensitive to the order.
3. The empirical evidence to support the claims about the weak semantic relevance's impact on ICL performance can be found in Table 5 on page 8. The traditional ICL takes in the concatenation of multiple demonstrations which are not necessarily relevant. We compare traditional ICL to our method that processes demonstration inputs and test input independently. As shown in Table 5, the traditional concatenation performance for GPT-Neo-2.7B and Llama2-7B is at least $7.94$ and $10.57$ worse than representing the inputs independently. The notable performance drop for two models indicates the negative impact of weak semantic relevance caused by the traditional concatenation.

Comment 2: Although the proposed method does not utilize label information from the test set, it does use textual information from it. Could we add more reliable baselines, such as performing few-shot ICL with random labels (distinct from a random few-shot retrieval-based baseline) or using self-generated labels?

Thanks for your kind reminder. In Table 6 on page 9, we have already compared our method with the few-shot ICL with gold labels from the training set. The results in Table 6 demonstrate that our method outperforms traditional in-context learning for GPT-Neo-2.7B, Mistral-7B, Llama2-7B, and Llama2-13B in three well-known settings. From your comment, we realize that it may be not easy for readers to find the comparison between our method and few-shot ICL. We will add a brief outline at the beginning of the experimental section to make it easy for readers to check different experiments.

Comment 3: Equation 11 seems somewhat arbitrary, with no explanation provided for why those specific hyperparameters (e.g., 0.4 and 0.6) are applied in the weighted sum. Could you elaborate on this process?

Thanks for your careful reading of the details. We will add the discussion of selecting hyperparameters in Equation 11 to our paper. The selection relies on the intuition and some simple experiments. We discover that it is important to assign larger weights to the representation of the original test input, rather than giving the same weights to them.

Dear Reviewer,

Thank you for the time and effort you have dedicated to reviewing our paper. We sincerely appreciate your insightful comments and suggestions. Your comments have been highly valuable to us, and we have put effort into addressing your concern. We would love to receive feedback from you. If your concern is addressed, we humbly invite the reviewer to consider increasing the score. Your support is deeply appreciated!