CoLLEGe: Concept Embedding Generation for Large Language Models

We thank the review for their helpful comments on our work and for pointing to prior work relevant to our evaluations.

The evaluation is a bit slim and ad-hoc.

All of our evaluations are done zero-shot, without any additional finetuning. We built our tasks to test a wide array of settings, including complex verbal reasoning, definition modeling, and a more natural slang setting. CoLLEGe is trained on general purpose language modeling data to enable broad transfer.

GRE experiment seems fine, however I was't able to find any discussion about the role of definitions, and given that increasing the number of definitions for the considered terms boosts performance significantly [...].

To clarify, we only present the model with one definition alongside 1–4 support sequences. For GRE, we treat each answer choice as a new concept represented with a new token.

Definition generation [...]

Thank you for the pointers to this research. With the exception of Dict-BERT, we were unaware of these, so this is very helpful. We reached out to the authors of Generationary, as suggested, but have yet to hear a response. We were able to compare our work with that of Giulianelli et. al. (2023) by generating definitions 1-shot from their Oxford dataset and replacing the target word with a new token. The results can be found here. CoLLEGe and Prompting + CoLLEGe score comparably to FLAN models trained for definition generation when the target word is replaced with a new token.

We also evaluated the FLAN models on our task, with results here. CoLLEGe outperforms in terms of ROUGE-L and performs on-par or slightly better than FLAN when evaluated with BERT-F1. Importantly, the FLAN models are trained specifically to generate definitions given an example sentence. CoLLEGe, on the other hand, does not undergo any additional training or finetuning on definition modeling and is able to perform about the same or better.

Twitter slang experiment [...]

Thank you for the suggestion. Our Twitter task is related to WiC but distinct. WiC is binary classification, but in Twitter we provide a choice between one correct and three incorrect (slang, definition) pairs. We are in the process of comparing these methods to ours on both WiC and on our Twitter task, and we plan to add these additional results during discussion.

We thank the reviewer for their helpful suggestions and comments on our work.

Several works have been proposed recently on learning static vector representations of concepts such as MirrorBERT, Numberbach, etc. Perhaps authors should discuss these concept embeddings and use them instead of simply averaging mentions.

This is an interesting idea, which we may explore in future work. MirrorBERT seems like a promising embedding method and worth trying. Numberbatch embeddings may be less directly applicable, but they may be useful alongside contextual embeddings.

more detail about the sampling and knowledge distillation process in section 3

We first select words to be used as new concepts during training based on word frequency in the original training corpus, since we want words which do not appear too often, because they would be too easy to learn, or words that are too rare, because their representation would be too noisy. For each such word, we identify pairs of query and support sequences.

For distillation, we retain the query sequence with the original word form (e.g. for “My ◾is complaining I never have any time to relax”, the original would be “My beige flag is complaining I never have any time to relax”). The intuition is that CoLLEGe should represent “is complaining I never have any time to relax” in the query sequence similarly to how LLaMA would represent it in the original sequence. This encourages CoLLEGe to generate embeddings such that the output embeddings and the logits for that subsequence are close (in cosine distance and mean-squared error, respectively) to those of the LLaMA model for that same subsequence originally

Could the authors explain how the concept embedding is taken into account in the LLM to produce the definition?

We sample from the LLM conditioned on the prompt "The word '◾` is defined as'" in order to generate a definition for the new concept. CoLLEGe learns to generate an embedding for the new concept token ◾, and that embedding is projected to the input and output embedding space of the LLaMA model. The LLaMA model then makes use of those embeddings when applying the embedding the prompt (and subsequent sampled tokens) and when it computes the output logits.

Could the authors add more clarification about the sampling process?

We believe you are referring to the sampling process for the definition generation task. All of our definitions are generated using greedy decoding at temperature = 1.0.

We thank the reviewer for their helpful comments about our work.

New concepts come constantly and naturally and should be identified and learned naturally by the language model itself in the particular contexts of the concepts.

This is an interesting direction for future research and we believe CoLLEGe is an important first step in addressing it. In order to eventually identify and learn new concepts constantly, we first develop a method which can learn identifiable new concepts on the fly.

After identifying such concepts, positive and negative examples have to be prepared in sophisticated ways in order to learn the concepts.

Our goal is to present a method which learns a new concept given only a few examples. To train our model, select words to be used as new concepts and train our model to predict query sequences using them with a few support sequences. The new concept is represented by a new token, guaranteeing that the language model has not encountered it before. Negative examples along and distillation are used to improve the generated embeddings.

Training even involves meta-learning, which makes the task even more difficult.

We choose a meta-learning objective that blends into the pretraining objective, which is not much more difficult than standard autoregressive training and allows CoLLEGe to generate task-general embeddings with rich semantic knowledge.

How do you know that the language model does not have a concept already in the system? Experiments are conducted but again artificial.

In our Twitter evaluation, we hand-select very recent slang terms. Some incorrect answers to questions about slang terms generated by ChatGPT-3.5 and LLaMA-2 can be seen here. Additionally, our Twitter and GRE evaluations contain multi-word phrases as single concepts, which LLaMA-2 model has not seen. Finally, if LLaMA could simply make use of its pretraining knowledge, then the baselines would not perform so poorly.

The first area is model editing. Another area is continual learning, [...]

Thank you for these pointers. We intend to cite them as relevant background, but do not believe they are directly relevant. Applying model editing to the new concept learning setting would mean manually editing the LM for each new concept, which seems more artificial and sidesteps our research problem. Also, our problem is not continually adapting to new domains or tasks, like in Ke et. al.

Thank you for your response and for increasing your score, as well as for suggesting we compare with Knowledge Editing methods. To address your concerns, we evaluated some of these methods on our GRE task. Here, we use the definition of the new concept for editing and also present support sequences in-context. IKE [1] uses in-context learning for knowledge editing, whereas ROME [2] and MEND [3] update the model weights. To train MEND, we use the zsRE dataset and we use the implementations provided in EasyEdit [4]. In the first table below, we provide only support sequences in-context and in the second we also provide definitions.

We find that knowledge editing methods score better than our Prompting baseline in some cases, when the definition is not provided in-context, but around the same or slightly worse when it is provided. In either case, CoLLEGe achieves much higher accuracy than all KE methods.

We also tried using the support sequences as well as the definition during editing. Unfortunately, the time it took to evaluate ROME in this setting was much longer, and we were only able to complete 2 evaluation runs (as opposed to 5 above). Nonetheless, we found that performance was sometimes a bit worse than if we just use the definition for editing. As above, in the first table we provide only support sequences in-context and in the second we also provide definitions. In both cases, definitions and support sequences are used for editing.

We find that ROME performs worse than Prompting in most cases, and each achieves lower accuracy than CoLLEGe.

[1] Zheng, Ce, et al. "Can We Edit Factual Knowledge by In-Context Learning?." Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

[2] Meng, Kevin, et al. "Locating and editing factual associations in GPT." Advances in Neural Information Processing Systems 35 (2022): 17359-17372.

[3] Mitchell, Eric, et al. "Fast model editing at scale." arXiv preprint arXiv:2110.11309 (2021).

[4] Wang, Peng, et al. "Easyedit: An easy-to-use knowledge editing framework for large language models." arXiv preprint arXiv:2308.07269 (2023).

During the evaluation, we noticed that KE methods took much longer to evaluate than CoLLEGe. To quantify this, we computed the average amount of time each method took on each GRE evaluation example, which we present below. CoLLEGe and Prompting + CoLLEGe are 7x and 3x, respectively, faster than the fastest KE method, MEND. CoLLEGe is also 50x faster than the slowest method, IKE. One caveat is that we are using the implementation of these methods from EasyEdit and did not separately optimize their efficiency.