Resolving Lexical Bias in Edit Scoping with Projector Editor Networks

We thank Reviewer uwxU for their time and feedback, which we address below.

1. It is difficult to tell exactly what parts of the paper are novel contributions. I think the main difference is that in GRACE the cookbook representations are manually maintained whereas here they're learned. The related work section needs to tell the reader why this work is different from the previous works, not just describing them.

Novelty: This is the first work that identifies the issue of lexical bias in model editing. The explicit modelling of a representation space designed to improve paraphrase success while minimizing mismatch with neighbouring examples is a novel solution to the problem at hand. To the best of our knowledge, PENME is the only model editing method designed to maintain both high locality and generalization. The results in Table 1 demonstrate the efficacy of PENME where none of the other methods, whether weight-preserving or weight-modifying, improves both locality and generalization. We have highlighted the novelty of our work in the introduction. We have further improved the Related Work section with a more explicit comparison between our method and other model-editing methods.

The key distinction between GRACE [1] and PENME lies in their approach to model representations: GRACE utilizes a codebook with cached representations as keys, whereas PENME employs learned representations generated by a projection network. GRACE adopts small thresholds or deferral radii to address challenges with neighbourhood prompts; however, this approach is less effective when lexically similar prompts are in close proximity. Additionally, maintaining small radii necessitates storing paraphrases as extra codebook entries alongside their corresponding edits to enhance generalization, which significantly increases retrieval time. The radius-based design further introduces the risk of edit forgetting, as overlapping cluster radii for similar edits necessitate the resizing of radii. This creates a trade-off between locality and generalization, where small radii preserve locality at the expense of generalization, and larger radii enhance generalization but reduce edit localization. In contrast, PENME effectively addresses these issues by managing lexically similar prompts more robustly and leveraging its learned representation space to restrict each edit to a single codebook entry, thereby simplifying storage and retrieval processes.

To illustrate the aforementioned challenges in GRACE and how PENME alleviates these issues, we present an ablation experiment comparing PENME with GRACE across various sample sizes of edits, ranging from 50 to 300 in increments of 50, on the Counterfact dataset, as shown in the table below. The results indicate that PENME outperforms GRACE in speed due to the number of codebook entries, while also revealing key shortcomings in their scoping methods. Specifically, the 'Edits Forgotten' and 'Edit Conflict' columns highlight the significant number of keys lost during training due to conflicts in the deferral radii of edits.

We have updated Section (RELATED WORK) to highlight this better.

2. The results in Table 1 show that GRACE is a strong baseline. PENME, which extends GRACE by using a projection network to map data representations, needs to clearly highlight the differences between PENME and GRACE.

Please refer to our central response for clarification.

3. Figure 3 and Figure 7 illustrate the "Percentage of samples where edits are closer to unrelated neighbors," but this is insufficient to demonstrate lexical bias. At lower model layers, high similarity may result from underdeveloped sentence representations, while at higher layers, the reduced percentage indicates greater differentiation between sentences.

We agree that the lower layers of a model may have underdeveloped sentence representations, while higher layers may have developed a better sense of semantic similarity. The Figure 3 bar chart shows that the lexical bias exists in higher layers as well; however, it is relatively low compared to lower layers. Moreover, the issue of lexical bias has also been observed in the recent NeurIPs paper [1], which specifically analyzed the last layer representations of a diverse set of large language models.

4. PENME focuses on addressing lexical bias, so it should perform well on Loc and Para. However, in Table 1, only Para shows improvement, which is insufficient to fully support the paper's contributions.

PENME achieves a balanced outcome, delivering high scores in both generalization and locality without compromising performance in either aspect. The deferral radius in GRACE [2] and MELO [3] which utilize the same retrieval system is dynamically adjusted, initially set to a small epsilon to avoid interference with other prompts, ensuring high locality, while still permitting the execution of some paraphrases for generalization. It is important to note that multiple paraphrases during training are essential for achieving effective generalization. While the default deferral radius is small, enabling high locality, it leads to a redundant solution with minimal generalization as evident from the results. If the radius is expanded it leads to better generalization but at the cost of locality, given that lexically similar neighbours exhibit proximity to the edits, this leads to low locality scores.

5. Some results in Table 1 is not clear enough (close to 0.0), such as why GRACE on zsRE get the 0.00 in Loc on Llama2-7b?

For our evaluations, we employ the default parameters and layer settings provided in the EasyEdit library [4]. It is important to note that Llama-2-7b operates within a representation space that differs significantly from that of the GPT series models. Alternative model editing approaches, such as MEMIT, have been shown to perform sub-optimally for model editing tasks in this context. We direct the authors to a recent NeurIPs paper [5], where the appendix Section D (Appendix /Additional Experiments) demonstrates that the results of MEMIT and GRACE for the LLaMA model align closely with those observed in our study.

6. In Figure 6, it is better to add the resutls on GRACE.

Conducting model editing methods evaluations across different scaling experiments is computationally intensive. To address concerns related to scalability, we focused our primary experiments on comparing PENME's performance with other editing approaches for 2,000 edits, as shown in Table 1. Notably, experiments in relevant literature typically evaluate performance with up to 1,000 edits [2], [3], [5].

[1] Sri Harsha Dumpala, Aman Jaiswal, Chandramouli Sastry, Evangelos Milios, Sageev Oore, and Hassan Sajjad (2024). Sugarcrepe++ dataset: Vision-language model sensitivity to semantic and lexical alterations. In Proceedings of NeurIPS.

[2] Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems

[3] Lang Yu, Qin Chen, Jie Zhou, and Liang He. Melo: Enhancing model editing with neuron-indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence.

[4] Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, et al. Easyedit: An easy-to-use knowledge editing framework for large language models. arXiv preprint arXiv:2308.07269, 2023a

[5] Xiusheng Huang, Jiaxiang Liu, Yequan Wang, and Kang Liu. Reasons and solutions for the decline in model performance after editing. In Proceedings of NeurIPS.

We appreciate Reviewer hiiz's time and attention in reviewing our paper, as well as the detailed feedback, which we intend to address below.

1. This paper suggests that lexical bias refers to different editing subjects with the same relation, such as "The twin city of Pittsburgh is" and "The twin city of Portsmouth is." However, the prevalence of such cases in the CounterFact and ZsRE datasets is unclear.

It is better to give more example to shown what is lexical bias and lexical overlap in the paper.

To quantify lexical bias, we compute token overlap using Jaccard similarity and ROUGE metrics between pairs of (edits, paraphrases) $(x_i, p_{ij})$ and (edits, neighbours) $(x_i, nb_{ij})$, and also present examples from both datasets in the tables below. From the token overlap metrics table, it is evident that the edit prompt and neighbours show a high overlap in Counterfact, whereas the overlap is minimal in ZsRE. The table with dataset examples further demonstrates that neighbours share a similar lexical structure but often refer to different entities. Interestingly, these entities can also be semantically related, as seen in the fourth example, where Thomas Arne and Bill Brandt are both well-known British figures.

This, coupled with the experiment in Section 6.1 (LEXICAL DOMINANCE) highlights the challenging nature of the Counterfact dataset, attributable to its inherent lexical bias.

---

Random Samples from the Counterfact and ZsRE Datasets

---

As highlighted in our response to Part 1, GRACE falls short of satisfying a fundamental criterion of model editing: meaningful generalization. While its high locality may initially appear noteworthy, the results underscore a critical limitation of GRACE, as it achieves less than $2$% generalization on Counterfact across all models. This performance is comparable to a simplistic input-matching mechanism reliant on caching. The primary scientific challenge in model editing lies not only in preserving locality but also in ensuring that targeted factual updates propagate effectively across semantically related queries. PENME is designed to address both the challenges of generalization and locality, demonstrating strong performance in both areas.

To illustrate how GRACE's performance evolves as its deferral radii are adjusted in an attempt to approach PENME's generalization performance, we increase the deferral radii and evaluate the approach on 2000 Counterfact samples for GPT2-XL, as shown in the table below.

Thank you for your response. The results indeed demonstrate the advantages of PENME in scenarios involving extensive continual editing.

However, it seems to rely on a crucial component, the "PROJECTION NETWORK," which raises a question: Is the PROJECTION NETWORK pre-trained, or does it play a role in continual learning?

In the context of continual editing, the assumption is that the model do not have all the knowledge updates at once but receive them sequentially.

Therefore, the training of the PROJECTION NETWORK appears to be a small-sample training problem. Has the author considered this aspect?

You could check the paper "https://arxiv.org/pdf/2405.14768", they didn't use all edits before editing, but for PROJECTION NETWORK, does it need all edits for training?

We would like to thank Reviewer UAfd for taking the time to review our work and for the useful feedback. We hope to address the stated concerns below.

1. The writing, especially in the experiments section, needs clarification. The baseline setup is hard to follow as the authors haven't provided an overview of all compared baselines (e.g., MELO). For instance, it’s unclear why Llama-2-7b wasn’t tested on MELO and no T5 for MEMIT.

We apologize for the lack of clarity. As detailed in Appendix B.2, we utilize the EasyEdit library [1] for our evaluations. This library extends the original codebases for model editing approaches, enabling the use of additional models that were not supported previously. However, it is important to note that not all models are currently supported for every approach (e.g., Llama-2-7b for MELO). Additionally, MEMIT is specifically designed to work with decoder-only models, which is why we did not conduct evaluations using the T5 model. Working details for MELO are provided in the Related Work Section of the main text.

2. Could you provide a visualization of the representation space showing edits, paraphrases, and neighbours before and after editing? This would nicely complement the current analysis.

We have included the requested visualization in Appendix Section F (VISUALIZATIONS). The visualization shows that in model representations edits are closer to their neighbours than to their respective paraphrases. Additionally, edits tend to be closer to other edits, which can result in misfires in similarity retrieval systems. In the representation space of the projection network, we can see that the neighbours are far from the edits and the edits are farther away from each other. This visualization is referenced in Section 6.2 (DISENTANGLED PROJECTION SPACE) of the main paper.

3. Fig6: LAMA$->$LLaMA

Thank you for pointing that out. We have updated the image to rectify this typo.

4. Adding more editing methods in the scaling experiment can help to confirm the robustness of PENME

Conducting model editing methods evaluation across different scaling experiments is computationally intensive. To address concerns related to scalability, we focused our primary experiments on comparing PENME's performance with other editing approaches for 2,000 edits, as shown in Table 1. Notably, experiments in relevant literature typically evaluate performance with up to 1,000 edits [2], [3], [4].

5. The presentation could be improved; all figures are quite blurry and lack high quality.

We apologize for the image quality issues and have updated all images to enhance image quality.

6. An ablation study on each proposed component would strengthen the analysis.

There are three major hyperparameters in PENME. Edit-to-edit pairings, the margin $m$ in the contrastive loss and the data-driven threshold $\tau$. Abalations regarding edit-to-edit pairings and $\tau$ can be found in Section \S7.1 GENERALIZATION AND LOCALITY of the main text. For $m$ we conduct an ablation study and examine its implications for generalization and locality, utilizing 500 samples from the Counterfact dataset with the GPT2-XL model. The table below presents the results with adjustments to $\tau$ for achieving balanced outcomes for the metrics. Margins between 40 and 80 provide a balanced trade-off between generalization and locality. Notably, locality improves with increasing, which can be advantageous in scenarios where minimizing false matches is critical. We have added the results in Appendix Section D.1.2.

12. Why choose L2 distance in the loss function instead of cosine similarity?

Cosine loss only takes into account the direction of the representations, whereas L2 loss considers both the direction and the magnitude. Existing literature indicates that training with a contrastive objective using L2 loss leads to more separable clusters compared to cosine loss. One issue with cosine loss is that reducing the angle between positive representations increases the magnitude of the representations. Thus, enforcing cosine similarity may result in a leaked heuristic that allows the model to manipulate the magnitude in the projection space, leading to unexpected incompatibility with the downstream computation flow. Furthermore, cosine loss is prone to gradient diminishing effects due to this increase in magnitude, as well as when the initial angle between positive pairs is large. We refer the authors to [3] which outlines these issues. We also provide experimental evidence for these issues by training GPT2-XL on 500 samples from Counterfact, where we modify the objective function of the projector network to cosine loss within PENME’s pipeline. The results show that a negative $\tau$ needs to be set, which means that the paraphrases used during training will fail. Moreover, the performance is lower compared to training with the contrastive learning objective.

PENME Cosine Loss

---

PENME Contrastive Loss

[1] Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems

[2] Lang Yu, Qin Chen, Jie Zhou, and Liang He. Melo: Enhancing model editing with neuron-indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence.

[3] Andrew Draganov, Sharvaree Vadgama, and Erik J Bekkers. The hidden pitfalls of the cosine similarity loss. arXiv preprint arXiv:2406.16468, 2024.

We would like to thank Reviewer y63D for taking the time to review our paper and we appreciate the detailed comments which we hope to address below.

1. The authors didn't fully explain their method. In discussing the construction of key-value memory, they described how to create the keys and set the threshold but didn't explain how to obtain the corresponding values. How is the memory value in the key-value memory obtained?

The ZsRE and Counterfact datasets provide edit prompts $x_i$ alongside their corresponding new outputs $y_i$, as well as paraphrases $p_{ij}$ and neighbours $n_{ij}$. The keys are projector network representations and corresponding values containing a learned similarity threshold ($\delta$) and the new associated output $y_i$. The threshold for each edit $x_i$ is determined by computing the Euclidean distance between the projector network representations for the edit and its training paraphrases $p_{ij}$ and choosing the maximally far paraphrase distance + $\tau$ where $\tau$ is a hyperparameter. As highlighted in Section 4 (Projector Editor Networks for Model Editing), alternative playback mechanisms can be seamlessly integrated with this approach, offering a viable alternative to directly storing the new output information. We describe Key-value Memory in detail in Section 4.2.

2. The paper claims that PENME enables faster edit retrieval and simplifies edit removal or updates. However, it lacks supporting experimental evidence.

We conducted an ablation study to evaluate the number of codebook entries and retrieval time, scaling from 50 to 300 edits in increments of 50 samples per experiment. The results, shown below, reveal that both MELO [2] and GRACE [1] demand significantly more codebook entries, leading to slower inference times as the number of edits increases. We have added the results in the Appendix Section (COMPARISON SCOPING MECHANISM: PENME VERSUS MELO AND GRACE).

Caption: Runtime Performance Comparison of PENME versus MELO retrieval system. For PENME, the number of Codebook entries is the same as the number of edits.

3. The hyperparameter m in the loss function is crucial for the projection network's performance, yet the paper lacks ablation studies on this.

What impact does the hyperparameter m have on the projection network's performance?

We conducted an ablation study on the hyperparameter $m$ for the GPT2-XL model. The table below presents the margin $m$ alongside the corresponding adjustments to $\tau$ to achieve a balance between generalization and locality. Margins between 40 and 80 provide a balanced trade-off between generalization and locality. Notably, locality improves with increasing $m$, which can be advantageous in scenarios where minimizing false matches is critical. We have added the results in Appendix Section D.1.2.

1. Q5 has not been answered: Is the memory value obtained through training or other methods?

Response: Please recall from our previous rebuttal (Part 1, 1) response that the memory value is a tuple consisting of (data driven threshold, $y_i$ the edited output from the dataset), we highlight that any playback mechanism such as vector playback or LoRA indexed blocks can be used as well. The threshold is not learned but is a hyperparameter that is empirically determined.

Thank you for raising this point again, we will ensure this is as clear as possible in the paper as a result of this discussion.

2. I agree that placing multiple parameters in a single figure can better demonstrate their combined effects. However, without a clear explanation, it might confuse the readers even more. Additionally, the overlapping lines in Figure 5 further increase the difficulty of interpretation.

Response We have updated the caption which now reads "Figure 5: Shows the trade-off between generalization and locality performance across different hyperparameter settings. The distance threshold $\tau$ varies from $0.01$ to $0.2$ ($0.01$ increments and $\tau$ is normalized by 100), while the edit-pairing similarity threshold $\phi$ ranges from $0.5$ to $0.9$ ($0.1$ increments). Higher $\phi$ values enforce stricter edit similarity requirements. The results showcase the effect of hyperparameter tuning on the projector network's learning capacity and overall performance.". To make the lines more clearly visible we moved the the minimum value on the y-axis from 0.35 to 0.6.

3. When changing the margin m, how was the corresponding threshold determined? Is there a quick way to find a suitable threshold?

Response: The higher the margin m is kept the higher the value for $\tau$ needs to be. As stated before threshold is not learned but but is itself a hyperparameter that is empirically determined based on the data which in the case of Llama-2-7b is 10 for original experiments (Table 1). A quick way to find a suitable value for $\tau$ is to utilize unseen samples. Using 100 unseen samples we find that 6 is the optimal $\tau$ value for a balanced outcome in generalization and locality for those samples. Utilizing this value for original experiments yields the results shown in the table below. We observe that the scores are high for both generalization and locality but with a stronger emphasis on locality.

Dear Reviewer y63D,

As the discussion phase draws to a close, we kindly ask whether our response has resolved your concerns or if there are any remaining issues that we can further clarify. Your insights are invaluable in refining our work, and we are eager to ensure that all your concerns are fully addressed.

Thank you once again for your time and effort in reviewing our manuscript.