Resolving Lexical Bias in Model Editing

Comment: Although adaptable across architectures, the paper may not fully explore how the approach scales with increasingly larger and more complex models

The proposed approach operates at a single layer of the model, making it lightweight and efficient, with minimal dependence on model size. Scalability is not a major concern, as the method avoids full model retraining and extensive parameter updates. In our experiments, we employed a direct output strategy by modifying the architecture to support multi-output generation. However, generation control mechanisms such as playback vectors and lightweight LoRA blocks have also demonstrated robust performance and can serve as effective alternatives to direct output. These design choices suggest that the approach can scale well to larger models or more complex generation tasks without incurring significant computational overhead.

Comment: The success of precise edit localization hinges on the quality of the learned disentangled representation, which might require extensive tuning and may not generalize perfectly across all domains.

Our analysis identifies the key factor contributing to locality challenges in the model’s representation space, which is lexical bias. We have evaluated PENME’s generalization capabilities by assessing performance transfer from the CounterFact to the zsRE dataset. While our current experiments focus on general QA data, the results suggest that PENME can adapt to new domains with minimal supervision. We hypothesize that for specialized domains, PENME can generalize with a small amount of in-domain data.

Comments: There is no definition given for Lexical Bias in the paper.

The evidence presentation is a bit messed up. For example, Figure 2 uses "Irrelevant Prompts" while Figure 3 uses "Neighbors". ....... should be Llama instead of LAMA

There are inconsistencies between figures. For example, Figure 1 and Figure 4 shows the same evaluation,... than gpt2 in Figure 1 while has smaller percentage in Figure 4.

We thank the reviewer for pointing out the typographical error; neighbours and irrelevant prompts refer to the same examples. This will be corrected in the final version. We will improve the images so that the bars are separated, we provide colour correction for Figure 5: (https://imgur.com/a/bFPCBEB).

Regarding figures 1 and 4, the experiments were conducted using different random splits, which accounts for the observed discrepancies in the reported numbers. We will update the figures to ensure consistency in the reported percentages.

We define lexical bias as prompts with similar lexical tokens but different semantics that are closer together in the representation space as compared to a prompt and its respective paraphrases.

Comment: For locality, do you compute the train retain rate or the test retain rate (c.f. GRACE paper)?

We compute the locality metric on the test set same as GRACE.

Comment: It's not clear how the codebook ..... LORA indices, how to implement this?

We apply the projector once after receiving the user input. From that point, the generation process proceeds normally without intervention—unless the similarity between the input and the nearest codebook key falls below a predefined threshold. In such cases, memory retrieval is triggered. As you have pointed out, the values are stored as strings and they are simply appended to the end of the generated sequence. These values are the edits themselves. Learned vectors can be used to steer the model's generation by adding them to the token representations after identifying the edit scope. Similarly, LoRA blocks can be trained per edit to modulate generation in response to each edit.

Comment: It's not clear which token embedding is used for training the projection network. The authors only mentioned that the projection network is applied to only one certain layer, but it's not clear the embedding of which token in the prompts / answers is used for training.

We use averaged token representations for input encoding. While representations of the final token can also be used to train PENME, they tend to be heavily biased, often inflating similarity between prompts that share the same ending token.

Comments: Many typos / errors in the methodology section. For example, equation 2, .........how to compute the euclidean distance between two strings?

We apologize for the typographical errors. In Equation (2), the argmin operation applies only to the keys $k_i$. The variables $\vec{x_i}$ and $\vec{p_{ij}}$ refer to the output representations from the projection network, which these implied through text rather than a formal definition. We will revise the text to clarify this and eliminate any confusion.

Comments: The 4.3 title is misleading. There is no learning process to determine the threshold and tau. tau is a hyperparameter and the threshold depends on the paraphrases chosen.

We will replace "Learning" from the title with "Finding" to precisely reflect the content of the section.

Comment: Have the authors tried other contrastive learning losses ...thus hurt the locality, which is shown in Table 1 for Counterfact Loc metric.

The balance is implied in terms of average score for generalization and locality in comparison with other approaches, where there is a significant deterioration of one metric as compared to the other. We initially evaluated several contrastive learning approaches and found that none consistently outperformed our formulation. This result aligns with the findings of [1], which demonstrate that, in general, the performance of various metric learning methods tends to be comparable.

Comment: In addition, MEMIT as a baseline ... include this one as a baseline.

MEMIT is a weight modifying approach which improves over ROME with adjustments to model parameters for all layers instead of one. It is a high performing baseline for weight modifying approaches. We will add the citation alongside ROME for the final version. Currently, the work is cited in experimental setup details provided in Appendix D.1. Experimentation Setup.

Comment: Why the Para metric is a bit low on the easier zsRE dataset?

This is for zero-shot transfer from CounterFact to zsRE where some drop in performance is expected. Despite no training on zsRE, PENME still outperforms most methods.

[1]Musgrave, Kevin, Serge Belongie, and Ser-Nam Lim. "A metric learning reality check." In Computer Vision–ECCV 2020: 16th European Conference"

Comments: The contribution is incremental rather than fundamental it represents an important refinement to adapter based editing but does not propose a fundamentally new editing mechanism

The applicability to complex generation tasks for example long form generation reasoning tasks is not explored

Q1, How does PENME perform on long form generation tasks where maintaining locality over extended sequences may be challenging?

We appreciate the reviewer’s perspective. However, we respectfully argue that our work addresses a fundamental and previously underexplored challenge in the weight-preserving model editing paradigm—namely, the problem of scoping, i.e., determining when an input should invoke the edited knowledge. Our proposed scoping mechanism contributes to the weight-preserving model editing paradigm and is not limited to our editing method, PENME. Moreover, the lexical bias has been highlighted as a substantial issue in text and vision models in the recent paper [1]. Our work is the first work that attempts to address this issue.

We did not test PENME on long-form generation due to limited available datasets and baselines for meaningful comparison. Long-form generation may require incorporating techniques such as LORA or other lightweight trainable components. We believe that this is out of the scope of the current focus of the paper and is a promising direction for future work.

Comment: Figure 3 and Figure 7 illustrate the "Percentage of samples where edits are closer to unrelated neighbors," but this is insufficient to demonstrate lexical bias. At lower model layers, high similarity may result from underdeveloped sentence representations, while at higher layers, the reduced percentage indicates greater differentiation between sentences.

We agree with the reviewer that lower layers of a model may encode less developed sentence representations, while higher layers tend to capture more abstract semantic similarity. However, as shown in the bar chart in Figure 3, lexical bias persists even in the higher layers, suggesting that this issue is not confined to shallow representations. Furthermore, similar findings have been reported by [1], who observed significant lexical bias in representations across a diverse range of vision and text language models. These observations reinforce the broader relevance of the issue we address in this work.

Comment: Q2, Could the authors comment on PENMEs generalization under domain shift for example specialized domains like healthcare or legal texts?

We have evaluated PENME's generalization capabilities by assessing performance transfer from the CounterFact to the zsRE dataset. While our current experiments focus on general QA data, the results suggest that PENME can adapt to new domains with minimal supervision. We hypothesize that for specialized domains such as healthcare or legal texts, with a small amount of in-domain data, PENME would generalize effectively to any domain.

Comment: Q3, Have the authors considered adversarial training to further enhance the robustness of locality and paraphrase generalization?

The CounterFact dataset used in our experiments is inherently adversarial, as irrelevant prompts often share significant lexical and semantic overlap (e.g. the subjects are both actors) with the edited facts. Additional adversarial examples could indeed be constructed using larger language models to further enhance robustness and generalization. However, generating such data at scale would increase computational cost and resource requirements for training. This is a valuable future direction that leads the way towards PENME generalization to a wide set of domains.

Comment: Q4, How might PENMEs projection space handle multi hop reasoning prompts where lexical and semantic relationships are more complex?

We posit that a knowledge base will be needed, and such prompts would need to be stored in the codebook with links to the main edit. PENME's learned representation space would allow it to localize this information better in the representation space, thus allowing for scoping multi-hop or one-hop questions. At inference time, these can be used to link back to the edited information, which can then be used for informed generation through some training of additional parameters, e.g. LoRA block.

[1] Dumpala, S. H., Jaiswal, A., Sastry, C., Milios, E., Oore, S., and Sajjad, H. SUGARCREPE++ Dataset: Vision Language Model Sensitivity to Semantic and Lexical Alterations. In Conference on Neural Information Processing Systems, Dataset Track (NeurIPS), 2024.

Comment: The paper establishes the issue of lexical bias in model editing which provides strong insights. Additionally the PENME framework is a intuitive next step. However, this leads PENME's contributions to be less significant methodologically.

While we understand that PENME may appear as a natural extension, we respectfully argue that our work addresses a fundamental and previously underexplored challenge in the weight-preserving model editing paradigm—namely, the problem of scoping, i.e., determining when an input should invoke the edited knowledge. Our proposed scoping mechanism contributes to the weight-preserving model editing paradigm and is not limited to our editing method, PENME. Moreover, the lexical bias has been highlighted as a substantial issue in text and vision models in the recent paper [1]. Our work is the first work that attempts to address this issue.

[1] Dumpala, S. H., Jaiswal, A., Sastry, C., Milios, E., Oore, S., and Sajjad, H. SUGARCREPE++ Dataset: Vision Language Model Sensitivity to Semantic and Lexical Alterations. In Conference on Neural Information Processing Systems, Dataset Track (NeurIPS), 2024.

Comment:"The only issue was that no downstream performance analysis done."

We evaluate the general capabilities of the Llama-2-7b used in our study and compare its downstream performance before and after applying PENME. To ensure a diverse and representative assessment, we select three distinct tasks: Natural Language Inference-NLI (RTE dataset) evaluated using F1 score, Summarization (CNN/DailyMail) assessed via average ROUGE-1, ROUGE-2, and ROUGE-L scores, and Sentiment Classification (Diar-AI/Emotions), using F1 score.

The results show competitive performance across all tasks, with only a small drop in performance in the NLI task.

Comments:" How do these edits interfere... generation of every token?

In discussing the construction ... didn't explain how to obtain the corresponding values.

How are the values incorporated... What is the inference process?

How does this work .... generation of every token?"

We apply the projector once after receiving the user input. From that point, the generation process proceeds normally without intervention—unless the similarity between the input and the nearest codebook key falls below a predefined threshold. In such cases, memory retrieval is triggered. As you have pointed out, the values are stored as strings and they are simply appended to the end of the generated sequence. These values are the edits themselves. We did not evaluate generative settings where multi-token or continued generation was necessary. Instead of using a stored string, we could use the approaches examined in GRACE, where a learned vector is the value, and MELO, where a LoRA block is the value. As a point of emphasis, our proposed scoping mechanism is independent of these design choices, and any generation method can be freely selected.

Comment:"The reason why parameter preserving .... evaluations for 2k edits is does no longer enough .... effectiveness for these methods."

While AlphaEdit does improve upon MEMIT, it is important to clarify that the “sequential” setting used in AlphaEdit involves 3,000 edits applied in batches of 100, making it more accurately described as consecutive batch editing. In contrast, true sequential editing—where updates are applied one at a time, as explored in GRACE—poses a more challenging scenario due to the cumulative nature of parameter shifts (3000 vs 30 updates). We provide results for 3000 sequential edits on the Llama-2-7b model. PENME shows high performance with a minor drop in generalization and locality metrics.

Comment: "Consecutive Model Editing ....one of the baselines which came out in March 2024"

We thank the reviewer for highlighting this work. While the initial codebase of the work supports GPT series models, adapting it to LLaMA requires additional effort. Notably, Table 1 in the paper shows lower generalization and edit success performance on GPT2-XL compared with our method, PENME. We will add the comparison in the final version of the paper.

Comments:"How does the key storage ... I dont really know GRACE or MELO?

Claim 2: Use contrastive learning .... But the improvement is... not as clear for CounterFact. ..."

GRACE and MELO store direct model representations corresponding to the final hidden token of the input. Their approach relies on maintaining a very low similarity threshold, which necessitates adding a large number of entries to the codebook per edit. To mitigate this overhead, they merge entries with similar outputs. However, when new edits fall within the similarity threshold, the similarity thresholds are reduced for these edits. This threshold adjustment and merging strategy can result in edit forgetting, as discussed in the reference section.

In contrast, PENME avoids the need to store multiple codebook entries by learning a projection space where irrelevant inputs are mapped farther from each other, while paraphrased edits remain close. This design enables faster retrieval and helps prevent issues like edit clashes and forgetting, without compromising precision.

We evaluate GRACE by increasing its similarity threshold on GPT2-XL, aiming to match the generalization performance of PENME. The results overall demonstrate how PENME improves upon the direct use of representations.

f"""You have a new fact: \{edit prompt\}.
Based on this fact, complete the following sentence to answer the question: \{query\}
Your answer should specifically incorporate the new fact I've shared.

Paragraph: \{query\}"""