Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuning

Re-evaluate ROME and MEMIT.

We thank the reviewer for providing this valuable information. Yes our experiments were conducted based on EasyEdit. In response, we conducted a systematic re-evaluation of MEMIT and ROME following your suggestion. Due to time constraint, we only finished experiments on LLaMA-2 and Gemma. We have updated corresponding results in our draft (highlighted in red). We will update the draft as soon as other experiments are finished. According to our new results, MEMIT and ROME still fall short in challenging sequential (and batched) scenarios, as observed in [1], and perform worse than our BaFT.

[1] Wang et al. WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models. 2024.

Balance efficiency and effectiveness trade-off.

The effectiveness of BaFT can be improved by using more parameters, given that their parameter efficiency is already much higher than WISE. When the latter's parameters number is reduced from "full" to "light", its performance degrades drastically. In comparison, BaFT uses even much less parameters, but maintains a highly comparable performance. Given this, we expect that using better training hyper-parameters such as learning rate to make mild performance improvement, and more parameters are needed. To validate this, we tried to add intervention to all layers (a common practice in ReFT [1]) and increase the subspace rank to 16. This made BaFT performance on editing LLaMA-2 with 100 ZsRE knowledge increased from 0.80 (Rel: 0.73, Gen: 0.68, Loc: 0.98) to 0.82 (Rel: 0.77, Gen: 0.73, Loc: 0.95). However, we noted that going higher subspace rank didn't help. Therefore, we conjecture that to build larger BaFT (and ReFT), we need to incorporate sparsity on basis activation as well. This can help alleviate unintentional parameter updates as in GRACE [1] and WISE [2]. In addition, such a sparsity opens the door of automating position selections: as when all bases are inactivated, BaFT makes no updates on the representation, which is equivalent to dropping the position from the fine-tuning process. We plan to explore this direction in our future work.

As an update representation for achieving editing, I believe this is not a reliable knowledge editing method compared to directly updating parameters.

Updating representations has proven effective in generic post-training as shown in [1], and in this work, we showed that it is also a promising direction for knowledge editing (after fixing the linearity limitation). However, we agree with these successes are not sufficient to draw a conclusive statement that representation-based methods are superior than parameter-based methods. Critically, it is still an open question if representation-based method is capable of fitting any editing learnable for parameter-based methods, and we plan to explore this direction in our future work. We included this discussion in our draft.

[1] Wu et al. ReFT: Representation Finetuning for Language Models. 2024.

Typos

Thank you for catching the error! We fixed this mistake and a few others in the revised draft. We didn't highlight these rewordings in the revised draft.

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

Limitation and Potential issue of BaFT

We are pleased to see that our main contribution: (1) revealing how linear updates can lead to inherent editing-locality trade-off, and (2) how this inherent limitation motivates BaFT as an efficient non-linear generalization, are admired by the reviewer. Following the reviewer's suggestion, we updated our draft to include the following limitations and future improvement of BaFT:

1. The empirical success of BaFT was mainly established on standard benchmarks [1], which may not be sufficient to reflect the diverse real-world applications.

2. BaFT as a generalization of ReFT requires hyper-parameter tuning to determine proper positions and layers to add interventions. Our choice was selected based on recommended values from ReFT [2]. We plan to explore automating this process by imposing proper sparsity constraints on weights in our future work.

3. The promising performance of BaFT demonstrates its potential for efficient knowledge editing. However, it is still an open question if representation-based method is capable of fitting any editing (or updates) learnable for parameter-based methods. We plan to explore this direction in our future work.

[1] EasyEdit. https://github.com/zjunlp/EasyEdit.

[2] Wu et al. ReFT: Representation Finetuning for Language Models. 2024.

The experiments which may not comprehensively reflect the performance of BaFT in various real-world applications.

Our experiment design follows previous works [1, 2]. Following the reviewer's suggestion, we further include unrelated downstream task for additional locality evaluations (please see below for more details), and we showed that BaFT is again highly competitive with existing baselines while maintaining notable parameter efficiency. However, we agree that existing benchmarks are insufficient to reflect diverse real-world applications. In response, we discuses this as a limitation in the updated draft, and will explore knowledge editing using unstructured data or in the wild.

[1] EasyEdit. https://github.com/zjunlp/EasyEdit.

[2] Wang et al. WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models. 2024.

Previous work REMEDI.

We thank the reviewer for bringing this paper to our attention. After carefully reading through the paper, we agree that REMEDI is an important work that studied if knowledge can be updated in a pretrained model through the lens of modifying representations. We updated the "first work" claim in our draft accordingly. We noted that this does not affect the main contributions of our work (as summarized by the reviewer, being "the first" alone is not a key contribution). In addition, we noted the following difference between REMEDI and ours:

1. REMEDI relies on a large-scale dataset to train the model. This can be challenging to obtain especially in frequent knowledge editing, and is not assumed accessible in our setting.
2. REMEDI still applies a linear (affine) update and may suffer from the same linearity limitation.

Assumption 2.2 has been proposed in GRACE.

We thank the reviewer for pointing out this connection. After rechecking the paper, we agree that the two assumptions share the same idea. We have updated the draft to highlight this.

the "locality" of these methods to downstream tasks.

We agree with the reviewer that downstream tasks as a measure of to forgetting issue can provide useful information. Following the suggestion, we experiment the model performance on a commonsense dataset as in [1]. Due to time constraint, we only tested continually editing LLaMA 2-7b with 100 ZsRE knowledge. The results are reported in the Table below and have been updated in the Appendix of our draft. We noted that our BaFT achieved competitive locality as state-of-the-art baselines.

[1] Yao et al. Editing large language models: Problems, methods, and opportunities. 2023.

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

Implementation Details.

We implemented ReFT as a special case of BaFT, and set their shared parameters to the same values tuned based on a model basis using ReFT. One expectation is in batched editing --- as we used larger batch size here, we used smaller learning rates, and more iteration numbers. Therefore, when editing different knowledge, all the hyper-parameters are the same. We provide details from the question. For others, please refer to Table 5 in App B of our draft.

• Position: Following ReFT [1], we tuned the position as a hyper-parameter. In execution, we chose to learn (and apply) to the last 3 tokens in the input, and all in the output. This is the same for different knowledge.

• Layer: We altered layer 9; 18; 24; 28 of LLaMA-2 and LLaMA-3; and layer 18;20;22;24 of Gemma. These choices were made as a simple subset of recommended values from ReFT [1].

Conceptually, BaFT should be less sensitive to the position choice, as it can assign unimportant bases with low weights, and setting all to 0 is equivalent to unchoosing this position. We plan to explore automate position and layer selection in BaFT in our future work.

[1] Wu et al. ReFT: Representation Finetuning for Language Models. 2024.

why the loss used in RaFT can alleviate the problem?

Conceptually, the proof of Thm 2.3, how ReFT suffers from the general-loc trade-off, heavily relies on the linearity of ReFT. In words, the linearity "guarantees" that locality will be affected. Our BaFT, on the contrary, is non-linear. This makes the above proof fail to apply, and this "guarantee" disappear.

How BaFT alleviates the problem lies in not only its non-linearity, but also how it handles different data non-linearly, as non-linearity itself does not directly imply better trade-off. We take the insight that the failure of linear updates in ReFT stems from the fact that it applies to all inputs indiscriminately. Built upon this, we learn to "downweight" the influence of (unneeded) updates on unrelated inputs, as formulated by the loss used in BaFT. To be specific, BaFT conducts a fine-grained basis (directional)-level "downweighting". In Sec 3.4, we showed that doing a coarse subspace-level control can result in worse trade-off.

Finally, we agree with the reviewer that a more in-depth theoretical analysis on the expressiveness of BaFT can be helpful, and and we will explore this direction in our future work.

Assumption 2.2 is a bit subjective.

After checking the literature, we noted that a similar assumption has been made in GRACE [1]. We have updated the draft accordingly.

[1] Hartvigsen et al. Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors. 2023.

Typos

Thank you for catching the error! We fixed this mistake and a few others in the revised draft. We didn't highlight these rewordings in the revised draft.

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

The improvements of the method presented in this paper compared to ReFT are limited.

This work establishes a formal analysis on how fine-tuning representation can be applied to edit learned knowledge in LLMs. To this end, we first show that how linearity in existing representation fine-tuning can induce an inherent trade-off between editing new knowledge and maintaining unrelated knowledge. Built upon this, we proposed BaFT as a flexible framework to break linearity. Conceptually, BaFT allows us to control how different representations are edited in the subspace a case-by-case way. We would like to highlight BaFT does not directly target on a better editing performance (here refers to learning new knowledge): By design, BaFT and ReFT leverages a same-sized subspace to edit representations. ReFT only needs to fit new knowledge, while BaFT takes locality into consideration as well. Therefore, the improvement of BaFT lies in how it achieves better locality without sacrificing editing capability, as shown in our experiments.

The Applicability of Asmp 2.1 is questioned on tasks involving multi-hop reasoning.

We thank the reviewer for the insightful comments on our Asmp 2.1. Our Thm 2.3, together with the two assumptions, is to reveal how linearity in ReFT can inevitably hurt locality, even if it appears successful in editing. Therefore, our focus is on cases where ReFT is capable of conducting the edits, given its effectiveness in diverse post-training tasks [1]. Presuming such a success, given that ReFT can only update representations, we assume that by updating representations to some targeted (possibly unknown) value, ReFT steers output $y$ to convey the desired knowledge, as presented in Asmp 2.1.

We have updated our draft to make this clear.

[1] Wu et al. ReFT: Representation Finetuning for Language Models. 2024.

Baseline LTE, MELO, Stable KE, and InstructEdit.

Following the reviwer's suggestion, we update the draft accordingly to incorporate and discuss these related works. We also add MELO (using EasyEdit [1], the benchmark used in this paper) for comparison. Due to time limitation, we only run experiments on LLaMA-2 and Gemma and update corresponding results in our draft (highlighted in red). As shown in the updated draft, we note that the performance of MELO degraded faster than our BaFT in challenging sequential scenarios.

[1] EasyEdit. https://github.com/zjunlp/EasyEdit.

MEMIT results in Batched and Sequential Scenarios.

We thank the reviewer for providing this valuable comment. Our experiments were conducted on EasyEdit [1], and we note that some updates on MEMIT and ROME implementation and evaluations had been made until very recently therein. In response, we conducted a re-evaluation of MEMIT and ROME. Due to time constraint, we only finished experiments on LLaMA-2 and Gemma. We have updated corresponding results in our draft (highlighted in red). Our results now match [2]. We will update the draft as soon as LLaMA-3 experiment is finished. According to our revised result, MEMIT and ROME still fall short in challenging sequential (and batched) scenarios, as observed in [2], and perform worse than our BaFT.

[1] EasyEdit. https://github.com/zjunlp/EasyEdit.

[2] Wang et al. WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models. 2024.

Typos

Thank you for catching the error! We fixed this mistake and a few others in the revised draft. We didn't highlight these rewordings in the revised draft.

Thank you for the detailed response to my questions. However, a major concern remains: the performance of BaFT appears to be significantly limited compared to other baselines, such as ReFT and WISE, even in terms of locality. Furthermore, the authors do not seem to provide a clear explanation for this in the experiments. For instance, it is unclear why BaFT's performance does not surpass that of the baselines.

Additionally, some statements in the paper seem to contradict each other. For example:

1. "BaFT and ReFT used a subspace of the same rank, so its editing performance should be upper bounded by ReFT in this case," and
2. "BaFT was capable of maintaining a better editing-locality trade-off: it achieved better locality and portability than ReFT with no degradation of the editing effectiveness."

In my opinion, these two statements cannot both be true.

Dear reviewer pjBe,

We are glad to see that our response helped address other concerns. In the following, we address these two comments and questions one by one, which will also be updated in the revised paper.

The performance of BaFT appears to be significantly limited compared to other baselines, such as ReFT and WISE, even in terms of locality.

Update:

We realized that the reviewer might refer to Table 1 Singe Edit performance here. In this table, LLaMA-2 baseline results were taken from [1] and showed "higher locality" than our BaFT. However, we noted that these results are unreliable, and there has been a very recent correction update on [1]. In response, we also re-evaluated all methods on LLaMA-3 and Gemma, and updated Table 1 in our draft accordingly. According to the revised results, our BaFT in fact achieved the best (much better than the original) editing performance (AVG) and locality than baselines in all cases.

[1] Zhang et al. A Comprehensive Study of Knowledge Editing for Large Language Models. 2024

---

According to Table 2 in our paper, BaFT outperformed ReFT in all cases in achieving higher locality. For the discussion convenience, we included these results from Table 2 below: BaFT achieved locality (Column Loc.) $\geq$ 0.92 in all cases, while ReFT's locality can fall below 0.7.

The locality of BaFT is slightly lower than WISE, but as discussed in Sec 3.3 (Parameter Efficiency), WISE takes 10-20 times more parameters than BaFT, and had a much lower parameter efficiency (As an evidence, when WISE parameters number is reduced from "full" to "light", its performance degrades drastically). We added this discussion, together with potential directions to further improve BaFT by adding more parameters, in the revised draft.

Contradicted Statements.

We apologize for causing the confusion.

In the two statements, both "editing performance" in the first statement and "editing effectiveness" and in the second should refer to "editing success" (reliability, Rel.), they do not take "locality" into account.

Here we would like to provide a few more clarifications. The first statement, "BaFT and ReFT used a subspace of the same rank, so its editing performance should be upper bounded by ReFT in this case...", highlighted that BaFT and ReFT are assigned with a same-ranked subspace to edit representations. Conceptually, ReFT only needs to fit new knowledge, but BaFT takes locality into consideration as well. This additional locality consideration is a regularizer that restricts BaFT's behavior. In contrast, ReFT can learn the subspace freely. Therefore, at a colloquial level, we called unrestricted ReFT an "upper bound" of BaFT in terms of the editing success (reliability, Rel.).

The second statement, "..it achieved better locality and portability than ReFT with no degradation of the editing effectiveness", conveys that, when we restrict BaFT's behavior on a fine-grained basis-level (through the locality regularizer), we can actually achieve better "locality" (and "portability") with minimally hurting its editing success (reliability, Rel.).

We thank the reviewer again for pointing out these confusing statements. In response, we have revised the draft to make these statements clearer.

I also checked EasyEdit GitHub, while their announcement is

2024-11-19, we update the Table 4 results in the paper "A Comprehensive Study of Knowledge Editing for Large Language Models" after optimizing certain methods (related to AdaLoRA) and fixing computational bugs (related to ROME and MEMIT) in the EasyEdit (More details in #427). **These improvements have led to better results than before. ** We will continue updating this paper and welcome everyone to discuss and exchange ideas.

I don't understand why these baseline results are lower than the first-release results in the paper in terms of locality.

However, the primary reason for this score is not as previously stated. The main concern is that the performance of BaFT seems to be considerably constrained when compared to other benchmarks like ReFT, especially in terms of locality. While I acknowledge that the performance gap may have widened slightly after the results were updated, I'm puzzled as to why issues in EasyEdit only impact the baselines and not your method in terms of locality. Additionally, in the initial release and current version, the results did not effectively support the rationale for higher locality.

(Main Concern) The performance of BaFT seems to be considerably constrained when compared to other benchmarks like ReFT, especially in terms of locality.

As reported in our previous response, BaFT is capable of achieving far more better locality than ReFT (Table 2). In Table 1, BaFT achieved slight but consistently higher locality than ReFT. Notably, in Table 1, due to the lack of off-the-shelf locality data for regularization, BaFT does not incorporate Locality Regularization. In Sec 3.4 we conducted a careful ablation study, showing that both basis-level weighting and locality regularization are critical for improving locality, and the basis-level weighting is capable of alleviate the editing-locality trade-off due to its fine-grained nature.

Why these baseline results are lower than the first-release results in the paper in terms of locality. Why issues in EasyEdit only impact the baselines and not your method in terms of locality.

According to the revised results, the performance of all methods, including baselines and ours, are "improved". On LLaMA-2, baseline methods showed "lower" locality compared to the previous version. However, their editing success and portability significantly increased, resulting in an increased average score. Similarly, on LLaMA-3 baseline methods had "higher" locality and average score.

SERAC and MEND methods were not reported in this paper.

As mentioned in Sec 3.1, we choose baselines that "do not require a larges-scale hard-to-access training data, or training additional models". Since these assumptions of access to such a large-scale data and substantial computational resource are far more restricted than the assumptions made in other methods, making a direct comparison unfair. Therefore, we didn't include SERAC, which requires additional training an in-scope classifier and a separate LM serving as the editing model, and MEND, which requires additional training a HyperNet on large dataset.

Fluency is not reported.

The primary focus of this work is on improving editing-locality trade-off especially in challenging scenarios such as where frequent updates are desired. As a result, we follow recent works on this direction (to just name a few, GRACE, WISE, MELO) and focus on editing related metrics.

We highly appreciate your effort and time spent reviewing our paper and thank you for your expertise and constructive comments. In the following, we address your comments and questions one by one, which will also be updated in the revised paper.

This paper lacks visual illustrations such as diagrams or figures.

Following the reviewer's suggestion, we added an illustration figure of our BaFT in the updated draft in Appendix A. We will reorganize the material to put the figure in the main body in the future revised version.

There is an absence of publicly shared code in the paper or on platforms such as OpenReview.

As advised by the reviewer, we share the implementations of our method on the Anonymous link [1]. Our codes are mainly based on EasyEdit [2] and Official implementation of ReFT [3].

[1] https://anonymous.4open.science/r/baft-E782/

[2] https://github.com/zjunlp/EasyEdit

[3] https://github.com/stanfordnlp/pyreft

While the motivation behind the study is well-articulated, the methods deployed are somewhat conventional.

We are glad that the inherent limitation of linear representation fine-tuning, which motivates our BaFT, is admired by the reviewer. Our formal analysis on how linearity in representation fine-tuning can limit its performance is a main contribution of this work. This analysis sheds light on other fundamental tasks that require selective updates, such as in machine unlearning. For the proposed BaFT, its merit lies in the flexibility to admit a variety of improving directions. As we did in Sec 3.4, having some bases share weights, BaFT can be seen as learning a mixture of (smaller) subspaces. If sparsity constraints are further imposed, BaFT can naturally benefit from recent advance in sparse mixture of experts, where each subspace becomes an "expert". BaFT may also help understand how LLMs see knowledge by scrutinizing what bases (subspaces) combinations are chosen for different knowledge. We plan to explore these directions in our future work.

Dear reviewer v1Lr,

We are glad to know that our responses addressed your concerns, and are grateful for your positive rating! Please feel free to share with us any other suggestions if you find helpful for further improving the quality of our paper.

Best regards,

Authors