We thank reviewer for detailed and constructive questions.

Typos in the paper

We apologize for this oversight. Crucially, all our experiments and comparisons used the knowledge-editing version of AnyEdit; this was a citation typo and does not affect any results or conclusions. We will fix this typo and conduct a thorough proofread of the manuscript to resolve any remaining formatting or writing issues.

Inconsistency between the reproduced numbers and the reported numbers in AnyEdit

Our reproduced numbers are produced by the latest code from https://github.com/jianghoucheng/AnyEdit with default parameters. We hypothesize the difference might come from environment setups, as we found the provided environment configuration is incomplete in the repo and caused non-trivial errors. We had to upgrade PyTorch and Huggingface's transformers library according to a GitHub Issue. We will include a full list of packages required for reproduction.

Statistical reliability

We run three trials for each experiment since we did not observe significant variance, as we report the standard deviations below.

UnKEBench:

CounterFact:

MQuAKE:

Datasets and number of edits.

We followed AnyEdit and ran the full datasets for each benchmark: UnKEBench (1,000 samples), AKEW-CounterFact (975 samples), AKEW-MQuAKE (354 samples). Below are the batch editing results with different numbers of edits.

UnKEBench:

CounterFact:

MQuAKE:

We appreciate the reviewer’s thoughtful comments and address them below.

Potential reason for locality performance drop on certain metrics

We think this issue might be related to the granularity of localization. Note that AnyEdit and μΚΕ inherit pre-defined editing layer selection from MEMIT and UnKE, where we consider all working memory shifts happen on the same layer(s). Intuitively, an unstructured editing target, which can be long, can encompass knowledge of different levels, which likely requires more fine-grained designs in localization and corresponding updates. Due to the complexity of the designs, we leave this to future work.

Computational overhead

We see a ~4.2% increase in time elapsed running with cache and ~33.8% without cache, which includes IO costs and intermediate inference, among others. We observe a negligible peak VRAM usage increase. Since efficiency is not the primary focus of this work, we leave this to future work.

Limited baseline comparison, lacking lifelong editing frameworks.

We perform additional experiments with a recent lifelong editing baseline, WISE [1], and adopt code and hyperparameters from EasyEdit [2]. We use Llama and Qwen as shortcuts to Llama3-8B-Instruct and Qwen2.5-7B-Instruct. We observe incomparable and unstable performance of WISE compared to AnyEdit and μΚΕ under the default settings.

WISE on UnKEBench:

On CounterFact:

On MQuAKE:

On EditEverything:

Technical details could be consolidated or moved to the appendix to improve narrative flow and focus.

Thank you for your suggestion; we will simplify this part and move the details to the Appendix.

[1] Wang, Peng , et al. “WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models.” NeurIPS 2024

[2] https://github.com/zjunlp/EasyEdit

We sincerely thank the reviewer for the constructive review.

Clarification on unstructured editing dataset and evaluation

We would like to first clarify about the datasets and evaluation setups we adopted for unstructured editing.

• We follow the evaluation setups in existing unstructured editing studies [1, 2]. We evaluated our approach on most existing benchmarks from previous work, including diverse types of knowledge editing (counterfactual, multi-hop) and diverse formats of knowledge (news, chemistry, etc.).
• The poem case study is just an illustration of the capability of our improved method in achieving higher success and more accurate edit on the lexical level. We agree with the reviewer that certain benchmarks might not be the most appropriate for evaluating unstructured editing. We will follow the reviewer's advice to add a discussion in the paper for the proper selection of benchmarks.

In the rebuttal period, we conduct another experiment on the hallucination reduction benchmark from SelfCheckGPT [3], which is a more realistic scenario. As shown in the following table, μΚΕ consistently outperforms AnyEdit and achieves a reasonably high score for hallucination reduction.

The definition of “working memory”

The “working memory” here indicate the specific hidden states corresponding to the located layers that are hypothesized to contain the knowledge to be edited and can be updated with a bias term using gradient descent. Conceptually, this can be connected to definition in human working memory in psychology [4] and cognitive neuroscience [5, 6]. In short, both in human cognition and in the μΚΕ approach, “working memory” acts as a short-lived, capacity-limited store that is actively manipulated (through attention or gradient steps) to change future outcomes (behavioral response or token probabilities). The cognitive-science emphasis on neural circuits in prefrontal/parietal areas echoes the μΚΕ reliance on a specific transformer layer ($l_\text{WM}$) whose activations effectively serve the same functional purpose: keeping relevant information “online” so that it can be selectively updated and then used to guide subsequent processing.

[1] Deng, Jingcheng, et al. "Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models." ICLR 2025

[2] Jiang, Houcheng, et al. "AnyEdit: Edit Any Knowledge Encoded in Language Models." ICML 2025

[3] Manakul, Potsawee, Adian Liusie, and Mark Gales. "SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models." EMNLP 2023

[4] Hitch, Graham James, and A. D. Baddeley. "Verbal reasoning and working memory." The Quarterly Journal of Experimental Psychology 28.4 (1976): 603-621.

[5] Finn, Emily S., et al. "Layer-dependent activity in human prefrontal cortex during working memory." Nature neuroscience 22.10 (2019): 1687-1695.

[6] Degutis, Jonas Karolis, et al. "Dynamic layer-specific processing in the prefrontal cortex during working memory." Communications biology 7.1 (2024): 1140.

Thank you for the supportive review!

More Ablation Studies on the selection of coefficients in Eq.(12)

We conduct ablation studies on the selection of coefficients during the rebuttal period. Due to the time constraint, we ablate on the vanilla matryoshka (with all coefficients set to 1), and coefficients computed by figure affinity with different steps of optimization. We show μΚΕ results based on Qwen, 100 samples in the following table. Results show that vanilla matryoshka and that of adaptive coefficients are effective in edit success, yet adaptive coefficients help in improving generality of the edit (para. results), which is consistent with our previous finding in Figure 6 in the original paper.

---