Dear Reviewer kk6q:

Thank you for your positive feedback on our work!

We are happy to discuss the details of our work and hope to address all your questions.

---

For Other Strengths And Weaknesses:

Equation (9) requires more clarification. A detailed explanation would help readers better understand this notation.

Thank you for your correction! The terms $\mathcal{L} _ {e _ i, \mathcal{W} _ {t-1}}$ and $\mathcal{L} _ {loc _ {i}, \mathcal{W} _ {t-1}}$ in the equation represent the loss functions corresponding to the editing knowledge at time step $i$, computed on the model parameters $\mathcal{W} _ {t-1}$ at time step $t-1$.

We agree that Equation (9) indeed needs clearer explanation, and we will add detailed clarification in the revised version to enhance readability.

Minor: Some typos. (e.g., Line 97, right column: "updated" should be "updates"; Line 159, left column: "he" should be "The".)

Thank you for identifying these typos! We will correct these issues in the revised version, including changing "updated" to "updates" in the right column of line 97 and "he" to "The" in the left column of line 159.

For Questions For Authors:

Figure 5 only examines the LLM's general capacity after 3,000 edits across different methods. Given that you've extended lifelong editing to over 10,000 edits, I believe the general capacity evaluation should be expanded to cover more editing steps. Have you conducted related experiments?

We fully agree with your suggestion. To verify that RLEdit can maintain LLM's general capabilities after large-scale continuous editing, it is necessary to evaluate the model's performance after completing 10,000 edits.

Following your suggestion, we have supplemented our experiments to assess the impact of large-scale model editing on the model's general capabilities. Consistent with Section 4.5 of our paper, we used a 200$\times$100 configuration (i.e., 200 batches with 100 edits per batch) to edit Llama-3-8B-Instruct model on the ZsRE dataset. We conducted GLUE tests after every 2,000 edits, recording F1 Scores for SST, MMLU, MRPC, CoLA, RTE, and NLI metrics. The experimental results are as follows:

From the experimental data, we can observe that as the number of knowledge samples continues to increase, our RLEdit method consistently maintains stable general capabilities of the LLM. Notably, even after 20,000 knowledge edits, the model maintains general capability metrics comparable to its initial state. This strongly demonstrates RLEdit's exceptional stability in large-scale continuous knowledge editing scenarios.

I'm curious about how the choice of LLM layers affects model editing performance. Have you explored varying the selection and number of edited LLM layers and investigated their impact on the results?

In our experiments, we found that the choice of editing layers significantly impacts the editing effectiveness. Specifically, when selecting early layers for RLEdit, the final metrics might decrease by 5%-10% compared to selecting middle or later layers; certain layer choices might result in very high Efficacy but very low Generalization and Specificity.

Therefore, the selection of LLM layers is a hyperparameter that requires careful tuning, which also demonstrates the complexity of LLM knowledge storage mechanisms.

Currently, several studies are focusing on investigating the impact of layer selection on editing effectiveness. We will include references to these works and add discussion of this issue in our subsequent revision.

---

Thank you again for your positive feedback and valuable suggestions on our work! If you have any other questions, we would be happy to continue the discussion.

Best regards

Authors

Dear reviewer TQMf:

Thank you very much for your recognition and support of our work!

We are very willing to engage in in-depth discussions with you regarding the research details.

---

For Experimental Designs Or Analyses:

Thank you for your valuable suggestions!

Following your suggestion, we have conducted additional experiments. Due to the word limit constraints of the rebuttal, detailed experimental results can be found in our response to Reviewer kk6q.

The results shows that even after completing 20k knowledge edits, the model maintained stable GLUE scores. We will include these experimental data in the revised paper to ensure evaluation completeness.

For Essential References Not Discussed:

We strongly agree with your suggestion.

In the revised version, we will add citations to key literature on RL theoretical foundations and supplement discussions of recent research developments in lifelong editing methods.

For Weakness 1:

Thank you very much for your suggestions!

We understand the importance of comparing RLEdit with parameter-preserving editing methods like GRACE and SERAC, although they operate on different mechanisms from RLEdit.

Following your suggestion, we have conducted comparative experiments between RLEdit and GRACE, SERAC on 20$\times$100 tasks. The experiments were performed on Llama-3-8B-Instruct using both ZsRE and CounterFact datasets. The results are shown in the following table:

We observe that RLEdit shows comparable performance to parameter-preserving editing methods, demonstrating the effectiveness of the RLEdit approach.

For Weakness 2:

Thank you again for this key suggestion!

As stated above, we have added relevant experiments. Detailed results have been presented in our response to Reviewer kk6q.

For Other Comments Or Suggestions:

Thank you very much for your careful review of the paper!

We will carefully revise these errors in the revised manuscript to ensure that the final submitted paper meets high quality standards.

For Question 1:

Thank you for highlighting this important observation. We have conducted an in-depth analysis of this phenomenon and identified several main reasons:

1. This high-density, long-sequence editing task significantly amplifies the cumulative effect of editing errors, potentially triggering catastrophic forgetting or even severely damaging the LLM's basic functionalities.
2. Although our baseline comparison methods perform excellently in single or short-sequence editing scenarios, most were not specifically optimized for long sequence continuous editing scenarios.

This finding was indeed one of the key motivations that drove us to develop RLEdit, aiming to achieve more precise and robust long-sequence lifelong editing capabilities.

For Question 2:

Thank you for raising this profound question.

Our analysis suggests this phenomenon is primarily related to inherent limitations in hypernetwork training mechanisms.

Parameter changes and potential errors introduced by the previous batch of edits continue to accumulate and propagate to subsequent edits, and the model instability caused by this error accumulation effect significantly weakens the overall editing performance of these methods.

While complete retraining might theoretically help, as you point out, this indeed poses serious computational efficiency challenges, especially in large-scale lifelong editing scenarios.

For Question 3:

Thank you for your attention to the results in Figure 8. We indeed systematically explored the performance of various baseline methods under different batch configurations.

Our research found that, with a fixed total number of editing samples, the performance of most baseline methods is highly correlated with specific editing configurations. Particularly, when reducing batch size while increasing batch number, these methods generally show significant performance degradation.

We believe RLEdit's stability stems from its sequence-level perspective. RLEdit hypernetwork training focuses on the entire knowledge sequence rather than individual knowledge samples, enabling the hypernetwork to handle diverse editing configurations.

---

Thank you again for your detailed and thorough feedback! If you have any other questions, we would be happy to continue the discussion.

Best regards

Authors

Dear Reviewer 2E3w:

Thank you for your positive feedback on our work!

In brief, we propose a hypernetwork training method for lifelong editing tasks that can adaptively adjust LLM parameters based on knowledge update sequences, achieving efficient and low-interference lifelong editing.

We appreciate your recognition of our paper writing and language organization, and we are happy to further simplify the technical descriptions in the final version to make them more accessible.

---

For Relation To Broader Scientific Literature:

I think Retrieval-Augmented (RAG) is a popular way and drawing the most attention for updating LLM's knowledge.

Thank you for your insights.

Indeed, Retrieval-Augmented Generation (RAG) is a popular technique that effectively enhances LLM's generation capabilities by retrieving relevant information from external vector databases as contextual input. The main value of RAG lies in expanding the model's knowledge scope and improving domain-specific performance, and it can also be used for knowledge editing, thanks to LLM's powerful contextual learning abilities.

However, we believe RAG has several limitations in the field of knowledge editing:

1. Effectiveness Challenges: When correcting outdated or incorrect knowledge stored in LLM parameters, RAG methods inevitably lead to knowledge conflicts. Specifically, contradictions between retrieved new knowledge and model's built-in knowledge significantly reduce editing effectiveness, particularly evident in factual update tasks.
2. Insufficient Persistence: The core objective of model editing is to permanently modify LLM's internal knowledge representations. Unlike methods that modify model parameters, RAG can be characterized as a prompt tuning technique, where knowledge updates are only effective in specific contexts, making it difficult to achieve persistent model editing.

In Model Editing research, we categorize methods like RAG that add auxiliary modules without changing base model parameters as parameter-preserving editing methods (as stated in Appendix B, Line 836). If you're interested in such methods, we recommend referring to the related research cited in our Related Work section.

For Other Comments Or Suggestions:

typo: Section 3.1.1 "he training process" should be "the"

Thank you for your thorough review! The typo you identified in Section 3.1.1, where "he training process" should indeed be corrected to "the training process", is well noted.

We greatly appreciate your attention to this detail and will immediately correct this error in our next revision.

For Questions For Authors:

I wonder compared to RAG, what is the pros and cons using the editing method.

As mentioned earlier, RAG represents a parameter-preserving knowledge editing method that assists LLM generation through external knowledge retrieval rather than directly modifying the model's internal knowledge representations.

Parameter-preserving methods (e.g., RAG, SERAC, GRACE) and parameter-modifying methods (e.g., MEND, MEMIT, our proposed RLEdit) each have their advantages and limitations for different application scenarios.

For RAG:

• Advantages: RAG primarily targets knowledge-enhanced generation tasks, effectively supplementing information beyond LLM's knowledge base to produce more comprehensive and accurate outputs. It offers easy deployment, high computational efficiency, and enables dynamic knowledge updates.

• Limitations: When target editing knowledge conflicts with existing knowledge stored in model parameters, RAG struggles to fundamentally alter LLM's internal cognition, potentially leading to inconsistent outputs or performance degradation. Additionally, RAG depends on external knowledge base quality and retrieval precision, increasing system complexity.

For Editing Methods:

• Advantages: These methods (like our RLEdit) fundamentally reshape model knowledge representations through direct parameter modification. Such modifications offer persistence, scalability, and end-to-end characteristics. The modified model can reflect updated knowledge without requiring additional modules, making them particularly suitable for knowledge correction and update tasks.
• Limitations: Parameter-modifying methods typically demand higher computational resources, especially for large-scale models. Furthermore, multiple consecutive edits may lead to knowledge interference and error accumulation, which is precisely the core problem our RLEdit attempts to mitigates.

---

Thank you again for your feedback! If you have any further questions or suggestions regarding our methodology, experimental design, or potential application scenarios, we very much welcome continued in-depth discussion.

Best regards

Authors