PropMEND: Hypernetworks for Knowledge Propagation in LLMs

Despite the paper's title emphasis on "knowledge propagation," both the proposed methodology and evaluation dataset exclusively address multi-hop question answering which only is a kind of knowledge propagation. This renders the broad title claim potentially misleading.

We believe that propagating knowledge along knowledge base relations is an important propagation type, frequently studied in prior work [1]. Techniques developed for this task could generalize to other types of propagation, such as logical implications of a fact using world knowledge (“Barack Obama was president of the United States” -> “Barack Obama was born in the United States”), as knowledge base relations are quite diverse.

[1] Evaluating the Ripple Effects of Knowledge Editing in Language Models; TACL 2024, Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, Mor Geva

---

The baseline comparison remains limited, as the paper evaluates only MEND and MEMIT methods. Other important knowledge editing techniques (e.g., SERAC、IKE...) are notably absent from the experimental analysis.

Thanks for suggesting relevant work! Our work focus on methods that create parametric (delta weight) to the model weight. This is a more challenging editing scenario, but makes using the updated model a lot simpler. The suggested baselines do not have parameter updates. IKE uses a few-shot prompt to bias model to provide updated answers given the new fact and the question. SERAC maintains a memory and bias model’s behavior at test time, but does not modify the base model parameters. We will make the distinction clear in the paper.

---

Propmend primarily modifies only two loss functions (inner-loop and outer-loop) and the data input relative to MEND and is a simple extension to mend, absent of innovation.

Our contribution is that we spot a mismatch between the problem formulation and the objective that all knowledge editing methods optimize against. We consider it a pretty significant change to be able to edit based on a general sequence of tokens rather than QA pairs. We don’t think it’s obvious that the modifications we’ve made to this algorithm would work or that meta-learning can succeed on this task, compared to MEND’s task where the edit and test conditions are more closely aligned.

---

Can the authors provide experimental results for one or more knowledge editing methods on the StoryPropagation dataset?

In our Table 2 (main paper), we provide results for existing knowledge editing methods — CPT, MEMIT and MEND.

---

Will modifying the editing loss improve the other knowledge editing method 's effectiveness on the StoryPropagation dataset?

Possibly, but it’s not obvious to us how to make simple changes to other methods and achieve the same effect. Methods like MEMIT have a strict formulation of how its optimization problem is specialized for memorization-based knowledge editing, and adapting propagation questions to it is non-trivial.

---

Will the test results of LLM acc be higher than those of EM?

Yes. LLM-Acc will always be higher than EM-Acc since the LLM-Acc offers partial credit.

Due to space limitations, we focus on addressing the most important concerns. If the reviewer still has questions regarding other points in "Questions" section, we are more than happy to address them!

My major concern with the paper is in its little or incremental technical contribution ... is there any theoretical justification for it?

We agree that we make relatively small modifications to the existing MEND architecture, which is why we named our approach “PropMEND”! But our experiments show these modifications bring meaningful differences in performance. Furthermore, it allows the framework to be used in more general settings, where facts cannot be easily reformulated into QA formats. We also introduce a scenario where the target task (the outer loop of MEND) differs substantially from the training task (the inner loop of MEND), making the meta-learning problem harder, and shows the method can work well in this scenario as well, which was not tested previously. Our contribution is a new problem framing, evaluation framework (which allows OOD evaluation and targeted knowledge propagation), as well as an effective solution that adapts an existing method.

---

My next concern is with experiments -- Why are the experiments conducted with the base model (Llama-3.2-1B-base) and not the corresponding instruct version (Llama-3.2-1B-instruct)? Will this not circumvent the need for training with TriviaQA?

It is easier to inject knowledge to base models when the learning rate is still high. Prior work also uses a base model to study the learning of factual knowledge [1,2,3]. Anecdotally, it is difficult to inject knowledge into a model (via CPT or other means) after a long instruction training phase; this only works well if you do a longer training run with warmup and cool-down stages. We include evidence below. We include results when conducting a Continued Pretraining (CPT) experiment starting from Llama-3.2-1B-Instruct model (same experiment setup as Table 2)

To share a bit of insight about why Llama-3.2-1B-Instruct model performs so badly, we share a type of common output from the model below:

I couldn't find any information on an individual named Eric Ward being recruited as director at a specific organization

We follow prior work [1,2,3] in this domain, which choose base models to edit, as they can provide a cleaner testbed for studying knowledge injection without conflating it with how a model is aligned.

[1] How Do Large Language Models Acquire Factual Knowledge During Pretraining?; NeurIPS 2024, Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, Minjoon Seo

[2] Memorization without overfitting: Analyzing the training dynamics of large language models; NeurIPS 2022 Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan

[3] Dissecting recall of factual associations in auto-regressive language models; EMNLP 2023, Mor Geva, Jasmijn Bastings, Katja Filippova, Amir Globerson

---

Stronger baselines ....

Thanks for suggesting a relevant work!

Self-tuning isn’t quite straightforward to work with in our present data condition. The authors of Self-tuning haven’t released scripts they use to create the memorization/comprehension/self-reflection examples, so there isn’t a plug-and-play experiment that’s doable with this method. We were not able to reimplement their method during the rebuttal period. We conducted an experiment in a similar vein to Self-tuning: we conduct SFT training on propagation questions in a meta-training set, and run CPT on each injected fact. As shown in the first row in the table below, the additional SFT only mildly improves efficacy performance but damages specificity performance. (See "CPT (after SFT on propagation question)" in Table below)

We also note that the suggested Self-Tuning method only improves marginally over the CPT performance in their work. On StoryPropagation, CPT baseline provides a relatively small scale of improvement, from 8.3% to 18.1%. If we extrapolate a similar performance improvement over CPT, the performance of Self-Tuning will still be far below that of PropMEND.

---

Finally, what happens when you have to ingest multiple stories ....

We experimented with doing 5 and 10 edits at the same time, and training our hypernetwork for these settings. The performance degrades compared with the single-edit scenario, but we still outperform our baselines even when those baselines use single-edits (Table 2). To accommodate more edits, we believe a larger hypernetwork or different learning rate would be helpful, which is doable but requires some additional engineering and efficiency improvements beyond the scope of the current work.

---

Line 206-207: “Many questions involve tail entities,... I.e. do SFT with QA that represents the world knowledge (e.g. Fig. 6(a))

Great question. There is a distinction between a fact that the model has been exposed to at pre-training time [3], typically in various forms, and one that it hasn’t. Fine-tuning on a fact the model has seen before “resurfaces” that fact in a way that’s different from fine-tuning at instruction time. [1] shows that finetuning pretrained model on unfamiliar knowledge (based on the pretrained model) will encourage hallucination. Related observations are noted in [2]. RippleEdit often features much more obscure knowledge and this approach is expected to be less effective there. We will clarify this point in any future version.

[1] Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? EMNLP 2024, Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, Jonathan Herzig

[2] Understanding Finetuning for Factual Knowledge Extraction; ICML 2024, Gaurav Ghosal, Tatsunori Hashimoto, Aditi Raghunathan

[3] How Do Large Language Models Acquire Factual Knowledge During Pretraining?; NeurIPS 2024, Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, Minjoon Seo

---

Line 237: Why not use existing QA datasets like NQ or musique (for testing multihop reasoning) to test OOD generalisation?

We initially conducted experiments with MuSiQue. However, the dataset does not test the same kind of propagation as StoryPropagation, where part of the propagation relationship is hidden. Suppose we have an instance from MuSiQue with two paragraphs A and B containing relevant facts. Injecting both A and B leads to issues of memorization, since the answer tokens are present in either A or B; a baseline of repeating likely tokens from these paragraphs does very well. If we inject just one of these to avoid this, it’s unclear whether the model knows the content of the other paragraph. StoryPropagation explicitly avoids these reasoning shortcuts while testing the same kinds of effects.

This raises the concern that the method may not flexibly generalize propagation to new relations/entities, limiting real-world applicability, and undermining the tagline of “propagation” as a generally solved problem

Our approach improves over past work both in methodology and evaluation, yet do not frame it as solving propagation. Substantial past work on knowledge editing [1,2,3,4] studies datasets which are primarily about verbatim memorization, where the notions of generalization we study don’t apply.

[1] AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models; ICLR 2024. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, Tat-seng Chua

[2] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors; Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, Marzyeh Ghassemi, NeurIPS 2023

[3] Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks; Ting-Yun Chang, Jesse Thomason, Robin Jia, NAACL 2024

[4] InstructEdit: Instruction-based Knowledge Editing for Large Language Models; Ningyu Zhang, Bozhong Tian, Siyuan Cheng, Xiaozhuan Liang, Yi Hu, Kouying Xue, Yanjie Gou, Xi Chen, Huajun Chen, IJCAI 2024

---

The performance drops in OOD (Both) settings are substantial (), especially relative to the in-domain result (64%). Could the authors clarify what factors contribute most to this degradation? Is it primarily limited data exposure, inductive bias of the hypernetwork, or something else? Would curriculum training or meta-augmentation be promising future directions?

We agree understanding why degradation happens in OOD scenarios is important. We also thought that limited data or model size of the hypernetwork can be a bottleneck. Thus, we experimented with scaling the training data and the parameter size for hypernetwork, see the table containing new results below. While this helped the performance in all evaluation settings, OOD performance was still much lower than in-domain performance. We think curriculum training, new hypernetwork architecture, or changing the training data are all valid directions for future research, but do not have strong evidence supporting one solution over the other.

Data: Synthetically created training data from StoryPropagation knowledge triplets

Model: Llama-3.2-1B-base-QA-FMT

---

The focus remains on single-edit scenarios. However, real-world LLM deployment often requires continual, multi-point knowledge updating. The method’s suitability for such cases is not empirically or theoretically addressed (Limitations section).

As the method and training focus on single-edits, could the authors provide either preliminary empirical results or a more detailed theoretical discussion about how the method might extend to multiple, possibly conflicting edits in sequence? Are there clear bottlenecks that would require new hypernetwork designs?

We experimented with doing 5 and 10 edits at the same time, and training our hypernetwork for these settings. The performance degrades compared with the single-edit scenario, but we still outperform our baselines even when those baselines use single-edits (Table 2). To accommodate more edits, we believe a larger hypernetwork or different learning rate would be helpful, which is doable but requires some additional engineering and efficiency improvements beyond the scope of the current work. We think continual edits are an exciting but orthogonal line of work from ours.

We do notice that scaling to a larger number of edits will cause the runtime to increase. We also think doubling parameters of the hypernetwork is not the most elegant way to alleviate performance degradation. We believe future work focus could study the mechanism of knowledge propagation and leverage the insights to design a more efficient method.

In StoryPropagation, since each fact/story is surrounding a fake entity (e.g., Adam Jacobson) and efficacy question is specific to each fake entity, so we don’t expect knowledge conflict comes into play. However, we do expect multiple edits will introduce conflict in propagation. We believe our work lays the foundation for future exploration.

Compared to more recent developments in knowledge editing that address diverse evaluation criteria and architectural innovations, the technical advancement is somewhat incremental.

Our contribution is that we spot a mismatch between the problem formulation and the objective that existing knowledge editing methods optimize against. We don’t think it’s obvious that the modifications we’ve made to the MEND algorithm would work or that meta-learning can succeed on this task, compared to MEND’s task where the edit and test conditions are more closely aligned. For our study, we follow the conventional evaluation criteria from studies on knowledge propagation [1,2,3].

[1] Propagating Knowledge Updates to LMs Through Distillation; NeurIPS 2023, Shankar Padmanabhan, Yasumasa Onoe, Michael J.Q. Zhang, Greg Durrett, Eunsol Choi

[2] Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability; ACL-Findings 2024, Afra Feyza Akyürek, Ekin Akyürek, Leshem Choshen, Derry Wijaya, Jacob Andreas

[3] CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners. Arxiv 2025. Yunzhi Yao, Jizhan Fang, Jia-Chen Gu, Ningyu Zhang, Shumin Deng, Huajun Chen, Nanyun Peng

---

The evaluation is restricted to relatively small models (1B-3B parameters) and single-edit scenarios

Knowledge editing evaluation is more costly than many other LLM research topics, as each knowledge edit instantiates a new model. This experimental setting (i.e., using 1-3B parameter models) is used in substantial amounts of work over the past year [1,2,3]. We aren’t aware of any evidence that editing methods effective at the 1B-3B scale wouldn’t work at a larger scale as well. Additional work on GPU memory management is needed for this scaling (which holds for any hypernetwork-based method); this is technically possible and is an ingredient of our ongoing work but is orthogonal to the key technical claims of this work.

[1] AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models; ICLR 2024. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, Tat-seng Chua

[2] Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors; Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, Marzyeh Ghassemi, NeurIPS 2023

[3] Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks; Ting-Yun Chang, Jesse Thomason, Robin Jia, NAACL 2024

[4] InstructEdit: Instruction-based Knowledge Editing for Large Language Models; Ningyu Zhang, Bozhong Tian, Siyuan Cheng, Xiaozhuan Liang, Yi Hu, Kouying Xue, Yanjie Gou, Xi Chen, Huajun Chen, IJCAI 2024

---

The paper lacks detailed analysis of when and why the method fails, particularly in challenging scenarios. Understanding failure patterns across different types of knowledge and reasoning requirements would strengthen the contribution.

We agree providing analysis into failure patterns would be important!

Our dataset is designed with a control of well-known entities and relations, and allows us to study Out-of-Domain generalization by composing propagation questions with OOD entities or OOD relations (or both). As we noted in the paper (Line 257-259), we observe that OOD (Entity) performance tends to be higher than OOD (Relation). This result helps characterize the effectiveness of the hypernetwork at different challenging cases where it may have to edit different parts of the model than it’s seen during meta-training time. Designing evaluation along other axes (e.g., categories of propagation relation, like the reviewer suggested) can be great future work. We will add discussion on different analyses.

---

What are some simple and obvious ways to make the hypernetwork more parameter-efficient?

The hypernetwork (for a weight matrix) is a 2-layer MLP which consists of a learned down-projection and a learned up-projection. It’s possible to reduce the size of this MLP further, or use low-rank parameterizations of the matrices, but we believe this would introduce a size-accuracy tradeoff.

---

How does the method's performance degrade as the number of sequential edits increases? Are there interactions between multiple edits that affect propagation?

Thanks for the suggestion! Below we present new results of doing 5 and 10 edits at the same time, and training our hypernetwork for these settings. We list two baselines (CPT (Full) and MEMIT, the strongest baseline method from our work); the table is comparable to the results of Table 2 in the paper. The performance degrades compared with the single-edit scenario, but we still outperform our baselines even when those baselines use single-edits (Table 2). To accommodate more edits, a larger hypernetwork or different learning rate can be helpful, which is doable but requires engineering and efficiency improvements beyond the scope of the current work.

We don’t expect explicit conflicts of injected knowledge in the setting we consider, but we do expect multiple edits might require a higher-capacity hypernetwork to perform all of the necessary propagation. We believe our work lays the foundation for future exploration. We will add discussion and report results on multiedit scenarios in the revised paper.

---

Can you provide more analysis on why editing lower layers (mid-upper rather than top layers) is more effective for knowledge propagation?

According to papers in the interpretability literature [1,2], mid-upper layers are where semantic processing happens in LLMs. The very top layers are more responsible for the mechanics of next token prediction. [3] use mechanistic interpretability techniques to show middle and upper layers are important for multi-hop queries. These studies motivated us to explore editing different layers than the original MEND paper.

[1] Studying Large Language Model Generalization with Influence Functions; arxiv, 2023, Anthropic team

[2] Linguistic Knowledge and Transferability of Contextual Representations; NAACL 2019, Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith

[3] Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries. EMNLP 2024 Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, Amir Globerson

---

While the synthetic datasets are useful, the paper would benefit from evaluation on more realistic knowledge editing scenarios, perhaps using naturally occurring facts and propagation questions from knowledge bases -- picking a open-source model released few days ago and editing it with the facts post-training date might be an interesting experiment to try.

We agree! However, it is unfortunately difficult to devise a controlled experiment of this form. Facts added to knowledge bases may be present in text prior to their inclusion in the KBs; so without exhaustively searching the LLM pre-training data, it’s difficult to tell whether the model has actually been pre-trained on this fact. We opted for a more controlled setting (especially with respect to out-of-domain-ness of entities and relations) and reproducibility for future work.

Hi reviewer Cw9L,

Can you respond to author's rebuttal? It is mandatory, and reviewer-author discussion is closing out in 24 hours. Thanks.

AC