Taming Knowledge Conflicts in Language Models

Claim and Evidence 1: The Effect of Scaling JuNE

Thank you for the great suggestion. JuNE is an ablated variant of JuICE that only skips the second run but still incorporates and tunes its own scaling factor, as noted in Line 257 and Appendix E. We acknowledge that Proposition 5.5 may cause confusion, as JuNE is grouped with PH3 to emphasize its single-run nature, not the absence of scaling. We will clarify this distinction in the revised version of the paper.

Claim and Evidence 2: Further Analysis on Incorporating Activations from the First Run

This is indeed a great point! Proposition 5.5 is a proof-of-concept showing how dual-run methods improve intervention under simplified assumptions. It remains valid when using scaling factors rather than mere removal. In a high-level proof sketch, as the second layer transitions into a "factual memorizer" (due to the role alternation via scaling), single-pass methods continue to treat it as a context component, thereby suppressing its influence—whereas JuICE correctly preserves it, leading to more effective control.

Beyond the theoretical discussion in Section 5, our paper provides extensive empirical support for this dual-run strategy. In Section 3, we introduce the observations that motivate the approach; we then validate its effectiveness in Tables 3 and 4 by comparing JuICE and JuNE, and we conduct a more detailed analysis in Figure 4. Collectively, these experiments and analyses offer strong evidence of the proposed method’s efficacy.

Following your suggestion, we have also performed an ablation study examining the benefits of adding the activations obtained from the first inference to steer (without scaling) using Gemma on all parametric datasets. The details are provided in this anonymous link (see xf53_Claim_and_Evidence_2). Simply adding these activations already confers considerable performance gains, demonstrating JuICE’s effectiveness. Applying scaling with well-chosen coefficients further amplifies these improvements. This also connects to Q2 raised by reviewer GwP9, where we observed that JuICE exhibits strong robustness to different scaling coefficients.

Question: Comprehensiveness of the classified knowledge conflicts and effectiveness of the proposed method on new conflicts

A primary contribution of our work is the in-depth exploration of language model mechanisms, accompanied by methods grounded in these insights. We classify three key types of knowledge conflicts, providing a clean and controlled setting that illuminates the internal workings of LMs—an aspect largely overlooked by prior research, which tends to focus on high-level behavioral observations. While we do not claim to cover every possible knowledge conflict, our classification addresses a wide range of conflicts and yields actionable insights. Building on the well-defined and representative cases explored in our work, future research can extend these insights to more atypical and complex forms of knowledge conflict.

We sincerely thank the reviewer for the time and thoughtful feedback. Before addressing specific points, we emphasize that our primary goal is to understand the interplay between parametric (internal) and contextual (external) knowledge in LLMs. Our methodology primarily aims to validate the novel discovery, rather than exhaust engineering optimizations from all possible criteria, which, while having their own values, we see as orthogonal and suitable for future work.

Q1: General Abilities and Benchmarking LLMs with JuICE

JuICE is a targeted control tool that depends on whether the user intends to emphasize (a) parametric knowledge or (b) contextual information. Because to obtain the correct answer, such an intent varies across examples and remains an unknown prior in general benchmarks—even within the same dataset (i.e. question1 needs a while 2 needs b)—naively applying the same scaling factor to all examples can negatively affect performance. Consequently, JuICE is most effective when user intent is pre-defined, rather than for general-purpose benchmarking.

Q2: Sensitivity to Head Number and Scaling Factor

We provide the result of our sensitivity analysis in this anonymous link (see GwP9_Q2). We plot the number of heads $K$ (from 2 to 30) and scaling factors $\alpha$ (grid search over 0-5) against the average test accuracy (of three conflict types) with Gemma on World Capital. The result shows that JUICE is highly robust against the choice of hyperparameters.

Q3: Scalability to Larger Models

JuICE runs linearly in the number of attention heads. Systems capable of running a 70B model can also accommodate JuICE.

Experimental Design 2: Real-world Conflict Complexity

We agree that complex cases (e.g., partial conflicts) are important but currently unexplored. We have explicitly acknowledged this limitation in Appendix F of the original paper. Before diving into this more complex problem, the task of understanding conflicts between parametric memory and contextual information (which is the focus of this paper) poses a standalone and substantial challenge, which has not been fully addressed by existing works. Our study employs datasets drawn from diverse domains and explores both parametric and contextual perspectives, presenting the first rigorous, comprehensive analysis of memory-context knowledge conflict. Tackling mixed or partial conflicts likely requires a dedicated study, which we leave for future work.

Experimental Designs 3 & 1: Baseline Comparison & Prompting

Thank you for highlighting this relevant work. IRCAN is indeed one of the best RAG-based hallucination methods focusing on one-sided knowledge conflicts. We will add it to the citation list. The problem setup of IRCAN and our work has many differences, and there exist multiple practical constraints (we will explain later on in detail). Accordingly, a point-to-point complete comparison between the two works is infeasible. However, we managed to identify a shared experimental setting between our paper and IRCAN (the Proverb-Ending dataset, referred to as MemoTrap in IRCAN). Using data reported in the IRCAN paper, we compare the two methods as follows:

JuICE consistently outperforms IRCAN while requiring smaller identification sets. We will add this comparison to the revision.

Here we further explain the infeasibility of a point-to-point comparison between these two methods:

• Universal Component Discovery: IRCAN uses dataset-specific splits; it does not claim cross-domain generality. For our Table 4, we require either a domain-agnostic or no identification set to ensure fair comparison across tasks.
• Heavy Tuning: IRCAN requires ~288 hyperparameter configurations (a 16×18 search grid), plus a large validation set (same size as test). Our benchmarks (e.g., NQ-swap, 5k samples) make this computationally infeasible within the rebuttal window.

Prompting Baseline: The reviewer notes prompt baselines work better for large models (e.g., 70B). Unfortunately, our computing resources cannot even load such models. Our chosen model scales match or exceed those used in prior high-impact interpretability papers [1–4], including IRCAN. Consistent with prior work (e.g., PH3), we adopt prompting as our baseline.

[1] Locating and Editing Factual Associations in GPT, NeurIPS'22 (1143 citations)

[2] Inference-Time Intervention, NeurIPS'23 (409 citations)

[3] Improving Alignment and Robustness with Circuit Breakers, NeurIPS'24 (61 citations)

[4] Cutting Off the Head Ends the Conflict, ACL'24 (Baseline)

Typos

Thank you for your careful checking. We will amend these typos and format issues in the revised version of our paper.

Q1: Effectiveness of JuICE on different architectures

Thank you for the insightful question. Our main paper evaluates JuICE on a range of widely used dense attention-based open-source LMs. Following your suggestion, we additionally test JuICE on Mistral-v0.1-7B, a leading sparse attention-based model. Results below confirm that JuICE remains highly effective in this architecture, reinforcing the generality of our findings:

These results show that JuICE retains strong performance in sparse-attention models. We will include them in the revised version.

Q2: Insights on Model Design

Thank you for bringing up this excellent point. At present, there may be no definitive answer to whether superposition is inherently "harmful", which would require a new, independent investigation. On the one hand, superposition can impede interpretability by making it difficult to disentangle the roles of individual parameters. On the other hand, we speculate it may enhance a model’s compactness and expressiveness, allowing the same parameters to accomplish multiple tasks. The reviewer's comment has inspired us to come up with a bolder speculation: this phenomenon might shed lights on understanding the success of some relatively small models (such as QwQ-32b), which has demonstrated remarkable reasoning performance on par with much bigger ones (such as Deepseek-r1-671b) with much more compact parameter size.

In our paper, we propose methods that recognize the inevitability of superposition, a perspective that has received little attention so far. We would like to leave further investigation to future works: how the LLM components accommodating superposition contribute to model expressiveness and whether increasing such components can lead to more compact models with equal expressiveness.

Weakness: Additional Runtime

We agree that JuICE's strong performance comes with modest runtime overhead, as acknowledged in Appendix F. Future work may improve its efficiency. Additionally, users can trade off speed and performance by choosing JuNE, our lightweight variant, which runs faster and still performs on par with or better than other baselines.

Other Comment: Providing Visual Examples of Model Output Changes

Thank you for the excellent suggestion! We now visualize how JuICE shifts model output probabilities across conflicting scenarios. For each case, we plot the average probability mass over the parametric token, contextual token, and other tokens (calculated as 1 - P(parametric) - P(contextual)) on the World Capital dataset using Gemma. The plot is available at this anonymous link (see bLHn_comment). The results clearly show JuICE consistently promotes the desired parametric token across all conflict types. We plan to include this plot in the revised version.

Thank you for your thoughtful feedback and for recognizing the significance of our work. We agree with the reviewer that many of the ideas presented in our paper extend beyond the scope of knowledge conflicts, as superposition appears to be a widespread phenomenon. Gaining a deeper understanding of its underlying mechanism will ultimately lead to more robust and reliable use of language models.

Additionally, we really appreciate your careful checking and attention to detail in our manuscript. In the revised version, we will clarify the notations in Algorithm 1 according to your recommendations.