LLM Unlearning via Neural Activation Redirection

We thank the reviewer for your valuable feedback and for recognizing the strong theoretical grounding of LUNAR, its efficiency, and its contribution in addressing an important gap by quantifying controllability in unlearning. Below, we provide our detailed response, and hope it addresses your questions and concerns.

[W2/Q1]:

We agree that explicitly illustrating controlled unlearning’s practical benefits strengthens the motivation. Specifically, controllability is critical for practical applications in several ways:

1. Alignment with user expectations and regulatory initiatives: Real-world AI systems are increasingly designed to express their lack of knowledge explicitly and coherently. This aligns not only with user expectations but also recent regulatory initiatives advocating reliable, transparent, and safe AI behavior. Without controllability, models may generate hallucinations or nonsensical outputs in response to forget queries, as exemplified clearly in Table 1 of the paper.

For example, in the medical domain, if a patient’s record is unlearned from the system, we expect the model to explicitly acknowledge its lack of knowledge when queried, rather than hallucinate or fabricate the patient’s history. Generating uncontrolled responses in such high-stakes settings poses significant risks, potentially leading to highly harmful or misleading outcomes. This underscores the importance of unlearning methods like LUNAR that prioritize controllability.

2. Dynamic and context-aware responses: While some existing methods (e.g., DPO) mitigate hallucinations by generating fixed, static “I don’t know” responses, LUNAR improves controllability by producing natural, dynamic, and contextually appropriate acknowledgments of ignorance, closely matching base model behavior. This ensures higher-quality interactions, greatly enhancing user experience when interacting with unlearned models in practice.

We provided systematic and comprehensive discussions on the practical implications and limitations of uncontrolled unlearning in the Appendix (lines 678–743). We will bring a summary of this discussion into the main text to highlight these practical benefits of controllability in the camera-ready version.

[W1/W3/Q2]:

We thank the reviewer for the helpful suggestions. We agree that the current structure of Section 3.1 and 3.2 could benefit from clearer exposition regarding the sequential flow of the LUNAR pipeline. To clarify, LUNAR first computes an unlearning vector for each layer using Eq.5 in Section 3.1, based on a predefined forget set and a reference set. Then, we select the most effective intervention layer (based on the method detailed in Section 3.2). Finally, we optimize the down-projection matrix of the selected layer using the loss defined in Eq.7.

Our intention in the original ordering was to provide an overarching view of the full unlearning mechanism in Section 3.1 before delving into details of layer selection in Section 3.2. However, we acknowledge that this ordering may feel abrupt without explicitly stating the dependency between these steps. In the camera-ready, we will clarify this sequence at the beginning of Section 3 and explicitly state that layer selection is performed after the unlearning vector is computed but before the redirection is optimized. We believe this reordering of explanation and improved signposting will alleviate the confusion raised.

[Q3]:

We clarify that LUNAR selects and intervenes on a single layer only if the model is deployed in a black-box manner and thus insulated from skip-layer attack - a choice for ultimate computational and memory efficiency.

We considered the potential risk posed by white-box adversaries who might attempt to bypass intervention layer(s) (e.g., via layer-skipping attacks). In such scenarios, we defend against this threat by selecting and modifying multiple top-ranked layers (e.g., top-3 for Llama2-7B), as detailed in Table 4 and Section 6.4. In this case, even if the attacker skips all intervention layers, no unlearned information will be recovered. This strategy maintains robustness without significantly compromising efficiency.

In summary, we recommend practitioners adopting LUNAR to intervene on multiple layers for robustness under white-box threat models, while single-layer intervention is sufficient and most efficient in the common black-box deployment scenario.

[Q4]:

LUNAR targets the down-projection matrix for two key reasons: (1) its central role in factual knowledge storage (line 123), and (2) its suitability for efficient and stable optimization (line 194-196).

1. Prior work [1] has shown that the down-projection matrix within MLPs encodes factual associations, making it a natural site for intervention. Intervening this matrix allows LUNAR to directly overwrite internal representations of forget data. In contrast, other components (e.g., up projection or attention) are more involved in routing “keys”. Blocking the transport of “keys” is not the aim of unlearning.
2. From an optimization perspective, intervening on the down-projection matrix reduces the problem to a linear setting with a convex and smooth loss function (see Appendix B.2). This enables stable and efficient training.

[W4/Q5]:

Thanks for the question. We choose cosine similarity over L₂ norm for layer selection because it better captures semantic alignment with our target refusal behaviors. Specifically, we use Llama models to calculate cosine similarity, which normalizes the last-layer representation before the unembedding matrix, so they are trained to encode (token-level) semantic information in the direction, irrespective of the norm of the vector. Cosine similarity measures the angle between sentence embeddings, emphasizing direction (semantic meaning), which is crucial for evaluating whether a model’s response meaningfully aligns with desired refusals. This choice aligns more directly with the goal of producing coherent and contextually appropriate refusals.

[W5/Q6]:

We have conducted additional experiments on the ES metrics as suggested by the reviewer. As shown in the Table below, on the PISTOL dataset, all baseline methods fail to reduce the forget set ES score below 0.5, whereas LUNAR achieves a substantially lower forget set ES score of 0.25 while preserving a high retain set ES score of 0.97. On the TOFU dataset, all methods, including LUNAR, effectively reduce the ES score on the forget set to near zero. However, baseline methods such as GA, GD, and UKL exhibit a significant drop in the retain set ES score to around 0.3, indicating poor preservation of retained knowledge. Although DPO and NPO demonstrate performance comparable to LUNAR in terms of ES scores, they lack the key advantage of controllability. As discussed in the main paper, LUNAR ensures that the unlearned model abstains from answering rather than hallucinating incorrect content—a distinction not achieved by the baselines.

In summary, ES scores reaffirm that LUNAR consistently outperforms all baseline methods. We will include the results in the camera-ready.

[1] Meng et al "Locating and editing factual associations in gpt"

Dear Reviewer,

Thank you again for your insightful feedback, which has been instrumental in improving the clarity and strength of the paper.

We are writing to kindly ask if our rebuttal and new results have sufficiently addressed your concerns. In particular, we would greatly value your thoughts on the following updates:

• New Evaluation Results: Our additional experiments on ES scores re-confirm LUNAR’s state-of-the-art performance vs baseline methods.

• Benefits of “controllability” in unlearning: We elaborated on the motivation for introducing “controllability” as a critical objective for unlearning research and deployment, particularly through examples in high-stakes healthcare setting. We believe this requirement addresses common shortcomings of prior unlearning methods and supports safer and more practical deployment.

• Clarification of Implementation Details: We clarified intervention strategy and other implementation details raised by the reviewer.

We sincerely hope that our responses and new results have addressed your concerns. If so, we would be very grateful if you might consider revisiting your score. Of course, we are also happy to provide any further evidence, experiments, or analysis should you have additional questions or recommendations.

Thank you once again for your time and valuable guidance!

Warm regards,

The Authors

Thank you for your positive assessment of our rebuttal and for raising your score.

We are glad our response addressed your concerns, and we sincerely appreciate your insightful feedback, which has been invaluable in strengthening our work.

We thank the reviewer for acknowledging the technical soundness and practical contributions of LUNAR.

We appreciate the opportunity to clarify its novelty. While the representation mapping is a general idea exploited by different works, there are substantial technical differences in the details of LUNAR and the works that the reviewer pointed out. Notably, LUNAR uniquely addresses precise instance-level unlearning challenges caused by strong knowledge entanglement, as opposed to unlearning broad concept (line 27, 207)). Additionally, LUNAR delivers minimal parameter updates, provides robust defenses against white-box attacks that prior unlearning methods fail, and improves controllability in producing dynamic, contextually-aware model behavior, clearly distinguishing our contributions from prior work.

Below we address each point in turn and outline the additional material we will include in the camera-ready to make the distinctions more evident.

Comparison to ROME [1]:

Although we build on insights from ROME regarding the role of the down-projection ($W_{out}$), fundamental differences exist from multiple perspectives:

• Input computation: ROME generates inputs to $W_{out}$ by averaging 50 randomly generated token sequences preceding the subject. In contrast, LUNAR directly uses activations from prompting the forget samples once, significantly simplifying the process.
• Target specification: ROME explicitly modifies probabilities of specific tokens (requiring fixed target outputs). However, LUNAR considers predefining output tokens both unnecessary and a weakness, as it fails to emulate base model’s dynamic, contextually-aware response to unseen data. We find that finding model's internal understanding of "ignorance" and redirecting residual activations to this region is not only efficient but also highly effective at separating groups of highly entangled instances.
• Batched unlearning limitation: adapting ROME from editing to unlearning, it is known to struggle with batched edits [5], a limitation that we find can be avoided through LUNAR’s activation-based approach.

Comparison to Intrinsic Evaluation [2]:

LUNAR has several key differences with the work [2] that the reviewer kindly pointed out. First, [2] introduces an evaluation methodology rather than an unlearning approach. Second, [2] evaluates the performance of unlearning a broad concept rather than specific instance-level samples. These substantial differences determine the different scope of [2] compared to LUNAR, thus outlining LUNAR’s novelty. More details follow.

Concept unlearning methods, such as those discussed in [2], are insufficient for instance-level unlearning, which LUNAR tackles, due to strong entanglement caused by overlapping in tokens, semantics, and format. Specifically, [2] aims to identify vectors—located before the down-projection matrix—that are associated with vocabularies commonly used to express a given concept. Prior work has shown that such vectors act as keys for retrieving information stored in the down-projection matrix. Hence, fewer such vectors—resulting in fewer concept-related key tokens—reduce the model’s ability to retrieve information related to that concept.

However, instance-level data points, which are the focus of LUNAR, exhibit strong entanglement between the forget and retain sets (e.g., unlearning relationship between entities A/B while retaining information about the relationship between A/C or B/D). The critical insight is that removing the influence of entity A and B tokens (and broader concept-related tokens) with concept unlearning methods could be counterproductive for retain set performance for instance-level unlearning.

Comparison to Representation Steering [3,4]:

Previous works explored steering between clearly defined contrastive concepts. Indeed in [4], steering is from source concept of male to target concept of female for classification task and from the source concept of toxic to target concept of nontoxic for generative task. There is a lack of work exploring critical scenarios where there is no clear source concept to begin with, as in the case of instance-level unlearning. LUNAR bridges this gap, showing that instance-level samples, despite high token, semantic and format similarity, can be cleanly separated via simple activation steering without explicit contrastive concept pairs. Moreover, LUNAR advances prior representation steering literature by:

• Integrating steering directly within the existing model architecture rather than adding external steering modules.
• Conducting robustness analyses under strong white-box attack scenarios and proposing mitigation strategies, neither of which has been addressed in prior steering works.
• Demonstrating the effectiveness of simple activation addition compared to more complex affine steering procedures [4] for instance-level unlearning.

Lastly, we highlight differences with LACE [3]. LACE also seeks to erase concepts (e.g., gender) rather than specific data points. It casts the concept unlearning problem as a minimax game between a projection P and an adversarial classifier on that one source concept - a setup clearly unsuited for instance-level unlearning. Other notable differences include, among others, that LACE is not a steering based method, it does not optimize base model parameters, and has a persistent cost (one matrix–vector product per application) which steering doesn’t have.

We will further emphasize these differences and expand our comparisons in the camera-ready version.

Response to [Q1]

In the original model before unlearning, forget and retain set activations are not separable (i.e. the red dots cluster together with the green ones). The purpose of Figure 2 is to highlight that LUNAR effectively disentangles the two sets, despite strong original knowledge entanglement that caused suboptimal performance in prior SOTA instance-level unlearning methods.

[5] Li et al. “Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?”

Dear Reviewer,

Thank you again for your constructive engagement with our response, which is instrumental in improving the clarity on our contributions and distinguishing our approach from related lines of research.

We are writing to kindly ask whether our rebuttal has sufficiently addressed your concerns.

To summarize:

[RE our method vs editing]: We have elaborated on how unlearning instance-level knowledge requires the practical ability to dynamically emulate the base model’s behavior, since the unlearned model must behave as it had never encountered the forget data. We also explained the limitations of token-space manipulation in effectively disentangling forget and retain instances. These critical requirements set our work apart from prior research in model editing and contrastive concept steering, both in terms of training objectives, implementation, and practical impact on performance and robustness.

[RE contributions]: We have also highlighted our contributions across several dimensions including, but not limited to, the introduction of “controllability” as a key objective in unlearning, applicability of steering in instance-level unlearning, novel loss functions and implementation strategies, and the resultant state-of-the-art performance compared to existing instance-level unlearning baselines from multiple evaluation perspectives.

We sincerely hope our responses have addressed your concerns. If so, we would be very grateful if you could consider revisiting your score. We remain committed to provide further clarification, experiments, or analysis should you have any specific questions.

Thank you once again for your time and valuable guidance!

Warm regards,

The Authors

We thank the reviewer for their valuable feedback and for recognizing that our work offers important insights into conducting precise interventions for instance-level unlearning while maintaining efficiency and robustness against white-box attacks. We sincerely hope our responses below clarify your concerns and questions. Please let us know if there is anything further we can address.

[W1]:

Thank you for highlighting this important aspect. We considered this risk, and experimented with a variant that intervenes on the top-K layers identified in the layer selection process (we set K = 3, see Section 6.4). We validated the robustness of this variant to a set of attacks that include a ‘layer-skip’ attack, among others (LogitLens, Quantization, Paraphrasing, Reverse-direction). The results, in Table 4, demonstrate that this targeted intervention robustly defends against SOTA attacks that exploit untouched layers. We will better highlight this aspect in the paper.

Further, following your suggestion, we conducted additional experiments using the white-box logit lens attack, which further corroborated LUNAR’s resilience (See details in response to Q7 below).

LUNAR’s deliberate design to intervene in early-to-middle layers, where initial information accumulation and abstract feature formation occur [1], prevents meaningful internal representations of the forget set from forming, thereby achieving robustness alongside efficiency.

[Q1]:

Sorry for the confusion. The plots correspond to the activation space immediately post intervention. Forget and retain activations remain separated through to the last layer.

[W2/Q2]:

Thank you for pointing us to this valuable analysis. In response to your suggestion, we analyzed the layer-wise average L2 distance of data point activations between models before and after LUNAR unlearning for both forget and retain set through all layers.

Our results show a clear step wise increase in the average L2 distance for the forget set immediately after the intervention layer. For example in Llama2-7B, it rises from 0 to approximately 0.01 at the intervention layer 18 and continues to increase, exceeding 0.05 by the final layer. In contrast, the average L2 distance for the retain set remains very close to 0 immediately after the intervention and stayed almost unchanged through to the final layer. This behavior is consistent across base models. It indicates that LUNAR’s intervention is highly localized (well separate the forget and retain set despite knowledge entanglement) and the retain set activations stay close through to the final layer.

[Q3]:

We would like to clarify two points:

First, as shown in Eq.5, the retain set is not utilized when computing the unlearning vector; only the “center of mass” of forget-set activations determines the starting point for activation redirection.

Second, we include only as many randomly selected retain samples as there are in the forget set during optimization (Eq. 6), primarily to prevent optimization from drifting retain set activations. Following your suggestion, we conducted further ablation experiments. Completely omitting retain samples in Eq. 6 would cause unintended over-refusal on the retain set caused by the drift of retain set activations - refusal score of the retain set is 0.548 (unlearn PISTOL with Llama-7B as the base model). This supports the more robust performance of LUNAR in its current setup.

We believe that including only minimal retain samples is a practical, realistic scenario. We are happy to add this ablation to the paper if the reviewer finds it useful.

[Q4]:

Thanks for this interesting suggestion. We have performed the attack as you described. For Llama-7B, this attack resulted in a ROUGE1 score of only 0.025 on the forget set (similar for other base models), confirming no information being recovered and LUNAR’s robustness against this attack variant. We appreciate this valuable input and will incorporate relevant analyses and discussions into the camera-ready.

[Q5]:

Thanks for this suggestion. We have conducted the analysis - results show that performance remains at the same level of the base mode, confirming that LUNAR performs targeted, minimally invasive interventions. We include results on reviewer’s suggested metrics below and will include comprehensive results across model families in the updated version.

[Q6]:

Thank you for highlighting this point, which helps clarify an important aspect of our evaluation setup.

In the standard deployment scenario where skip-layer attacks are not a threat (e.g., black-box access), LUNAR intervenes on a single layer to maximize efficiency.

However, if the risk of skip-layer attack is real, we modify multiple layers using the same methodology. As shown in Table 4, modifying the top-3 layers (identified using the same selection criteria described in Section 3.2) is sufficient to defend against skip-layer attacks. That is even if all modified layers are skipped, the model does not reveal information from the forget set. Importantly, multi-layer intervention maintains the same level of unlearning performance and controllability, though it introduces additional computational cost.

We appreciate your suggestion and will clarify this in more detail, emphasizing that practitioners should adapt the number of intervened layers based on the anticipated risk of skip-layer attack. We will include expanded discussions to clearly differentiate these scenarios in the camera-ready version.

[Q7]:

Thanks for the suggestion. In the paper, we already evaluate LUNAR against four SOTA attacks, demonstrating strong robustness to recovery of the forget set. To further strengthen this analysis, we additionally applied LogitLens to assess whether the model still retains information about the forgotten content.

We conducted this analysis across several layers and reported representative results for layer 17 (before intervention), layer 18 (after intervention), and layer 32 (final layer). At layers 17 and 18, LogitLens produces only unrelated or gibberish tokens, indicating that forget set information is not recoverable immediately before or after the intervention point. At the final layer, the top prediction is the token “I”, consistent with LUNAR’s intended redirection toward refusals (e.g., “I apologize…”). These results further confirm that LUNAR effectively redirects memory traces of the forget set even under direct activation-to-logit mapping.

Prompt: What was the effective date of the contract between Wnzatj SAS and Jzrcws SA?
Expected Answer: 06-02-1998

[1] Gross et. al. “Studying large language model generalization with influence functions”

Dear Reviewer,

Thank you again for your insightful feedback, which has been instrumental in strengthening the paper.

We are writing to kindly ask if our rebuttal and new results have sufficiently addressed your concerns. In particular, we would greatly appreciate your thoughts on the following updates:

• Clarification on Intervention Strategy: We clarified the intervention setup, highlighting that modifying the top-K layers (K=3) offers robust protection against white-box attacks.

• New Robustness Results: Our additional experiments on logic-lens and variant reverse-direction attacks re-confirm LUNAR’s strong robustness.

• Deeper Activation Dynamics Analysis: Our deeper analysis of activation evolution across layers confirms that forget and retain instances remain well disentangled, providing further evidence of the robustness and precision of our intervention.

• Downstream Task Evaluation: New experiments demonstrate that LUNAR’s interventions are highly localized with no impact on the model’s general performance, reinforcing the practicality of our approach.

We sincerely hope that our responses and new results have addressed your concerns. If so, we would be very grateful if you might consider revisiting your score. Of course, we are also happy to provide any further evidence, experiments, or analysis should you have additional questions or recommendations.

Thank you once again for your time and valuable guidance!

Warm regards,

The Authors