SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs

We thank Reviewer mnFd for their thorough and insightful review. We are particularly pleased that the reviewer recognized DSG as a novel unlearning method that effectively leverages the interpretability of SAEs. We appreciate their acknowledgment that our approach addresses key limitations of dominant gradient-based unlearning methods, including high computational costs, hyperparameter instability, and poor performance in sequential and zero-shot unlearning scenarios.

We address the noted concerns below.

1. Limited generalization across model scales and architectures

We used Gemma-2-2B for these experiments for two primary reasons. Firstly, high-quality, publicly accessible SAEs for this model are available via SAE-Scope; these are widely used, have been thoroughly evaluated with well-vetted explanations (e.g., on Neuronpedia), offering a solid foundation for rigorously validating DSG's unlearning mechanism. Secondly, employing Gemma-2-2B ensures alignment with prior research [1] that also utilized this model for SAE-based unlearning, allowing us to show DSG's improvement more directly.

Although our current experiments use Gemma-2-2B, DSG is not fundamentally limited by language model architecture or size. DSG operates on activations within the model's residual stream, and its core mechanisms, such as Fisher Information-based feature selection (derived from SAE reconstruction error) and the dynamic classifier, apply broadly across architectures. The SAE itself is a modular component. Adapting and scaling SAEs to larger language models follows well-established scaling laws (Fig. 4 in [2] shows larger LMs typically requiring wider SAEs for comparable reconstruction error). DSG can readily leverage advancements in both SAE architectures and training recipes (Table 1 in [5]) as they become available. Our primary focus here was to introduce and demonstrate the efficacy of this new unlearning paradigm—combining SAEs with principled feature selection and dynamic clamping—on an established and well-understood setup. We hope the community builds upon our findings and applies DSG to models of larger sizes and varied architectures. However, given DSG's novelty, its theoretical underpinnings, and the extensive empirical evidence convincingly demonstrating its superior forgetting-utility balance and robust performance over gradient-based baselines, we believe this work offers a significant and timely contribution to the field of machine unlearning.

Finally, we believe our work resonates with the COLM guidelines on computational resources. These guidelines value research for its potential impact, even if initial studies like ours (demonstrating a novel, effective unlearning method) use accessible model scales rather than larger models that require extensive resources.

2. Heavy reliance on pretrained SAEs and interpretability assumptions

We acknowledge that our work currently leverages publicly available, high-quality SAEs—a practical approach similar to the broader field's reliance on large, pretrained foundation models. We anticipate a growing ecosystem where robust SAEs for various foundation models are increasingly trained and shared. SAEs exhibit characteristics conducive to modularity and adaptation: research [3] demonstrates their transferability between base and chat models, and work like [4] shows they can be finetuned to capture specialized information in specific subdomains, much like pretrained language models. A core contribution of our work is to demonstrate that these SAEs, when appropriately leveraged with a mechanism like DSG, serve as a highly effective foundation for practical and interpretable unlearning.

The reviewer raises a concern that SAE features may not be semantically aligned and interpretable enough to support zero-shot unlearning in complex domains. In this work, we show that SAE features capture interpretable information in real-world domains such as those covered by our news, books, biology, and cybersecurity datasets. Furthermore, recent research [9] demonstrates SAEs' capability to capture relatively sophisticated concepts like refusal, multilinguality, and arithmetic. The aspirational goal of SAEs, as explored in [7], is to comprehensively represent all of an LM's internal features with sufficiently wide SAEs. While achieving universal, perfect SAE interpretability remains an active research frontier, the field is making rapid progress. This progress includes: improved SAE training methodologies and architectural developments [5]; enhanced capture of rarer or more nuanced features [4]; and more sophisticated automated interpretability techniques [6]. Our approach to zero-shot unlearning directly benefits from these advances, enabling targeted interventions based on semantic understanding.

We thank the reviewer for their constructive feedback and for recognizing the strengths of DSG against gradient-based baselines on standard unlearning benchmarks. We address the points raised below.

1. Regarding Paper Clarity and Figure 3 Annotations

We appreciate the reviewer's feedback on the paper's density. We aimed to provide a comprehensive account of both theoretical justifications and empirical results, and we will incorporate more intuitive explanations and key takeaways in the final version to enhance overall clarity.

Specifically for Figure 3, the digits on the green and blue dashed lines represent the clamp strengths ($c$) used for those particular experimental configurations. This is also noted in the figure caption ("Clamp strengths (c) used for DSG points are shown as annotations"). These annotations help illustrate how varying the clamp strength impacts the trade-off between unlearning efficacy and MMLU accuracy.

2. Regarding Domain Specificity ($D_{\text{forget}}$ and $D_{\text{retain}}$)

The reviewer raised a concern about experiments where $D_{\text{forget}}$ and $D_{\text{retain}}$ are from different domains and suggested discussing performance when they are from the same domain. We would like to clarify that DSG has indeed been evaluated in such settings:

• MUSE Experiments Showcase Same-Domain Performance: While our WMDP experiments use data from different domains, our MUSE experiments (Section 4.2) specifically address scenarios with data from the same or very similar domains.

  • For the NEWS corpus, both $D_{\text{forget}}$ and $D_{\text{retain}}$ are drawn as disjoint random subsets from the same distribution of BBC news articles.
  • For the BOOKS corpus, $D_{\text{forget}}$ (Harry Potter books) and $D_{\text{retain}}$ (Harry Potter FanWiki content) originate from a highly similar knowledge domain. DSG's strong performance in these same-domain/close-domain MUSE settings (detailed in Table 2 and Section 4.2) underscores its applicability beyond cases with stark domain differences. We will ensure this is highlighted more explicitly in the revised manuscript.

• General Prerequisite for Unlearning: We would also like to emphasize that all approximate unlearning methods inherently depend on the model's ability to distinguish, at some level, between forget and retain data. There must exist a model-computed statistic $s(x)$ (for gradient methods, this is $s_{\text{grad}}(x) = \nabla_{\theta}\ell(M_{\theta};x)$; for DSG, $s_{\text{act}}(x)$ is derived from SAE activations leading to $\rho(x)$) whose distributions differ for $D_{\text{forget}}$ and $D_{\text{retain}}$ (i.e., $TV(P(s|D_{\text{forget}}), P(s|D_{\text{retain}})) > 0$). Gradient-based unlearning baselines such as RMU [1], which are trained to inject noise when forget data is detected, are effective only if such a separation provides a discernible signal to differentiate between forget and retain data; without it, their selective actions would be arbitrary.

• DSG's Transparency and Controllability: A key advantage of DSG is that it makes this separability assumption explicit and controllable. This is achieved via the Neyman-Pearson optimal test on $\rho(x)$ with a user-tunable threshold $\tau$, directly controlling the False Positive Rate (FPR). This contrasts with gradient-based methods, where the classifier distinguishing forget/retain data is implicitly embedded within the model weights.

3. Regarding Prior Work on Fisher Information

We thank the reviewer for referencing [2] and [3], which investigate Fisher Information (FI) for unlearning and we will cite these in our revision. DSG's application of FI is distinct due to its nature as an activation-based unlearning method:

• Both [2] (e.g., "FisherNoise," modifying model weights) and [3] (masking LLM weights based on FI contributions) use FI to guide direct modifications or masking of the LLM's weights.
• In contrast, DSG uniquely applies an FI-inspired metric (based on SAE reconstruction error) to score and select interpretable SAE features. Our intervention is then the dynamic clamping of these selected features' activations, not direct manipulation or masking of LLM weights.
• This positions DSG as a novel approach that targets specific, interpretable features within a modular SAE for intervention at the activation level. Our SSD baseline (cited in our paper), which more closely resembles the direction of directly modifying weights based on FI, is outperformed by DSG, highlighting the benefits of our targeted activation-clamping strategy.

---

References

[1] The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (Li et al. 2024).
[2] Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks (Golatkar et al. 2020).
[3] Unlearning with Fisher Masking (Liu et al. 2023).

We thank the reviewer for recognizing the strengths of DSG, including extensive, thought-provoking experiments, and for noting DSG's strong performance against gradient-based methods.

We used Gemma-2-2B for these experiments for two primary reasons. Firstly, high-quality, publicly accessible SAEs for this model are available via SAE-Scope; these are widely used, have been thoroughly evaluated, and provide well-vetted explanations (e.g., on Neuronpedia), offering a solid foundation for rigorously validating DSG's unlearning mechanism. Secondly, employing Gemma-2-2B ensures alignment with prior research [1] that also utilized this model for SAE-based unlearning, allowing us to show DSG's improvement more directly.

Although our current experiments use Gemma-2-2B, DSG is not fundamentally limited by language model architecture or size. DSG operates on activations within the model's residual stream, and its core mechanisms, such as Fisher Information-based feature selection (derived from SAE reconstruction error) and the dynamic classifier, apply broadly across architectures. The SAE itself is a modular component. Adapting and scaling SAEs to larger language models follows well-established scaling laws (Fig. 4 in [2] shows larger LMs typically requiring wider SAEs for comparable reconstruction error). DSG can readily leverage advancements in both SAE architectures and training recipes as they become available (Table 1 in [3]). Our primary focus here was to introduce and demonstrate the efficacy of this new unlearning paradigm—combining SAEs with principled feature selection and dynamic clamping—on an established and well-understood setup. We hope the community builds upon our findings and applies DSG to models of larger sizes and varied architectures. However, given DSG's novelty, its robust theoretical underpinnings, and the extensive empirical evidence convincingly demonstrating its superior forgetting-utility balance and robust performance over gradient-based baselines, we believe this work offers a significant and timely contribution to the field of machine unlearning.

Finally, we believe our work resonates with the COLM guidelines on computational resources. These guidelines value research for its potential impact, even if initial studies like ours (demonstrating a novel, effective unlearning method) use accessible model scales rather than larger models that require extensive resources.

---

References

[1] Applying sparse autoencoders to unlearn knowledge in language models (Farrell et al 2024).
[2] Scaling and evaluating sparse autoencoders (Gao et al 2025).
[3] A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models (Shu et al 2025).

We thank the reviewer for recognizing DSG's strong empirical performance and theoretical grounding.

[W1] Applicability limited to black-box adversaries: DSG is designed to offer a practical and effective unlearning solution for the predominant LLM deployment scenario of API-based access. In this scenario, users and potential adversaries interact with the model in a black-box or gray-box (e.g., API-based fine-tuning) manner, without direct access to model weights [1]. Our experiments confirm DSG's robust unlearning and resilience within this common threat model.

While the original motivation of unlearning has been to produce a model indistinguishable from one trained solely on $D_{\text{retain}}$, it is typically infeasible to attain such a baseline in practice for LLMs due to the expense of (re)-training and the difficulty of completely isolating the forget/retain sets. Thus, existing approximate LLM unlearning research and associated benchmarks almost universally focus on evaluating methods based on their efficacy in achieving output-level forgetting of $D_{\text{forget}}$ while preserving utility on $D_{\text{retain}}$. DSG demonstrably outperforms existing baselines against these practical and widely adopted output-level metrics. We also note that there are other LLM unlearning works [4,8] that adopt non-finetuning based approaches but underperform gradient-based baselines.

White-box setting: The reviewer is correct that in the completely white-box setting, it may be possible to remove DSG's filter. However, recent work has shown that it is also quite easy to perturb gradient-based unlearned models through simple techniques such as quantization [10] or few-shot finetuning on adversarial data [11] to attain pre-unlearning performance, and so it is not clear that alternatives yet provide a meaningful advantage in this challenging setting. We will clarify this in the draft and add relevant discussion to the revised manuscript.
We also note that DSG can be conceptualized as a dynamic internal guardrail that operates on SAE feature activations, rather than solely on input/output filtering like more traditional guardrails [2]. In light of this, an interesting approach in the white-box setting may be to consider that any dynamically intervened activation state A' produced by DSG can, in principle, be realized by an equivalent model weight configuration W'. Distilling such an intrinsic W' from DSG's interventions is a promising area of future research.

[W2] Potential domain specificity: We would like to note that DSG has been evaluated in settings where $D_{\text{forget}}$ and $D_{\text{retain}}$ are closely related.

• MUSE Experiments: Although our WMDP experiments utilize $D_{\text{forget}}$ and $D_{\text{retain}}$ from different domains, our MUSE experiments (Section 4.2) specifically address scenarios involving data from the same or very similar domains. For instance, in the NEWS corpus, both $D_{\text{forget}}$ and $D_{\text{retain}}$ are drawn as disjoint random subsets from the same distribution of BBC news articles. For the BOOKS corpus, $D_{\text{forget}}$ (Harry Potter books) and $D_{\text{retain}}$ (Harry Potter FanWiki content) originate from a similar knowledge domain. DSG's strong performance in these same-domain/close-domain MUSE settings underscores its applicability beyond cases with stark domain differences. We will clarify this further in the revised manuscript.
• General Prerequisite for Unlearning: We would also like to emphasize that all approximate unlearning methods inherently depend on the model's ability to distinguish, at some level, between forget and retain data. There must exist a model-computed statistic $s(x)$ (for gradient methods, this is $s_{\text{grad}}(x) = \nabla_{\theta}\ell(M_{\theta};x)$; for DSG, $s_{\text{act}}(x)$ is derived from SAE activations leading to $\rho(x)$) whose distributions differ for $D_{\text{forget}}$ and $D_{\text{retain}}$ (i.e.,$TV(P(s|D_{\text{forget}}), P(s|D_{\text{retain}})) > 0$). Gradient-based unlearning baselines such as RMU [3], which are trained to inject noise when forget data is detected, are effective only if such a separation provides a discernible signal to differentiate between forget and retain data.
• DSG's Transparency and Controllability: A key advantage of DSG is that it renders this separability assumption explicit and controllable. This is achieved via the Neyman-Pearson optimal test on $\rho(x)$ with a user-tunable threshold $\tau$, which directly controls the False Positive Rate (FPR). This approach contrasts with gradient-based methods, where the classifier distinguishing forget/retain data is implicitly embedded within the model weights.

Dear Reviewer JnnW,

Thank you again for your detailed feedback. We are following up on our rebuttal, which addresses the core concerns you raised.

To summarize our key clarifications:

• On Applicability (W1): In addressing the concern about the broader goal of unlearning, we clarified that DSG is intentionally designed for the API-based deployment scenario (as discussed in the paper, lines 653-654) precisely because the ideal of creating an indistinguishable model is often infeasible due to prohibitive retraining costs of LLMs. The field has therefore adopted more practical, output-level benchmarks for unlearning, and on these benchmarks and metrics, DSG demonstrably outperforms other methods. Regarding the theoretical white-box setting, we noted this is a broader challenge where no current approximate unlearning method has a clear advantage, given that even gradient-based unlearned models can be easily perturbed to restore pre-unlearning performance.

• On Domain Specificity (W2): Our MUSE experiments were designed to evaluate the fine-grained scenarios you described. For example, the forget/retain sets in the NEWS corpus are drawn from the same distribution of BBC news articles, and the BOOKS corpus uses content from a highly similar knowledge domain. DSG shows strong performance in these settings.

• On MUSE Experimental Results (W3): The lower absolute scores are attributable to our use of a smaller Gemma-2-2B base model and a finetuning phase of only 5 epochs. Since all baselines were run on this identical target model, DSG's strong relative unlearning performance over these baselines is a direct indicator of its efficacy.

We hope our response addresses your concerns, and we are happy to discuss any further questions you may have.