DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning

We sincerely appreciate the reviewer’s time and effort in reading our paper and offering thoughtful suggestions.

Q1 & Q2: Trade off score and model utility.

LLM unlearning involves a trade-off between forgetting effectiveness and model utility, as noted in prior work [1]. We evaluate the performance of unlearning with a trade-off score that accounts for both aspects, for example, using the deviation score on the TOFU dataset.

Model utility refers to the model’s general language capability, such as answering unrelated questions and performing downstream tasks post-unlearning. Our experiments are designed to balance these goals: removing targeted knowledge while preserving overall performance. Detailed meanings can be seen in Lines 236–239.

Q3: Forget Prompts. Forget prompts refer to the set of input questions or prompts sampled from the forget dataset D_f, which are used during training to guide the unlearning process. We will revise the manuscript to include a brief definition.

Q4: Exact Match Calculation. Yes, exact match calculations rely on the name or certain keywords. In the entity unlearning setting (TOFU), we store entity names (e.g., personal names) for exact match scores. This does not imply that the Unlearn Store retains or memorizes the entity internally. If privacy constraints prohibit storing exact names, the detection can rely on embedding-based similarity metrics instead. When names are available, exact match can be used; otherwise, alternative metrics are applied.

Q5: How are the safety policies defined?

The safety policies used in our framework are described in Appendix F3. They were initially generated using GPT-4o to cover a broad range of safety criteria and then manually refined for consistency and appropriateness. In practice, these policies are configurable and should be tailored to the specific application or compliance needs of the deployer. Our default policy serves as a general-purpose template, but the framework supports custom policies for more targeted unlearning. For example, in the TOFU dataset, the policy encourages generating fake author names, while in WMDP, it enforces refusal behaviors. This flexibility enables scalable and context-aware unlearning across diverse tasks.

Typos. Thank you for pointing out these detailed issues. We will correct all noted typos and formatting problems in the revised version.

Limitation1: Potential attacks on the detection module. To evaluate this, we conducted a series of attack experiments, as shown in Tables 13 and 14 (Appendix), demonstrating that our full framework remains robust under various adversarial conditions.

1. To isolate and further analyze the detection module’s resilience, we also conducted dedicated attack experiments focused solely on the detector (Table 1). These include AIM attacks, language-mixing attacks, and typo-based perturbations, as introduced in Appendix D.6. Instead of using Attack Success Rate, we report detection accuracy to directly measure the detector's performance under attack. A higher or comparable accuracy relative to the original setting indicates that the detector is robust to these attacks. Our results confirm that the detection module maintains strong performance even under these common adversarial manipulations.

Table 1: The detection accuracy on TOFU forget dataset under different attacks.

2. We also conduct experiments on out-of-distribution (OOD) prompts to evaluate the robustness of the detection module. Rephrased prompts are generated by GPT-4o through paraphrasing the original forget prompts to confuse the detector. Keywords and Short Phrases refers to prompts rewritten using only a minimal set of key terms or fragments. Adversarial prompts include small perturbations such as misspellings, Unicode homoglyphs, or unnatural spacing to evade exact-match detection. Detailed prompt generation instructions will be added in the revised version for reproducibility.

Table 2: Detection accuracy of the TOFU and WMDP detectors on various types of out-of-distribution (O.O.D.) prompts derived from the forget dataset.

3. To evaluate detection performance on non-forget-related, out-of-distribution content, we randomly sample 400 prompts each from SimpleQA and Alpaca datasets. These serve as control datasets not subject to unlearning.

Table 3: Detection accuracy of the TOFU and WMDP detectors on unseen, non-forget-related O.O.D. prompts from SimpleQA and Alpaca. (Forget is the positive class)

On the general set, our detectors correctly classify these prompts as non-forget, exhibiting a low false positive rate. This suggests that the performance of the main LLM on inputs unrelated to the forget set is unlikely to be negatively impacted.

Limitation 2: Increased latency (due to additional steps, and larger context)

We acknowledge that our framework introduces a modest increase in inference-time latency. However, this overhead is minimal and targeted:

• The detection module runs in ~5ms (Table 4), and policy retrieval is nearly instantaneous.
• For non-forget-related prompts, no further processing is triggered, so latency remains equivalent to standard LLM inference.
• For forget-related prompts, where safety and compliance are critical, we argue that an increase in latency is justified, especially when the prompt involves private, harmful, or regulated content that should not be answered.

Additionally, the larger context used for instruction injection contributes to more reliable safeguarding, and we identify future directions like context compression or prompt summarization to further optimize latency.

Overall, our method is designed to be modular and incrementally extensible, making it suitable for safety-critical and commercial LLM deployment settings where retraining is infeasible but continual unlearning is necessary. We propose a novel and systematic unlearning framework aimed at enhancing prompt-based unlearning, which is a largely underexplored area. Our extensive experiments demonstrate strong effectiveness in both forgetting quality and model utility, validating the practicality and impact of our approach. Future work includes improving the latency during in-context intervention, which may lead to a stronger unlearning method.

Table 4: Per-example latency (in milliseconds) for the detection module and unlearned prompt inference under open-ended generation.

Limitation 3: Computational overhead for training guard model. We would like to clarify that:

• We use a relatively small LLM (≤8B) as the guard model, which significantly reduces the computational burden (training takes around 30 to 50 minutes on two A100 GPUs using the Accelerate depending on the tasks).
• Cross-model and cross-phase applicability: Unlike existing training-based unlearning methods [1,2] that require repeated fine-tuning per task, per model, and per unlearning request phase in continual unlearning setting, our guard model is trained once and reused across models and unlearning requests.A single trained guard model can generalize to various base models (e.g., LLaMA3-8B-Instruct, Yi-34B-Chat) and even black-box LLMs (as shown in Figure 2) to enforce unlearning behavior. Additionally, it can be reused during continual unlearning, where new forget requests may arrive over time. This “one-time cost, many-time benefit” design improves efficiency and reusability.
• The practical benefits of the guard model far outweigh the computational overhead required to train it. Once trained, the guard model serves as a core component of our framework, effectively unlearning undesirable information. Importantly, the training process is straightforward and stable, consistently yielding the desired behavior (generate reasoning instruction). In contrast, training-based unlearning methods often struggle to achieve a reliable balance between unlearning effectiveness and preserving model utility [2], especially in real-world or continual settings.

[1] LLM Unlearning via Loss Adjustment with Only Forget Data

[2]  TOFU: A Task of Fictitious Unlearning for LLMs

Thank you for your valuable and detailed evaluation of our study and we would like to provide some clarification to address your concerns. Due to space constraints, some explanations were abbreviated. We’re happy to provide further clarification if needed.

W1: Why training-free methods are preferred in this paper. Training-based unlearning methods are widely used for their ability to deeply modify model parameters and achieve strong forgetting. However, they come with notable drawbacks: high computational cost, extensive hyperparameter tuning, difficulty preserving model utility [3], and incompatibility with API-based or proprietary models. They also struggle with continual unlearning, as each request typically requires new retraining, and risk catastrophic forgetting if not well-constrained.

In contrast, training-free methods are lightweight and more practical for deployment. They do not require access to model weights and are more scalable under resource constraints or privacy regulations. Recent works increasingly explore both directions, highlighting growing interest in training-free alternatives [1,2]. While we do not argue that training-free methods should replace training-based ones entirely, they offer a complementary, pragmatic solution for real-world use cases where retraining is infeasible or undesirable.

[1] In-context unlearning: Language models as few-shot unlearners.

[2] Large Language Model Unlearning via Embedding-Corrupted Prompts

[3] TOFU: A Task of Fictitious Unlearning for LLMs

W2: Clarification on Data Availability for Maintenance. The core challenge is the lack of access to original forget data in real-world settings (e.g., user deletion requests or expired licenses [4]). Once removed, this data cannot be reused for retraining, forcing training-based methods to operate on already-unlearned models, risking compounded degradation. Our training-free approach avoids this dependency by leveraging a detection module, enabling flexible, repeatable unlearning without accumulating utility loss.

[4] On Large Language Model Continual Unlearning

W3: Related work discussion. While several cited papers are already discussed, we acknowledge that [5–7] were not included and will revise the paper accordingly.

[5] Adaptive Localization of Knowledge Negation for Continual LLM Unlearning

[6] Remaining-data-free Machine Unlearning by Suppressing Sample Contribution

[7] Toward Efficient Data-Free Unlearning

W4: New contributions highlighting. Our work is related to In-Context Unlearning [1], as acknowledged in the Related Work section. However, our approach introduces several key innovations that distinguish it. We propose a robust and effective detection mechanism that combines a trained scoring model with the similarity-based metric as a secondary safeguard. In contrast, prior work typically assumes full access to the forget data [1] and relies on simple keyword matching or output filtering [8], which are less robust and flexible. Moreover, instead of relying on handcrafted demonstrations, we leverage the model's instruction-following ability to inject safety specifications directly and further enhance control using reasoning-based prompts. This allows for a more adaptive and scalable intervention mechanism. These two components enable effective, scalable, and training-free unlearning in both standard and continual settings.

[8] Guardrail Baselines for Unlearning in LLMs

W5: Importance of proposed evaluation metrics. Our goal is not to replace existing metrics but to complement them in real-world scenarios like harmful knowledge and continual unlearning.

1. Refusal Quality(RQ) goes beyond refusal rate by evaluating whether the model's refusal is coherent and appropriate, which is an important factor often overlooked in prior works, where refusals can be nonsensical [9].
2. DDS and DUS address limitations of static evaluation [4] by tracking the stability and cumulative impact of repeated unlearning over time. It can serve as a diagnostic tool for evaluating and comparing unlearning methods before deployment.

We continue to report standard metrics on TOFU [10], WMDP, and MUSE for comparability. RQ is used only for WMDP, while DDS/DUS are specific to continual unlearning. We will clarify the motivation and incorporate a more thorough discussion of related work in the revision.

[9] ReLearn: Unlearning via Learning for Large Language Models

[10] LUNAR: LLM Unlearning via Neural Activation Redirection

W6: Clarification on Sample and Concept Unlearning. In our work, sample and concept unlearning are used as complementary categories to structure the unlearning problem space, not as mutually exclusive definitions.

• Sample unlearning focuses on removing specific data instances (e.g., TOFU), where the forget set $D_f$ is well-defined.
• Concept unlearning targets broader semantic categories (e.g., harmful or illegal content), where exact instances are unknown, but suppression of a general class is desired.

These settings can also be combined—for example, WMDP involves removing both specific samples and the concepts they represent.

Regarding concept unlearning, we denote the forget signal as a concept set $C_f={c_1,...,c_n}$, following [11], and aim to ensure that the unlearned model $U(M_{θ_0})$ retains no actionable knowledge for any prompt drawn from $D_f$, which is generated forget prompts based on the concept. We will clarify these definitions in the revised version.

[11] When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?

W7: The MUSE results are in Table 10 in the appendix. 

W8: "Gray-box unlearning" refers to settings with access to model outputs (e.g., logits or responses) but not to weights or training data, e.g., GUARD [12]. While the term hasn’t been explicitly formalized in prior unlearning work, we will clarify its use in the revision.

Equations 4 and 5 describe a layered detection design: the scoring model provides a coarse-grained first-pass filter, and the similarity-based score offers secondary verification by comparing prompt embeddings to the curated unlearn store. This design improves detection reliability and accuracy. Table 5 empirically validates the detector’s effectiveness across tasks.

W9: On Whether Our Method Qualifies as Unlearning.

We respectfully disagree with the claim that our method does not constitute LLM unlearning. While some LLM unlearning methods often involve parameter updates via gradient-based training, our approach addresses a complementary and increasingly practical setting [1,2,12] where model weights are fixed, and unlearning must be enforced at inference time, such as in API-based or frozen-model deployments.

Our method explicitly detects and intercepts prompts related to forgotten content, and modifies model behavior to avoid generating responses tied to that information. In doing so, we fulfill the core objective of unlearning: preventing the model from retaining or revealing specific knowledge[13].

[12] GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection

[13] Large Language Model Unlearning

W10: Inference-Time Cost Justification. We acknowledge that our approach introduces a modest increase in inference-time cost compared to standard generation. However, both the detection module (Table 1) and the policy retrieval component are computationally inexpensive.

For non-forget-related prompts, the detection module runs once, and no further intervention is triggered. Thus, the inference latency remains effectively the same as standard LLM inference for the vast majority of input queries.

For forget-related prompts, safety becomes the top priority. In such cases, a modest latency increase is acceptable, particularly for sensitive or regulated domains where safety outweighs speed. Moreover, future enhancements like prompt summarization or context compression offer promising directions to further reduce intervention cost.

In summary, our method achieves practical, training-free unlearning with minimal deployment overhead. It is especially suitable for real-world scenarios where model internals are inaccessible (e.g., commercial LLM APIs) and latency constraints must be balanced with safety and compliance.

Table 1: Per-example latency (in milliseconds) for the detection module and unlearned prompt inference under open-ended generation.

W11: Comparison with GUARD. Our approach differs fundamentally from GUARD in motivation, design, and deployment feasibility. GUARD uses a trained classifier to filter forbidden tokens during generation, requiring fine-grained control over the decoding process. In contrast, we detect forget-related prompts before generation via a robust detection module that combines a trained scoring model with similarity-based metrics, and intervene using instruction-following with reasoning-based guidance. GUARD assumes access to internal decoding mechanisms, which is infeasible for black-box or API-only LLMs. Our framework is API-compatible and thus more deployable. Additionally, unlike GUARD, we avoid storing forget data by using synthetic negatives or embeddings, improving privacy and enabling continual unlearning. Our method also supports explainable, scalable intervention across multiple models and tasks.

Minor Notation. In Line 110, we use $h_θ$ to denote the conditional probability distribution over tokens, while $M_θ$ denotes the language model as a whole, which generates actual text through decoding.

We sincerely appreciate the reviewer’s time and effort in reading our paper and offering thoughtful suggestions and constructive feedback.

W1:Task-Specific Adjustments and Real-World Applicability. We clarify that our framework is designed to be modular and reusable, minimizing task-specific overhead in practice.

Tasks can be grouped into broad categories: private, harmful, and copyright-related information, each of which may contain multiple subtasks. For each category, the same detection and guard models can be reused with minimal tuning.

• The Unlearn Store is simple to maintain, as it consists of paraphrased or synthetic forget prompts.
• The scoring model is trained using lightweight text samples and can be quickly adapted to new tasks.
• Guard model training is performed once per category and reused across subtasks to generate CoT instructions.

While guardrails may require some task-specific policy definitions, these can be bootstrapped or automated using an LLM or agent guided by category-level templates. Overall, we propose a scalable, training-free unlearning framework that supports generalization with low maintenance cost compared with training-based unlearning methods, making it suitable for real-world deployment.

W2: Computational overhead. We thank the reviewer for this important comment. The generation of guardrail instructions consists of: (1) a fast policy retrieval (<1 ms), and (2) a CoT instruction, which is generated only when a forget-related prompt is detected.

While this generation step introduces additional latency, it is selectively triggered for potentially sensitive inputs, where response safety takes precedence over speed. In such cases, slightly longer inference time is often tolerable, especially in real-world applications like compliance-sensitive interfaces, moderation systems, or regulated domains.

Note that the guardrail is applied after detection, so only a small fraction of total queries experience any added cost. For non-forget-related queries (the majority), there is no latency overhead beyond the base model inference (detection module only takes ~5 ms on TOFU dataset).

Finally, there is significant room for optimization. We propose a novel and systematic unlearning framework aimed at enhancing prompt-based unlearning, which is a largely underexplored area. Our extensive experiments demonstrate strong effectiveness in both forgetting quality and model utility, validating the practicality and impact. This is a good start. Techniques such as prompt distillation, trace caching, or summarized CoT generation are promising directions to reduce this cost further, and we plan to explore them in follow-up work. We will clarify this in the revised manuscript.

W3 Robustness of the detection module and the framework.

To evaluate this, we conducted a series of attack experiments, as shown in Tables 13 and 14 (Appendix), demonstrating that our full framework remains robust under various adversarial conditions.

1. Robustness of the detection module. To isolate and further analyze the detection module’s resilience, we also conducted dedicated attack experiments focused solely on the detector (Table 1). These include AIM attacks, language-mixing attacks, and typo-based perturbations, as introduced in Appendix D.6. Instead of using Attack Success Rate, we report detection accuracy to directly measure the detector's performance under attack. A higher or comparable accuracy relative to the original setting indicates that the detector is robust to these attacks. Our results confirm that the detection module maintains strong performance even under these common adversarial manipulations.

Table 1: The detection accuracy on TOFU forget dataset under different attacks.

2. Robustness to out-of-distribution (OOD) prompts (Forget dataset). We conduct experiments on ****out-of-distribution (OOD) prompts to evaluate the robustness of the detection module. Rephrased prompts are generated by GPT-4o through paraphrasing the original forget prompts to confuse the detector. Keywords and Short Phrases refer to prompts rewritten using only a minimal set of key terms or fragments. Adversarial prompts include small perturbations such as misspellings, Unicode homoglyphs, or unnatural spacing to evade exact-match detection. Detailed prompt generation instructions will be added in the revised version for reproducibility. In Table 2, the detector module is robust to the generated OOD prompts regarding the forget dataset.

Table 2: Detection accuracy of the TOFU and WMDP detectors on various types of out-of-distribution (O.O.D.) prompts derived from the forget dataset.

3. Robustness to out-of-distribution (OOD) prompts (Unseen normal dataset). To evaluate detection performance on non-forget-related, out-of-distribution content, we randomly sample 400 prompts each from SimpleQA and Alpaca datasets. These serve as control datasets not subject to unlearning.

Table 3: Detection accuracy of the TOFU and WMDP detectors on unseen, non-forget-related O.O.D. prompts from SimpleQA and Alpaca. (Forget is the positive class)

On the general set, our detectors correctly classify these prompts as non-forget, exhibiting a low false positive rate. This suggests that the performance of the main LLM on inputs unrelated to the forget set is unlikely to be negatively impacted. Both Table 2 and Table 3 demonstrate the robustness of our detection module under OOD distribution.

W4: Motivation for Dynamic Deviation Score and Dynamic Utility Score.  While model providers aim to preserve both forget quality and utility, this becomes particularly challenging in continual unlearning settings, where new unlearning requests are issued over time. Although the utility degradation from a single unlearning step may appear negligible (e.g., from 0.00001), it can accumulate significantly over time (e.g., reaching 0.1), leading to noticeable drops in performance.

The DDS and DUS are designed to measure the stability and cumulative impact of unlearning methods across multiple rounds. These metrics enable researchers and practitioners to identify methods that are robust to continual unlearning and to detect early signs of performance drift before deployment.

Importantly, DDS/DUS do not replace standard metrics like forget accuracy or static utility; rather, they complement them by capturing long-term behavior in realistic deployment settings. They help identify approaches that maintain consistent behavior and avoid error accumulation, which is critical for real-world applications where continual unlearning is necessary.

Q1: Clarification on MUSE Benchmark and Guardrail Construction.  Our detection module integrates the learned scoring model that captures high-level prompt features to assess alignment and the similarity-based metrics that computes prompt-to-store sample distances for second verification. For the detection module used in MUSE, we first train a chunk-level classifier using forget and retain data split into text segments. To improve generalization, we generate various modified questions (e.g., paraphrased, partial) from this data and train a second, question-aware classifier. These two classifiers form the scoring model, capturing both content and query-level semantics. Additionally, we build an Unlearn Store that contains summaries of forget content, and use similarity-based matching as a second verification step to further reduce false negatives.

High KnowMem Retention. The dual-filtering mechanism allows the detector to accurately distinguish between forget and retain or non-forget content. This ensures that no intervention is triggered to queries from the retain set, contributing to the high KnowMem on it.

Good KnowMem Forgetting. For prompts identified as forget-related, we extract the relevant policy and generate a reasoning-based CoT trace using the trained guard model. These instructions leverage the LLM’s inherent instruction-following ability to enforce forgetting without retraining.

We sincerely appreciate your insightful evaluation of our study. Thanks so much for your positive and valuable review.

W1: Sensitive to truly novel (OOD) paraphrases. We conduct experiments on ****out-of-distribution (OOD) prompts to evaluate the robustness of the detection module.

In Table 1, the detector module is robust to the generated OOD prompts regarding the forget dataset. Rephrased prompts are generated by GPT-4o through paraphrasing the original forget prompts to confuse the detector. Keywords and Short Phrases refer to prompts rewritten using only a minimal set of key terms or fragments. Adversarial prompts include small perturbations such as misspellings, Unicode homoglyphs, or unnatural spacing to evade exact-match detection. Detailed prompt generation instructions will be added in the revised version for reproducibility.

Table 1: Detection accuracy of the TOFU and WMDP detectors on various types of OOD prompts derived from the forget dataset. (Higher the better)

On the general set (Table 2), our detectors correctly classify these prompts as non-forget, exhibiting a low false positive rate. To evaluate detection performance on non-forget-related, out-of-distribution content, we randomly sample 400 prompts each from SimpleQA and Alpaca datasets. These serve as control datasets not subject to unlearning. The result suggests that the performance of the main LLM on inputs unrelated to the forget set is unlikely to be negatively impacted. Both Table 1 and Table 2 demonstrate the robustness of our detection module under OOD distribution.

Table 2: Detection accuracy of the TOFU and WMDP detectors on unseen, non-forget-related O.O.D. prompts from SimpleQA and Alpaca. (Lower the better)

W2: Storage and latency grow linearly. We agree that latency may grow with the number of forget prompts during inference. However, in practice, both the storage and inference-time cost are well-controlled, and the incremental overhead per instruction remains modest:

Storage-wise: we do not store all individual forget prompts. Instead, only modified representative prompts or key information are stored in the Unlearn Store before unlearning. This avoids redundancy. Also note that the Unlearn Store serves as a secondary verification for our detection module.

Latency-wise: detection remains fast (~5 ms), and policy retrieval adds negligible cost. CoT generation is only triggered after detection and thus applies to a small fraction of inputs. This ensures that the average-case latency remains low, and high-latency responses are only incurred when safety is critical.

We acknowledge that scaling to millions of rules remains an open challenge. However, our framework is designed to be extensible. In scenarios with large-scale rule sets: The Unlearn Store can be scaled using representative vector selection to facilitate the detection process. The scoring model can be trained on larger rule datasets to generalize across prompt families. For in-context intervention, we can incorporate context compression or virtual tokens to reduce prompt length and memory usage.

Overall, our method is designed to be modular and incrementally extensible, making it suitable for safety-critical and commercial LLM deployment settings where retraining is infeasible but continual unlearning is necessary. We propose a novel and systematic unlearning framework aimed at enhancing prompt-based unlearning, which is a largely underexplored area. Our extensive experiments demonstrate strong effectiveness in both forgetting quality and model utility, validating the practicality and impact of our approach. To further improve scalability and latency, we plan to explore smarter indexing techniques, deduplication, and in-context guardrail compression in future work, which will lead to a stronger unlearning framework.

Weak 3: Novelty Justification. While we position DRAGON as a practical and deployment-friendly unlearning framework, we respectfully argue that our work offers meaningful algorithmic contributions, particularly in the design of the detection, and in-context intervention modules.

1. Our detection module is more than a simple semantic search. It integrates: a learned scoring model that captures high-level prompt features to assess alignment with unlearning objectives and a similarity-based metric that computes prompt-to-store embedding distances for second verification. These two signals are combined into a unified confidence score, enabling robust and adaptive thresholding to handle distributional shifts and paraphrased attacks. This dual-layer structure forms a lightweight yet effective algorithm for scalable and accurate detection. Table 5 in paper demonstrates the effectiveness of our detection module, which achieves nearly perfect detection performance.
2. In context intervention. Upon detection, we dynamically retrieve relevant safety policies from a curated policy store, ensuring that the response aligns with deployment-specific safety requirements. To enforce unlearning, we leverage the LLM’s instruction-following ability in a CoT-guided intervention strategy: We use rejection sampling to construct a high-quality CoT instruction dataset aligned with safety goals. A trained guard model then generates context-aware reasoning traces, which are prepended to the input to modulate model behavior at inference time. This setup introduces a soft unlearning constraint without modifying the underlying weights, offering a novel prompting-based LLM unlearning strategy that is interpretable, modular, and scalable to black-box settings.
3. Our method supports repeatable unlearning across requests, scales to large models (e.g., GPT-4o), and avoids retraining, offering both practical value and methodological novelty.

We believe these contributions offer not only practical impact but also methodological advancements, especially as the field shifts toward real-world unlearning under constraints like ****user-driven data deletion for continual unlearning setting and API-only deployments.  We will clarify these contributions more explicitly in the revised version.