MUSE: Machine Unlearning Six-Way Evaluation for Language Models

We thank the reviewer for their thoughtful feedback and for recognizing our comprehensive experimental evaluation and thoroughness. Below, we provide responses to the reviewer’s concerns.

Response to Weaknesses and Questions

• My major concern is that the experimental results are not well explained. For example, Task Vector's performance in Table 3 is somehow neglected. The results of Task Vector seem inconsistent with prior works such as [1].

Our experimental results in Table 3 show that Task Vector performs poorly in preventing verbatim memorization, knowledge memorization, and privacy leakage on both datasets. This finding aligns with the conclusions drawn in [1], where the authors explicitly mentions that "Task Vector is not enough to remove all harmful knowledge from the model." Their analysis in Table 1 positions Task Vector as one of the less effective approaches, ranking 4th and 5th among six evaluated unlearning methods. Thank you for this feedback regarding the discussion of experimental results. We will expand our discussion of these findings in the revised version.

• Though the paper proposes six properties. These properties may have already been discussed in related works. From my perspective, only C6 sustainability is the new criteria for evaluation.

We design our evaluation with an application-first approach, identifying practical scenarios where unlearning may be applicable (e.g., privacy and copyright protection). From these scenarios, we derive evaluation metrics that reflect the perspectives of both data owners and model deployers. While we acknowledge some overlap with prior works, our primary contribution lies in tying these metrics to real-world applications and demonstrating their use with large-scale datasets, such as our news and books datasets. Furthermore, we introduce practical evaluation scenarios that better reflect real-world challenges. For instance, we evaluate unlearning by removing Harry Potter books while retaining knowledge of its Fan Wiki. To ensure reproducibility and facilitate further research, we have open-sourced our entire codebase, including reimplementations of unlearning methods and the evaluation framework.

• C3, which compares MIA scores between unlearned models and retrained models, is not sound for under-unlearning.

We respectfully disagree with this point regarding under-unlearning. Even without access to retrained models, under-unlearning can compromise privacy: when it occurs, an adversary can detect membership information by observing that MIA scores for $D_{forget}$ samples align more closely with $D_{train}$ patterns than with $D_{holdout}$ patterns—indicating that $D_{forget}$ samples were part of the training data. This privacy leakage problem exists independently of whether retrained models are available for comparison. In our evaluation framework, we use retrained models primarily as calibration.

Furthermore, using MIAs to evaluate unlearning has been widely adopted by prior work [1] and the NeurIPS 2023 Machine Unlearning Challenge [2]. Our benchmark follows these prior standards for evaluating machine unlearning.

[1] Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy. Jamie Hayes, et al.
[2] NeurIPS 2023 Machine Unlearning Challenge, https://unlearning-challenge.github.io/

Dear Reviewer,

We'd appreciate it if you'd let us know if our response has addressed your concerns.

Thanks!

We thank the reviewer for their thoughtful feedback and for recognizing our comprehensive experimental evaluation. Below, we provide responses to the reviewer’s concerns.

Response to Weaknesses and Questions

• Evaluating Additional Unlearning Methods (e.g., Weight Pruning and Model Editing)

Thank you for suggesting additional baselines. In response, we have extended our evaluation to include the selective weight pruning method, with results summarized below:

The results demonstrate that while selective pruning achieves reasonable utility preservation, its effectiveness in removing memorized information is limited.

Regarding model editing, its primary goal is to modify a model’s behavior for specific input-output pairs, whereas machine unlearning focuses on completely removing a training sample's influence. For example, model editing might reassign the prediction of (x, y) to a target output $y'$, while unlearning ensures the model behaves as if (x, y) was never part of the training data. Given these distinct objectives, model editing methods are not directly applicable to the unlearning problem. However, we are happy to incorporate any other unlearning methods the reviewer may suggest.

Dear Reviewer,

We'd appreciate it if you'd let us know if our response has addressed your concerns.

Thanks!

We thank the reviewer for acknowledging the realistic utility preservation setup for Harry Potter and the comprehensiveness of our evaluation framework. Below, we provide responses to the reviewer’s concerns.

Response to weaknesses and questions

• High MIA performance requires justification. AUC of MIA against f_target is reported as 0.0—should it be flipped? Yes, this refers to the classic MIA distinguishing between members (D_train) and non-members (D_holdout). An AUC near 0.5 indicates an ineffective attack, while values near 0 or 1 indicate a strong attack. You’re right about flipping the labels for clarity. We’ll update the paper accordingly, as this won’t affect our results but will improve clarity. Thank you for the helpful feedback!

• Why do MIAs perform near-perfectly in your setup compared to others? Can you add ablations for fewer fine-tuning epochs? As you correctly pointed out, our setup does not have a distribution shift. To address your question, we conducted experiments to examine the impact of finetuning epochs on MIA performance. Specifically, we evaluated the AUC scores of the f_target model on news data fintuned for varying numbers of epochs. Additionally, as per your recommendation above, we flipped the labels of examples in D_train and D_holdout and reported the results below:

  Method	Epoch 1	Epoch 2	Epoch 5
  Target	0.97	0.99	1.00
  Retrain	0.52	0.52	0.52
  GA	0.49	0.50	0.50
  GA_GDR	0.13	0.06	0.01
  GA_KLR	0.89	0.94	0.98
  NPO	0.47	0.44	0.41
  NPO_GDR	0.14	0.08	0.02
  NPO_KLR	0.93	0.97	0.98

These results suggest that the MIA method performs well in our setup with different number of epochs. We are also happy to evaluate any additional MIA methods that the reviewer recommends and include the results in the paper.

• Metrics C4-6 are utility-based—should the utility metric be more elaborate? Thank you for your suggestion. We have incorporated additional evaluations to address your concern. Specifically, we added perplexity measurements on D_retain and the performance on HellaSwag for the target model finetuned on news. The updated results are summarized below, and we will include these experiments in the revised paper: | Method | KnowMem on D_retain (EM) | PPL on D_retain | HellaSwag (Acc %) | |--------|-------------------------|-----------------|-------------------| | Target | 55.2 | 7.63 | 72.5 | | Retrain | 55.0 | 7.72 | 73.2 | | GA | 0.0 | 26.1 | 38.2 | | GA_GDR | 27.3 | 17.5 | 42.5 | | GA_KLR | 44.8 | 9.4 | 52.4 | | NPO | 0.0 | 30.2 | 34.1 | | NPO_GDR | 40.5 | 12.2 | 55.3 | | NPO_KLR | 45.4 | 9.8 | 48.2 |

• Clarify the source of the holdout set for Harry Potter books. The holdout set for books is a split of the Harry Potter books, as shown in Table 8.

• f_retrain definition and terminology overlap with D_retain is confusing. Thank you for pointing this out. We apologize for the typo in line 125 of Section 2. It should be: “Exact unlearning f_unlearn ensures f_unlearn is behaviorally identical to the model resulting from retraining from scratch, denoted f_retrain.” To address the confusion, we will rename f_retrain to f_oracle in the final version for greater clarity and to avoid overlap with D_retain. Thank you for the suggestion!

• Clarifying the Role and Rationale for Metrics By constructing an evaluation framework that provides multi-dimensional metrics motivated from real world priorities from two key stakeholder perspectives, we provide a tool for profiling different unlearning algorithms so that different system providers can make algorithmic decisions based on their priority. In the following, we provide some examples which hopefully could clarify why our choice of metrics are motivated from real world scenarios rather than an arbitrarily chosen list.

C1. No Verbatim Memorization:
Consider a system provider that deals with unlearning requests related to memorization of sensitive personal information (e.g. social security number). A strong guarantee of reduced chance of model outputting the social security number after unlearning should be one of the top priority when choosing unlearning algorithms.

C2. No Knowledge Memorization:
Consider a different system that is designed to help people in creative processes (e.g. story writing). Then personal information leakage may not be a main concern, but knowledge memorization may be important when facing copyright related unlearning requests. For example, if an author found that the system is creating stories based on their work X, they may submit a request based on copyright infringement, and the system provider would want the model to no longer use knowledge from the work X after unlearning.

C3. No Privacy Leakage:
Consider yet another different system that is deployed in hospitals, with potential access to medical records and diagnosis. In this case, a much higher privacy requirement is generally needed than other typical cases for unlearning. Because in general whether an input was used to train the model may not be a top secret, but in this scenario membership inference may lead to direct linkage of a certain kind of diseases to a certain people, which may be an undesirable outcome in many cases.

C4. Utility Preservation:
This is a fundamental metric that needs to be considered in most cases.

C5. Scalability
Depending on the nature of the data and user bases, scalability could be high or low priority. For the example in C1, the unlearning requests related to personal information leakage are likely short, but for the example in C2, the unlearning requests may be much larger (e.g. a book or several books).

C6. Sustainability:
In some systems, such as the example in C3, unlearning requests may happen relatively infrequently but each time it happens, it is related to serious privacy concerns. As a result, it is not ideal to wait and batch many unlearning requests together, so the system needs to have the capability to handle a series of unlearning requests. In this case, this dimension of metrics would be very important for choosing the right unlearning algorithm to deploy.

In summary, our framework provides a multi-dimensional view into an unlearning algorithm, motivated from real world needs from both the system provider and data owner’s perspective. This provides a clear profile for balancing different priorities when choosing the right algorithm to deploy. Furthermore, we hope this framework could also guide future work on designing specialized unlearning algorithms that are especially strong at certain dimensions, for targeting specific application scenarios. Thank you for your valuable suggestion; we will include a detailed discussion in the revised version.

• Incorporating Machine Unlearning Methods into the Experimental Section
  Thank you for the suggestion. We will move the discussion of machine unlearning methods in Section 4 to the experimental section in the revised version

We thank the reviewer for appreciating the comprehensiveness of our evaluation framework and experimental analysis. Below, we provide responses to the reviewer’s concerns.

Response to Weaknesses and Questions

• Necessity of Metrics Such as C5 (Scalability) and C6 (Sustainability)
  C6 (sustainability) is important in real-world scenarios, particularly for compliance with regulations such as GDPR’s “right to be forgotten,” which mandates the timely processing of data removal requests. For example, when requests are submitted 31 days apart, they cannot be batched together, requiring efficient handling of sequential unlearning tasks.
  Similarly, C5 (scalability) also known as deletion capacity is crucial for understanding how algorithms perform with varying sizes of forget sets and request frequencies. For instance, our experiments (Figure 6) demonstrate that methods like NPO perform well with small forget sets but degrade significantly with larger ones. These trends inform system developers on selecting unlearning algorithms that align with their operational needs.

• Integration of Framework and Improvements Over Prior Work
  The key contribution of our framework is not merely adding metrics but selecting relevant ones to create a unified view that balances the perspectives of data owners and service providers. Unlike prior work, which often focused on theoretical or synthetic evaluations, our application-first approach tailors the evaluation to real-world scenarios, such as privacy protection and copyright compliance. For example, we evaluate unlearning by removing Harry Potter books while retaining knowledge of related fan wikis, providing a more practical evaluation scenario compared to TOFU’s synthetic data-based evaluations. Additionally, we have open-sourced our entire codebase, including reimplementations of unlearning methods and the evaluation framework, to facilitate reproducibility and further research.

Thanks for your detailed information, as the content shows the importance for future unlearning benchmark. I will increase my score to 6. I will highly recommend that authors would more focus on the reason that why the six domains are all important, because other indicators could show similar impact on the machine unlearning scenarios.

We thank the reviewer for acknowledging that our evaluation is comprehensive and well-designed. Below, we provide responses to the reviewer’s concerns.

Response to Weaknesses and Questions

• Overall the weaknesses are not significant. I think the scale of data and number of methods may be further extended, for example, the conclusion of a method's effectiveness may change when the forget set gets larger.

Number of Methods:
We appreciate the reviewer's suggestion on extending the evaluation to include more methods. In the next version, we will incorporate recently published unlearning methods, including:
[1] Heterogeneous Decentralized Machine Unlearning with Seed Model Distillation
[2] Machine Unlearning by Reversing the Continual Learning

Additionally, we have open-sourced the entire codebase and set up an easy-to-use leaderboard. This allows method developers to contribute their own implementations, enabling evaluation of any new unlearning approaches on our benchmark.

Scale of Data:
Our study evaluates unlearning methods across varying sizes of forget sets, ranging from 0.8M to 3.3M tokens (Line 432). This scale is significantly larger than those used in prior evaluation benchmarks. Figure 6 in our paper analyzes the scaling trends of different methods, providing insights into how effectiveness varies with the size of the forget set.

• Also the hyperparameter tuning may be sub-optimal and require more elaboration.

As discussed in Line 1001, we use a small validation set to identify the optimal hyperparameters for each unlearning method. We will provide additional details about our hyperparameter tuning process in the revised version to ensure transparency and reproducibility.