Response to Reviewer PnWC

We sincerely thank the reviewer PnWC for your valuable and constructive feedback!

Q1: Concerns about our assumptions and their practicality.

We would like to provide further explanations on our assumptions:

(1) In the black-box scenario, we assume that an adversary can query the model with target samples and utilize the outputs to infer their membership. This is the most fundamental assumption for Member Inference Attack (MIA)[1] and is widely considered in literature as well as real-world applications. To further align with real-world scenarios, we further investigate the strictest black-box scenarios, where the adversary can only obtain output scores (e.g., confidence scores) for MIA.

(2) In the white-box scenario, we assume that adversary has access to the model weights before and after unlearning. This assumption could also be realistic in practice. Here, the adversary could be a malicious collaborator, such as a company contracting with the model developer, with access to model weights for local deployment. According to the unlearning policy, the collaborator’s model is also required to be updated after unlearning, enabling the adversary to have the model weights before and after unlearning. Additionally, this assumption is commonly studied in literature[2][3], serving as an "upper bound" for exploring real-world risks.

Q2: Suggestions on utilizing real-world data.

We utilize synthetic datasets to rigorously ensure that the pre-trained models have not seen these samples during the pre-training phase. However, we are concerned that real-world data can not meet our requirement currently: (1) Pre-trained models are trained on vast amounts of real-world data, which is typically closed-sourced. (2) Recent studies indicate that we can not yet robustly detect whether a specific sample was used among massive pre-training data[4].

Therefore, we utilize synthetic samples, which are entirely based on fictional scenarios, and generated by GPT-4-based agents released after the models we utilized. Through these efforts, we hope to ensure that the setups are rigorous and thus avoid being potentially misleading.

Q3: The concern on computational cost (training multiple shadow models) on the proposed U-LiRA+ for LLM.

We train multiple shadow models to ensure the rigorous auditing on unlearning methods. With them, we could closely approximate the distributions of the model outputs and rigorously determine samples' membership through statistical tests.

Though this approach could incur much computational cost, it could also be applied to LLMs: (1) We could utilize Parameter-Efficient Fine-Tuning (PEFT) methods to effectively fine-tune and unlearn few audit samples. Since only few parameters are trainable, the computational cost would be significantly reduced. (2) Our approach is MIA-agnostic, as its core idea is to properly construct and inject rigorous audit samples before unlearning. If more efficient yet equally rigorous MIAs for LLMs emerge, our approach can be flexibly adapted to them.

Q4: Do we choose the models mainly for computational efficiency?

The selection of 1.5B-parameter models balances practicality and computational efficiency: (1) Lightweight LLMs are widely adopted in practice, such as autonomous driving and mobile AI assistants. (2) As mentioned, we aim to ensure rigorous auditing and evaluation, which requires some consuming operations, e.g., training multiple shadow models. Choosing an appropriately sized model enables us to conduct broader evaluations efficiently.

Furthermore, we highlight that the model scale is unlikely to compromise our findings: (1) We demonstrate that existing auditing methods are not rigorous because they fail to correctly select audit samples, which is independent of the target model. (2) We reveal that current unlearning methods fail to completely erase target samples because they cannot accurately identify and remove parameters encoding target knowledge from the large parameter space, which would persist across models of different scales.

Q5: The choice of TPR@LowFPR.

TPR@LowFPR is the most commonly used and rigorous metric for evaluating MIA, quantifying MIA's effectiveness and confidence by measuring the proportion of correctly inferred samples at a low false rate[5].

References

[1] Membership inference attacks against machine learning models, IEEE S&P 2017

[2] Machine Unlearning Enables Camouflaged Poisoning Attacks, NeurIPS 2023

[3] Adversarial Machine Unlearning Requests Destroy Model Accuracy, ICLR 2025

[4] Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data, IEEE SaTML 2025

[5] Membership Inference Attacks From First Principles, IEEE S&P 2022

Response to Reviewer 9rYC

We sincerely thank the reviewer 9rYC for your valuable and constructive feedback!

Q1: How about the proposed reconstruction attack (TULA-DR) against the exact unlearning method (including retraining)?

We mainly focus on TULA-DR against inexact unlearning because: (1) Exact unlearning is relatively safe, as its core idea is to delete the target sample and retrain the model from scratch. Due to the randomness in the training process, the difference in model weights before and after unlearning is less likely to be approximated by the adversary. (2) In practical deployment, developers often use the inexact unlearning methods considering computational efficiency. Thus, exploring reconstruction attacks against inexact unlearning could provide more practical insights.

Q2: How do the findings inspire the development of unlearning mechanism on language models?

Thanks for the insightful question. We hope our findings could inspire future research in the following three aspects:

(1) Developing more precise unlearning mechanisms. Considering the difficulty of existing unlearning methods to completely erase target samples, future work could explore efficient exact unlearning methods (e.g., retraining) or certified inexact approaches capable of accurately identifying target knowledge.

(2) Strengthening unlearning auditing before real-world deployment. Future research should establish rigorous auditing frameworks for unlearning mechanisms from broader perspectives. Beyond verifying whether an unlearned model successfully erases target data, it is also essential to assess and mitigate potential risks, such as reduced robustness.

(3) Rethinking the secure deployment of unlearning mechanisms. While prior research has primarily focused on how to unlearn, less attention has been given to how to deploy unlearning. As demonstrated in this work, future research should pay more attention to the secure deployment of unlearning mechanisms into real-world scenarios while mitigating potential new risks.

Response to Reviewer tLdQ

We sincerely thank the reviewer tLdQ for your valuable and constructive feedback!

Q1: How to determine the convergence of TULA-DR?

Our proposed TULA-DR is an optimization-based attack. Empirically, the optimized candidates converge gradually as the number of iterations increases. A criterion is that the loss of the attack stabilizes at a lower value and no longer continues to decrease. Therefore, when implementing the attack, we could set a maximum number of iterations to stop the optimization process.

Q2: What does the “Decoding” function in Algorithm 3 refer to?

This function refers to transforming the reconstructed embeddings into text space. To implement this function, we use the model’s embedding layer as a “vocabulary” and apply a similarity computation function (e.g., cosine similarity) to match the ordinal indices of the reconstructed embeddings. The resulting list of indices is then converted into readable text using a tokenizer.

Q3: Can U-LiRA+ potentially be applied to generation tasks for future work?

Our approach is task-agnostic, as its core idea is to properly construct and inject rigorous audit samples before auditing. Specifically, in order to implement rigorous unlearning auditing on a text generation task, we should first define the samples that are most vulnerable to unlearning on that task as audit samples. We then inject these audit samples into the training set and evaluate whether the target unlearning method could fully erase them. In addition, to implement U-LiRA+ on the generation task, we could utilize the next-word probability vector in response to a text sample as the model's output for MIA.

Q4: In-depth analysis on the performance differences of existing methods across different data modalities.

This study focuses on textual data and language models. However, our key findings could be broadly applicable to various data modalities:

(1) Existing unlearning methods can not fully erase target samples, primarily due to their inability to accurately identify parameters linked to target knowledge. This limitation could extend to other data modalities.

(2) Analyzing models before and after unlearning allows an adversary to infer membership information about unlearned samples or even reconstruct them. While different data modalities may exhibit varying degrees of privacy leakage, the underlying risk persists.

Q5: The suggestion for quantitatively analyzing the information residuals of unlearned models.

We thank you for your insightful suggestion, and it is indeed a very interesting perspective.

However, our proposed auditing method is currently sufficient to demonstrate that existing unlearning methods can not completely erase the target samples and result in large information residuals. With these findings, we hope to call for the development of more accurate unlearning methods before applying them in practice. Indeed, with the development of unlearning techniques, how to effectively and exactly measure “how much” information residual with theoretically guarantee will be an important problem. We thank you for your inspiration and would be happy to explore this question in future research.

Q6: The concern on "high-confidence detection" of our proposed U-LiRA+.

The reason for our claim is that we utilize the TPR@LowFPR metric. TPR@LowFPR is the most commonly used and rigorous metric for evaluating MIA[1], as it quantifies an attack's effectiveness and confidence by measuring the proportion of correctly inferred samples at a low false rate.

References

[1] Membership Inference Attacks From First Principles, IEEE S&P 2022

Response to Reviewer p1Cm

We sincerely thank the reviewer p1Cm for the valuable and constructive feedback!

Q1: Clear the differences between the proposed TULA and [1].

Here are the key differences:

(1) Broader and more realistic assumptions. [1] considers only the relaxed black-box scenario, where the adversary performs membership inference attacks (MIA) with access to output logits, model architecture and auxiliary dataset. In contrast, our proposed TULA could be further applied to a strict black-box scenario, where the adversary can only obtain output scores (e.g., confidence values), providing more practical insights on real-world risks.

(2) More powerful attack paradigm. For the target sample x, [1] trains the attack model using logits from randomly sampled unseen samples as negative supervision, resulting in a coarse-grained, average-level MIA. In contrast, we train multiple shadow models to accurately learn the logits when x is unlearned or unseen, enabling fine-grained, sample-level MIA.

(3) Target beyond MIA. While [1] focuses solely on MIA, we propose both MIA and data reconstruction attack against unlearning mechanisms, exploring the risks from adversaries with different targets.

(4) Focus on more challenging setting. [1] focus only on tabular and image data with ML models or CNN—traditional setups where MIAs are much easier due to high distinctiveness among samples. However, MIAs on textual data are much more challenging, necessitating a stronger attack [3]. Our results show that [1] is much ineffective against textual data and modern language models.

Q2: Clear the differences between the proposed U-LiRA+ and [2].

Compared to [2], we explicitly construct and inject a rigorous audit set with mislabeled samples before unlearning, ensuring a rigorous auditing in the worst case. We demonstrate that previous auditing methods, including [2], lack rigor due to their failure to properly select audited data, resulting in existing unlearning methods being significantly overestimated.

Q3: How does the adversary know when unlearning has occurred, and have access to model weights?

(1) The occurrence of unlearning could be easily detected in practice. An adversary can continuously query the model with target samples. If the outputs from two consecutive sets of queries change, the adversary could execute the attack to infer whether any samples have been unlearned.

(2) The access to model weights could also be realistic. The adversary could be a malicious collaborator, such as a company contracting with the model developer, with permission to model weights for local deployment. According to the unlearning policy, the collaborator’s model is also required to be updated after unlearning, enabling the access to model weights before and after unlearning. Moreover, this assumption is widely considered in literature[4], serving as an "upper bound" for exploring real-world security risks.

Q4: Why do we focus on text classification tasks?

Text classification tasks enabling us to design rigorous audit samples for rigorous unlearning auditing. Specifically, rigorous auditing requires rigorous audit samples, i.e., most vulnerable samples to unlearning, enabling the worst-case auditing. The utilized mislabeled samples have found to be most vulnerable to privacy leakage on image classification tasks[5]. Although our approach is task-agnostic, rigorous samples for text generation tasks are currently under-explored.

Q5: The Confusions on Figure 2.

We would like to explain Figure 2 based on the process of unlearning auditing: The audit set consists of training and unseen samples. For one original model, multiple unlearned models are derived by applying different unlearning methods to the training samples in the audit set. MIA-based auditing methods are expected to exhibit high attack accuracy on the original model (dashed lines in Fig. 2) and low accuracy on unlearned models (bars in Fig. 2).

Q6: The Meaning of Metric "NTS@1NFS".

NTS@1NFS refers to the Number of True Samples @ 1 False Sample, quantifying the number of correctly inferred samples with only one error for MIAs.

Q7: The occurrence of "</s>" before the synthetic texts.

To fairly evaluate the reconstruction attack, we additionally add padding characters (</s>) to fix the length of unlearned texts.

References

[1] When Machine Unlearning Jeopardizes Privacy, CCS 2021

[2] Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy, preprint 2024

[3] Do Membership Inference Attacks Work on Large Language Models?, COLM 2024

[4] Adversarial Machine Unlearning Requests Destroy Model Accuracy, ICLR 2025

[5] Evaluations of Machine Learning Privacy Defenses are Misleading, CCS 2024

Response to Reviewer c3T7

We sincerely thank the reviewer c3T7 for your valuable and constructive feedback!

Q1: Is it possible for the proposed U-LiRA+ to be adopted to other MIAs?

Yes, our approach is MIA-agnostic (membership inference attack), as its core idea is to properly construct and inject rigorous audit samples before auditing. For an unlearning auditing task, we start by defining a set of the most vulnerable samples (e.g., mislabeled samples) as audit samples on that task. Use half of them as training samples and half as unseen samples. After unlearning the training samples with the target unlearning method, we can utilize any kind of MIA to test whether it is possible to successfully distinguish between the training samples and the unseen samples in the audit set.

Q2: For the TULA-DR, how do the authors initialize the candidates?

We randomly initialized the candidate embeddings with Gaussian distribution. Additionally, it is also acceptable to utilize other approaches for initialization, such as a random-selected sentence's embeddings or uniform distribution. Empirically, the key insight is that the values of the initialized embeddings are preferably distributed within the value domain of normal embeddings.

Q3: What about other ways of calculating the difference for the TULA-MI in the strict black-box case?

Thank you for this insightful question. Indeed, we used the value difference (i.e., A-B) to capture the loss changes on the model before and after unlearning the target sample. However, other approaches such as ratio and logarithm are also acceptable, as long as there is eventually a value to quantify the loss change.

Q4: Why do mislabeled samples represent a "worst-case scenario" for auditing?

Compared to normal samples, mislabeled samples are a small number of counterfactual samples, ensuring the model cannot generalize to them unless explicitly trained. As a result, the trained and unseen mislabeled samples will be very different in output distributions and thus are the most vulnerable to unlearning auditing. Therefore, we inject the mislabeled samples into the training set as the audit samples, in order to simulate the worst-case auditing.