We thank the reviewer for their feedback and time. We respond to specific concerns below.

The paper claims dataset-independent unlearning costs based on quantized k-means, but fails to provide adequate justification for why this specific combination would be uniquely effective for LLMs. While quantized k-means is a known technique, its application to LLM embeddings lacks a foundation, particularly regarding stability in high-dimensional spaces.

We wish to point out that our experiments showed Q-Kmeans was as effective as ACoT (which uses standard kmeans++) on the LLM eval datasets we tested, and also as effective as parameter fine-tuning with SISA. While in general this might not be true if we change the embedding dimension used in ERASE, a contribution of our paper is showing that for the tasks we tested (and when using BERT embeddings) it was effective.

The proposed metric C(M) for Holistic Unlearning Cost represents a ratio between unlearning and inference costs, but without proper empirical validation, and it lacks experimental measurements of actual computational costs (such as GPU time and memory usage) to substantiate its claims.

Our experiments did measure actual computational costs. The results in Tables 3 and 4 (in the appendix) computed the FLOPS for training and inference, which are used to directly compare between methods (relative improvements are shown in Table 1). We use FLOPS as opposed to GPU time and memory usage because FLOPS are independent of available hardware and, thus, more accurately capture algorithmic costs.

The methodology section suffers from insufficient comparisons. The authors did not compare their approach against alternative clustering methods (like hierarchical clustering), making it difficult to substantiate claims of method superiority.

There are a limited number of efficient unlearnable clustering algorithms. We are unaware of efficient unlearning operations for hierarchical clustering.

The choice of Big-Bench as an evaluation benchmark is questionable, as it was not designed for unlearning tasks.

We agree other datasets would be beneficial, but want to point out our evaluation was over 15 different tasks. These tasks already allowed us to conclude that which is better between in-context learning and fine-tuning is data dependent.

The paper lacks comparisons with existing in-context unlearning methods, particularly the work by Pawelczyk et al.

In this paper we obtain exact unlearning guarantees, while the mentioned paper (also cited in our related work) only does approximate unlearning (and that too without any guarantees). It is currently difficult to comprehensively evaluate such approximate unlearning methods, e.g., [1], which further motivates our paper. We will make this clearer in our related works section.

[1] Hayes, Jamie, et al. "Inexact unlearning needs more careful evaluations to avoid a false sense of privacy." arXiv preprint arXiv:2403.01218 (2024).

The paper lacks a proper unlearning evaluation framework. Specifically, there is no clear distinction between forget sets and remaining sets, nor is there a comprehensive evaluation of both model utility preservation and forgetting ability.

To clarify, this distinction is only present in approximate/heuristic unlearning where their effectiveness depends on the sets. Our method is an exact unlearning method, which by definition always works. Our evaluation thus focuses on exact unlearning compute.

Regarding the forgetting ability, the authors do not employ standard unlearning metrics, such as membership inference attacks or data extraction leakage tests, which are crucial for validating privacy guarantees.

As mentioned earlier, our method is an exact unlearning method and so by definition will always have perfect unlearning performance under the approximate unlearning metrics. To elaborate, we are producing exactly the retrained models, and those metrics measure differences (of approximately unlearnt models) to the retrained model.

Although the paper cites Pawelczyk et al.'s work on in-context unlearning in the related work section, it fails to provide any meaningful discussion or comparison with this directly relevant approach.

See our response to the earlier question about this paper; we will add more discussion elaborating the difference between their approximate unlearning approach and the guarantees of exact unlearning (which our method does) in our revised draft.

We thank the reviewer for their time and feedback. We discuss specific points below.

Some of the underlying assumptions related to this work need to be mentioned. For example, for exact unlearning in this setup, the underlying assumption is that the ICL examples were not used during any stage of pre-training or instruction tuning.

That is correct, and we will make that assumption clearer in our revised draft.

The paper focuses on unlearning for a specific approach, ACoT. Although the paper motivates its advantage over vanilla CoT and showcases the results in Fig. 2 & 3, its significance over Random COT (random ICL example) selection is unclear in the context of unlearning. I understand that ACoT achieves better results than Random selection. But since Random CoT can perform O(1) deletion, the ideal way to evaluate would be to have a tradeoff between per query deletion cost and the overall performance (normalized aggregate score). In the current version, the paper reports only the normalized scores or inference cost at a time.

We would like to point out one more subtlety, which is ACoT and ERASE do not always perform better than random; this is dataset dependent. We decided to present solely the performance cost as it was already explained earlier in the paper that Random COT has O(1) cost, so it is up to the model provider to decide if the performance loss (on some datasets) is worth the cheaper unlearning cost. It is not clear to us how to value performance gains relative to unlearning computation. We will add this discussion elaborating the performance vs. unlearning consideration to our revised draft (in Section 5.2 if space allows).

The above point also raises a question about the holistic evaluation approach. Apart from inference and unlearning costs, shouldn’t we also consider the overall performance of the algorithm as a dimension to consider? For example, Random CoT has the same inference cost as ACoT and a much lower unlearning cost but the overall performance is poor compared to ACoT.

See our response to the previous question, which provides a detailed discussion on the context dependent nature of the performance and unlearning comparison: it is not clear to us how to value performance with changes in compute. This was unlike inference and unlearning operations, which are both measured with the same computational units (e.g., flops).

Line 102: “While no past work studied unlearning specifically for the fine-tuning stage (i.e., task adaptation stage) of LLMs” — this claim isn’t accurate. There is this recent paper [1] that exclusively focuses on erasing fine-tuning data. Moreover, existing exact unlearning algorithms like SISA and others, can be modified to adapt to the fine-tuning setting.

Thank you for bringing this paper to our attention! We will change our claim to mention this work, but note that it actually does not satisfy exact unlearning (according to our reading of their paper). Specifically, they propose to revert to a checkpoint that did not observe the datapoint to unlearn, but this does not reproduce the apriori trained model that would come from not having a datapoint in that slice. Furthermore, the selection of models amongst those trained from many permuted slices again means we do not reproduce the apriori distribution of models coming from not having a datapoint. This latter technique also seems analogous to the forging attack to fool exact unlearning [1]. More generally, the mentioned paper’s implicit definition of exact unlearning (which is anything that does not “use” a datapoint is exact unlearning) falls under the algorithmic definition issues pointed out in [1]; unlearning needs to be defined w.r.t a specific training/unlearning algorithm, as allowing anything that does not use a datapoint means we can trivially unlearn. After the submission cycle we will reach out to those authors to let them know of this issue.

Note, the exact unlearning definition we follow removes this issue, by defining a specific training algorithm and unlearning with respect to it.

[1] Thudi, Anvith, et al. "On the necessity of auditable algorithmic definitions for machine unlearning." 31st USENIX security symposium (USENIX Security 22). 2022.

We thank the reviewer for their time and feedback! Below we discuss questions raised in the review. Other suggestions will be implemented in our revised draft.

the main weakness to me is the title and positioning itself. It is the first time that I see ICL defined as a fine-tuning method...

We agree that the terminology in-context learning is more clear than fine tuning and propose to change the title (“fast exact unlearning for in-context learning data for LLMs” sounds good to us) and contributions accordingly. See our response to Reviewer s2NR for specific changes to the contribution bullet points. To clarify, this does not change any of the methodology or experimental setup and the results hold.

the error bars in most experiments are huge, it would be good to have a sense of the statistical significance of the results of the paper.

While the error bars are large in terms of accuracy, the methods perform differently (in a statistically meaningful way) when it comes to unlearning cost. That is, the FLOPS required to unlearn varies between the methods, despite their accuracy being comparable.

the paper "In-context unlearning: Language models as few shot unlearners.” seems related, it would be good to clarify more the differences with the present approach.

In this paper we obtain exact unlearning guarantees, while the mentioned paper (also cited in our related work) only does approximate unlearning (and that too without any guarantees). It is currently difficult to comprehensively evaluate such approximate unlearning methods, e.g., [1], which further motivates our paper.

[1] Hayes, Jamie, et al. "Inexact unlearning needs more careful evaluations to avoid a false sense of privacy."

“our models converge after one epoch of SISA finetuning.” isn’t it contrary to the conventional wisdom that repeating data a few epochs improves performance?

It is common with LLMs to only train for one epoch. Supporting this, we generally found that the training loss for our tasks reached near 0 within a single epoch; we hence stop at one epoch to avoid overfitting. We will add a note about this in the appendix with associated training loss graphs.

It would be good to clarify the costs of inference for ICL. The fact that it is linear with the number of token puzzles me, my understanding is that attention is quadratic in the number of tokens. One can use caching to make the cost linear with the ICL prompt length, but that cost still seems quadratic the first time it is done.

Thank you for pointing this out. You are correct that the asymptotic relationship should be quadratic in terms of number of tokens in the context. The asymptotic costs in table 1 will be updated in the revision with “t” -> “t^2”. We note that this has a negligible effect on our experimental results as we manually measured inference flops. The primary effect is improving ERASE’s relative asymptotic cost efficiency against the baseline SISA method.

In fig 2, how can such a slight modification of the algorithm yield such changes ? can we conclude anything based on this, looking at the error bars?

We were unsure which “slight modification” you are referring to. We will list our response to a couple of modifications:

Random vs ACoT: Our results suggest that ACoT performance benefits are task dependant. The ACoT paper also states that it “matches or exceeds the performance of the CoT paradigm that requires manual designs of demonstrations." However, we believe further work is required to accurately understand this dependance.

UMAP vs non-UMAP: The dimension of our text embeddings is large (1536). It is a known that clustering may work poorly in high dimensional settings. Thus, applying dimensionality reduction to our text embeddings could have potentially improved clustering results.

ACoT vs ERASE: We find that in-context learning performance is very sensitive to which examples are chosen, causing high variance for each method. We found on average ERASE and ACoT perform similarly, and believe a significant portion of the variance is intrinsic to the choice of in-context examples.

How does the method scale with dataset size? I expect that at large datasets sizes, ICL starts lagging behind fine tuning.

We agree that if there is an abundance of data, and the task is sufficiently different from the pre-trained model’s data, fine-tuning will perform better. However, this comes with additional unlearning costs. We will make it clearer in the paper that our focus is on the cases where in-context learning performs similarly to fine-tuning.

We thank the reviewer for their feedback and time.

Some of the claims in the contribution are difficult to verify for instance. 1) The first claim in line 073-074 is vague and I am not sure how to verify it. 2) The claim 3 and 4 are also not novel and vague.I request the authors to re-write the whole summary of contributions clearly.

Thank you for pointing this out, we propose to change the contributions bullet points as follows:

1. Showing that for certain datasets, exact unlearning can be efficient by using in-context learning.
2. Proposing an exact unlearning algorithm ERASE for in-context learning, that has dataset and model-size independent unlearning operation costs.
3. A study of when in-context learning provides more efficient unlearning deployments than fine-tuning (and repeating fine-tuning from scratch to unlearn), including changes to inference costs.

The authors need to explain well why unlearning during in-context learning (i.e. no change in parameters) is an important problem. A naive baseline in this setting in removing samples from incontext rather than analyzing the embeddings. The authors have explained this point from line 161-172, however this naive baseline is missing from results.

To clarify, we understand the reviewer to be referring to random in-context selection, whose unlearning operation is replacing the selected example (which by rejection sampling one can see now samples from the correct unlearnt distribution). We have comparisons to this in Figure 2.

We wish to also clarify that only removing the unlearnt in-context example is not a correct unlearning operation for any in-context learning algorithm. Specifically, unlearning is an algorithmic definition, and if the original selection used group statistics (e.g., clustering), we must simulate those decisions on the dataset without the example to unlearn. Motivating this, clustering based in-context learning has been observed to occasionally outperform random in-context selection, and a contribution of this paper is to note this is still efficient to unlearn.

We will incorporate this previous paragraph into the related works to help motivate in-context unlearning.

A key limitation of this work based on my understanding is in-context fine-tuning doesn't work well for preference or instruction tuning like settings. So, ERASE in it's current form can't be applied to this important and widely used setting of fine-tuning.

We will acknowledge this limitation in a revised draft. To be clear, the main claim of the paper is that for some fine-tuning datasets, exact unlearning can be easy as in-context learning can be as performant as fine-tuning.

The comparison to SISA in sec 5.3 and 5.4 seems out of place as SISA is built for unlearning gradient based methods and ERASE is unlearning in-context finetuning.

The claim of our paper is that, when possible, we should do in-context learning instead of fine-tuning as it is much more efficient to unlearn. To show this we must compare in-context methods (ERASE) to their fine-tuning based counterparts (SISA). These sections show that in-context methods have comparable performance (on the datasets we tested) while dramatically reducing unlearning cost.

Why the naive baseline of removing samples from context not reported?

It is in Figure 2, see our response to an earlier comment.

The model and training details are very confusing to me. For instance which LLaMA?

As mentioned on line 267 we use the original LLaMA model from Touvron et al. (2023). The “Fine-Tuning Setup,” “Hyperparameter Selection,” and “Prompt Formatting” sub-sections lines 305-329 outline what we believe to be the most important training hyperparameters. In our revision we will improve readability by adding a table to our appendix showing a complete list of all training hyperparameters.

I think the evaluation needs to be extended some harder tasks such as math generation gsm8K, MMLU , hellaswag and other metrics commonly used in LLM literature for fine-tuning.

Our existing evaluation includes 15 tasks capturing a variety of difficulties, and already allows us to conclude that which is better between in-context learning and finetuning is data dependent.

Do you believe ERASE can be extended for unlearning instruction samples from an instruction tuning dataset?

We see no issue in using ERASE on other datasets where in-context learning is possible. However, the primary question is whether in-context learning is effective at learning from that dataset, and as pointed out by the reviewer, in-context learning tends to perform poorly on instruction tuning.