Adaptive Localization of Knowledge Negation for Continual LLM Unlearning

We appreciate the reviewer for the constructive and positive comments.

Con 1: lack discussion of accumulative decline

1. As for the accumulative decline in model utility, the rate of decline should not diminish because each unlearning process could cause the same degree of model parameter changes, thereby the same degree of general utility degradation.
2. Given the widespread presence of cascading degradation, the damage to utility from each unlearning task generally increases progressively. Therefore, the rate of decline usually does not diminish but increases. Please refer to the experiments in Figure 2 for evidence.

Thanks for pointing out the issue. We will extend the above discussion in our revision of the paper.

Con 2: Task vector does not incorporate retain data

1. Although GA training can incorporate retain data, the limited retain data struggles to represent the broad knowledge possessed by the LLM. As a result, methods like GA+RT or GA+KL still lead to a significant decline in model performance.
2. Although vanilla task vector training cannot incorporate retain data, our proposed method utilizes retain data in updating the gradient mask. By incorporating it during mask updates rather than directly involving retain data in the optimization objective, this approach mitigates the bias introduced by retain data to some extent.
3. The experimental results on several datasets demonstrate that the task vector method exhibits comparable ability of utility preservation with GA+RT and GA+KL, even if it does not incorporate retain data.

Experiment: Ablation study of hyperparameters

Thanks for pointing out the issue. We have supplemented ablation studies for hyperparameters $s$ and $\lambda$ on the TOFU dataset. The results of the final task are displayed.

Please refer to the responses of Q4 to Reviewer ex1z for more ablation studies.

Weakness 1: Whether cascading degradation occurs commonly in real-world scenarios.

1. In real-world applications, continuously arriving unlearning tasks are typically correlated in terms of data format and content. For instance, as shown in the example in Figure 1, data from different users on the same website tends to be quite similar. Therefore, such inter-task dependency commonly results in cascading degradation in real-world application.
2. Even if different tasks are unrelated to each other, the decline in general utility caused by preceding unlearning tasks can lead to partial forgetting of subsequent tasks, resulting in cascading degradation. We conducted an experiment to verify this: after completing the WHP unlearning task, the performance of the model (F-Rouge) on the TOFU and MUSE News forget set also exhibited some decline, as shown in the table below.

Weakness 2: Advantages of TRAVIS

Please refer to the responses of Q1 to Reviewer ex1z.

Weakness 3: Computational complexity and memory consumption

We propose a computationally efficient algorithm and a memory-efficient algorithm for the mentioned issue. The running time and memory consumption of our method are comparable to baselines. Please refer to Appendix and the responses of Q4 to Reviewer ex1z.

Q1: How the method addresses the issue of accumulative decline

1. Our method alleviates the decline in utility during conducting each unlearning task, thereby minimizing the overall utility degradation in the continual unlearning process.
2. The components of our method also incorporate mechanisms to address accumulative decline. For instance, the Dynamic Gradient Sparsity module facilitates selective fine-tuning of different parameter sets for distinct tasks, preventing the cumulative drift of model parameters that leads to accumulative decline. Additionally, the Adaptive Parameter Modulation module applies smaller learning rates to model parameters fine-tuned by preceding tasks, further avoiding cumulative parameter drift.
3. The mechanisms in our method to mitigate cascading degradation also serve to alleviate accumulative decline, as cascading degradation causes the utility decline of each unlearning task to be increasingly severe, exacerbating the issue of accumulative decline.

Q2: Why not calculate the gradient sparsity only once

Thanks for the insightful question. We apply dynamic gradient sparsity to obtain a trade-off between fine-tuning all parameters and drastic changes to a small subset. We also propose a computationally efficient algorithm for gradient sparsity. Please refer to Appendix E.4 and E.2 for more details.

We would like to thank the reviewer for the positive and insightful comments.

Q1: Empirical validation regarding TRAVIS dataset

Thanks for the constructive advice. We would like to explain it as follows:

1. The TRAVIS dataset consists of a wide variety of topics, enabling a comprehensive evaluation of the model’s overall utility.
2. TRAVIS is generated by the LLM prior to unlearning, thus allowing an accurate assessment of the LLM’s original utility.
3. To evaluate the sensitivity of the TRAVIS dataset to model performance, we conducted a synthetic experiment by adding a small amount of random Gaussian noise to the model parameters. We compared the Rouge scores of the TRAVIS dataset, the TOFU dataset, and the WHP dataset. As shown in the table below, the TRAVIS dataset is the most sensitive to changes in model utility, with TRAVIS Rouge showing a decline when the noise standard deviation reaches 0.5%.

Q2: When the unlearning tasks are not correlated

Thanks for the valuable question. We would like to explain it as follows:

1. Since our method is adaptive, it can effectively handle cases where there is no correlation between tasks. When the unlearning of previous tasks does not affect the performance of subsequent tasks, the soft labels in the Entropic Soft Labels module reduce to one-hot labels; the Dynamic Gradient Sparsity module naturally selects different model parameters for different tasks, and consequently, the Adaptive Parameter Modulation module applies normal learning rates to these non-overlapping parameters. Overall, our method is directly applicable to uncorrelated tasks.
2. In Figure 5, we conducted an experiment on continual unlearning with unrelated tasks, where tasks from different datasets were interleaved. The experimental results demonstrate that the performance of our method surpasses baseline methods in this scenario.

Q3: The dynamic gradient sparsity may lead to incomplete unlearning

Thanks for the insightful question. We would like to explain it as follows:

1. Vital parameters are dynamically selected during the training process. At the beginning of training, the threshold is set low, allowing a larger number of model parameters to be chosen for fine-tuning. As training progresses, the threshold is gradually increased, resulting in fewer model parameters being selected. This approach enables us to achieve a better trade-off between the two objectives of effective unlearning and utility preservation, thus avoiding incomplete unlearning. Please refer to Appendix E.2 and E.3 for more details.
2. To validate the effectiveness of forgetting, we attacked the unlearned model using a relearning method. Specifically, we fine-tuned the model with gradient descent using a small portion of the forget set data and assessed the extent to which the model’s performance on the full forget set (F-Rouge) recovered, thereby determining whether the unlearned knowledge could be easily 'reawakened.' This allowed us to evaluate whether the unlearning algorithm thoroughly forgets the target data. As shown in the table below, compared to other baseline methods, the knowledge unlearned by our method is less likely to be 'reawakened' by this attack approach.

Q4: The dynamic gradient sparsity may lead to incomplete unlearning

Thanks for pointing out the issue. We supplement the following ablation study on the TOFU dataset and will extend it in our revision of the paper. Specifically, we validate whether using the efficient threshold calculating algorithm in Appendix E.2 (eff-threshold) and whether using the memory-efficient algorithm in Appendix E.4 (eff-memory) result in performance degradation. As shown in the table, using these two efficient algorithms sacrifices a slight amount of precision but does not significantly impact model performance.

Weakness: Empirical validation and ablation studies

Thanks for your valuable suggestions. Please refer to the responses above and ablation studies in the paper. Please also refer to ablation studies in the responses to Reviewer HgFq. We will supplement and extend the above experiments in our revision of the paper. If the reviewer has any further concerns, we are more than happy to address them.

We would like to thank the reviewer for the positive and very valuable comments. Below are our responses to the comments.

Q1: The connection to catastrophic forgetting in continual learning

We agree with the reviewer that the connection can be further clarified.

The cascading degradation issue we studied in Section 2.3 is similar to, yet distinct from, the catastrophic forgetting problem in continual learning. The intrinsic causes and outcomes of the two are different.

• Catastrophic forgetting happens in continual learning primarily due to model parameters overwriting. And it results in utility declines of previous tasks.
• Cascading degradation happens in continual unlearning primarily due to inter-task relationships. And it results in drastic declines in normal utility.

We will expand this discussion in the related work section in our revision.

Q2: The method is ad-hoc

Thanks for the insightful comment. We would like to explain it from two aspects.

The proposed method is an integrated system, not merely an aggregation of separate components:

1. The core idea of our method is the dynamic modulation of model parameter changes during each unlearning process. Based on this idea, we propose three components regarding the training objective, model parameters, and optimization, with the core idea running throughout.
2. The different components of our method are interconnected as shown in Figure 3. For example, the Adaptive Parameter Modulation module leverages the learnable mask $M$, computed by the Dynamic Gradient Sparsity module, to represent the relationship between model parameters and the features of each task. This enables adaptive learning rates to prevent the re-unlearning of already forgotten features.

We believe that our work introduces multiple new contributions that inspire further exploration. Here, we would like to list some factors that merit follow-up studies:

1. We are the first to study the cascading degradation phenomenon in continual unlearning, which can easily lead to complete model failure when handling continuous unlearning requests. Although we propose a highly promising method to address this issue, performance of the algorithm can still be further improved.
2. Our proposed method designs the algorithm through three perspectives, paving several potential paths for effective LLM unlearning.
3. The task vector method has received limited attention in LLM unlearning. We validate the effectiveness of the task vector method in LLM unlearning from both theoretical and experimental perspectives, establishing it as a highly promising direction. Our research also aims to inspire interest and motivate further in-depth exploration of this important direction.
4. The method we propose may also provide insights for other domains. The approach of adaptively identifying crucial model parameters based on data and dynamically adjusting the parameter mask could provide insights for the fields of model editing and model sparsity.

We sincerely appreciate your comment and we will add the related discussion in the introduction as well as the conclusion sections in our revision.

Q3: Theoretical discussion in 2.3 seems disconnected from the method

Thanks for the very constructive comments. We would like to explain it as follows:

1. Proposition 2.1 in Section 2.3 is presented to formally verify the existence of the cascading degradation issue. Specifically, we study how the first unlearning task influences the second task.
2. Our method is built on the task vector method. Theorem B.1 and Theorem B.2 mentioned in Section 2.2 theoretically compare the GA and task vector methods, demonstrating that task vector is less prone to over-unlearning and may cause less utility degradation in LLMs.
3. Our method includes formulas obtained through theoretical derivation. For instance, the update rule for underlying vector $m^t$ is derived based on a utility-retaining objective in Equation 8. The derivation details are in Appendix D.
4. We also introduce the following corollary that validates the effectiveness of our proposed method, which will be added in our revision:

Corollary 2.1. Consider the optimization scenario where the model successively unlearns on $D^s$ and $D^f$ with entropic soft labels, yielding intermediate and final parameters: $\theta_s^{E}$ and $\theta_{CUL}^{E}$. The parameter changes in such a scenario during unlearning on $D^s$ are less than vanilla continual unlearning:

$||\Delta\theta_{CUL}^E||_{X^TX}=C^E||\Delta\theta_{CUL}||_{X^TX},$

where $0<C^E<1$ is a constant depending on the datasets. The corollary is based on Proposition 2.1 and it demonstrates that using the proposed entropic soft labels method yields fewer parameter changes in continual unlearning and may result in milder utility declines of LLMs.

Q4: Hard to read Table 1

Thanks for the valuable advice. We will replace it with figures in our revision of the paper.

We sincerely appreciate the reviewer’s positive and encouraging feedbacks, as well as their recognition for the value of our work. We are delighted that the reviewer finds our approach and theory compelling. In our future work, we plan to build upon this foundation by further refining our methods and exploring additional applications of our findings such as sequential model editing. If the reviewer have any additional questions, we are more than happy to address them.