CodeUnlearn: Amortized Zero-Shot Machine Unlearning in Language Models Using Discrete Concept

3. Ablation Studies

From the table, we observe that while both ReLU and linear activations lead to effective unlearning, models with ReLU activation generally exhibit more stable and consistent performance, as highlighted by their lower improvement drop percentages across metrics like BLEU, METEOR, BERT-P, and BART. Specifically: ReLU outperforms linear activation in terms of stability, particularly on topics like "Julien" and "Wish," where the BLEU and METEOR improvement drops are less variable.In cases where performance metrics are critical, ReLU mitigates excessive degradation, ensuring smoother unlearning transitions.

Thank you for raising these important points. We appreciate the opportunity to clarify and address your concerns. Below is our detailed response to your queries:

1.more baselines

We acknowledge that including more baselines, especially zero-shot unlearning methods, would strengthen the paper. However, most existing unlearning methods are tailored for classification tasks, making them challenging to adapt for generative and context-heavy tasks like ours. This limitation partly influenced our decision to omit a detailed comparative analysis. We recognize the importance of this and will explore broader baseline implementations in future work to better contextualize our method's effectiveness.

2. Additional Metrics

To the best of our knowledge, standard metrics for evaluating unlearning in LLMs are still evolving. Related works commonly use BLEU and BERTScore for measuring performance changes (e.g., Sijia Liu et al., 2024). In our case, we needed to evaluate the model’s performance on both topic and non-topic content. Absolute metrics can be biased or misleading in such scenarios, so we proposed the Normalized Improvement Drop metric to provide a clearer, less biased assessment of unlearning effectiveness. Using the zero-shot model as a baseline offers an intuitive representation of performance changes. We acknowledge that human evaluations or task accuracy measures could better capture fluency and performance post-unlearning. These additional metrics will be considered in future studies to enrich our analysis.

4. Risk of Unintentional Information Loss

We agree that unintended removal of valuable information is a critical challenge in unlearning. Our method inherently introduces a trade-off between unlearning the target topic and preserving unrelated information. For instance:

• The normalized improvement drop metric indicates that non-topic content does experience minor degradation, although it is significantly less than the target topic. Mitigating these risks is an open research problem. Future iterations of our method could incorporate context-aware mechanisms to refine the selection of codes for deletion, ensuring that critical, non-topic-related semantics are retained.

Thank you again for your insightful questions and suggestions. We believe these additions and clarifications significantly enhance the clarity and comprehensiveness of the manuscript.

Thank you for your detailed feedback and for highlighting both the strengths and weaknesses of our work. We would like to clarify some points and seek further elaboration.

1, Machine Unlearning vs. Machine Learning:

Our research focuses on machine un-learning, specifically removing or neutralizing the influence of targeted information in LLMs. The process ensures the model no longer retains or utilizes sensitive knowledge for relevant tasks. If there are specific aspects of our approach that seem unclear in this regard, we would appreciate further clarification.

2, Use of Sparse Autoencoders:

The Sparse Autoencoder (SAE) acts as a bottleneck to regulate the flow of information, helping isolate and disentangle sensitive features for unlearning. We think there should be additional ablation experiments here, please give us some time.

3, Leveraging Sensitive Knowledge:

You raised an interesting question regarding the use of sensitive knowledge during unlearning. Our approach is for models to be able not to exploit private or dangerous data to avoid negative exploitation. We would like you to clarify any further questions you may have about this location.

Minor Questions & Suggestions

12. Placement of Code Selection Process
    Thank you for pointing this out. We have elaborated on the placement of the codebook transformation after the residual connection in the main text.
13. Placement of Section 3.2
    We agree with your suggestion and have moved Section 3.2 to the Related Work section for better flow and to maintain focus in the Method section.
14. Understandability of the Example
    We have added detailed explanations for the examples presented, including translations and clarifications for non-French-speaking readers.

About 9,10,13, we will reply later. Thank you for your patience and understanding. We hope these responses clarify part of your concerns. Thank you again for your thoughtful feedback, which has greatly improved the quality and clarity of our work.

Thank you for your thorough and insightful review. Your comments and suggestions have significantly helped us improve the clarity and technical soundness of the paper. We are grateful for the opportunity to address your concerns and explain our revisions. Below are detailed responses to your comments:

Method

1. Relationship Between Sections 3.1 and 3.3 Thank you for pointing this out. Upon further consideration, we agree that Section 3.3 introduced redundancy. We have now simplified the explanation in Section 3.3 and made its relationship with Section 3.1 more explicit. Section 3.1 describes the foundation of the codebook transformation, while Section 3.3 extends this by introducing the encoder-decoder structure to improve sparsity and enhance the representation's interpretability and effectiveness for unlearning.

2. Differentiability of Code Selection This is a very interesting question. The topk operation filters indices, which are then used to select vectors from the codebook for output and loss calculation. However, the indices themselves do not participate in the gradient computation. We realize that the implementation might give an impression of gradient flow through the indices, but in practice, the gradient is computed only for the selected vectors. Thank you for raising this question, as it allowed us to clarify this aspect.

3. Security Through ReLU Thank you for pointing out the confusion regarding this explanation. Upon reflection, we realize that our initial representation of "security through ReLU" was both unclear and misaligned with the core methodology. We have removed the section. We appreciate your feedback, which helped us refine the clarity and relevance of this section.

4. L1 Penalty and Sparsity
   Thank you for pointing this out. We refer to Adly Templeton et al. (2024) and Samuel Vaiter et al. (2012), which discuss how the L1 penalty promotes sparsity in autoencoders and signal reconstruction, respectively. Based on these works , we decided to use L1 as the penalty term.

5. Requirement of $D_T$ and $ D_{\tilde{T}} $
   Yes, While $D_T$ and $ D_{\tilde{T}} $are necessary for the current implementation, they are generated from the training data without requiring external data. This ensures a self-contained unlearning process. Future work may focus on reducing dependency on these datasets, such as labeling code information during training to enable direct unlearning.

6. Motivation for Using Equation 14
   You're right, it's indeed a problem. There are alternative metrics for measuring enrichment, such as the Kullback-Leibler (KL) divergence, log odds ratio, or weighted frequency ratios. However, we chose the enrichment metric in Equation 14 due to its simplicity and interpretability. Specifically, the log-transformed ratio effectively highlights codes with a substantial difference in activation frequencies between $ D_T $ and $ D_{\tilde{T}} $, making it a straightforward tool for identifying enriched codes. While the chi-squared test is an additional step, it serves to ensure statistical robustness by accounting for low-frequency scenarios where spurious activations might otherwise skew the results. Therefore, we acknowledge that exploring more sophisticated and potentially computationally efficient methods for code enrichment analysis is an important future research direction. Metrics that naturally handle low-frequency scenarios without supplementary tests could streamline the process further and enhance scalability.

Evaluation

8. Unlearning Performance Metrics
   To the best of our knowledge, standard metrics for LLM unlearning are still evolving. Related works often use BLEU or BERTScore (e.g., Sijia Liu et al. (2024)). However, since we evaluate both topic and non-topic performance, absolute metrics might misrepresent the results. Thus, we proposed the normalized improvement drop to provide a clearer and less biased measure of unlearning effectiveness. Using the zero-shot model as a baseline provides a more intuitive and straightforward representation.

9. Relationship Between Codebook Quality/Size and Performance
   We acknowledge this as an interesting and important area for exploration. Preliminary analyses suggest that codebook size and quality significantly impact unlearning performance, and we plan to include more detailed investigations in future work.

Thank you for taking the time to review our work. While we appreciate your efforts, we believe there are some ambiguities in your review.. As such, we find it challenging to address your concerns effectively. We kindly request clarification on your comments to provide a more comprehensive and accurate response.

We are very grateful for your time and respect in reading our paper, and thank you for your thoughtful and constructive feedback. Your comments have provided us with great help in revising the paper and experiments. Whether or not our paper is accepted, we are happy to receive your comments.

We have revised the document extensively in response to your comments and address the following concerns:

Writing and Presentation:

1. Clarification of Model, Task, and Dataset (Points 1, 5, 7, 9):

   • We have revised the experiment and result sections to clearly specify the model architecture, task, and dataset. The LLM used in our experiments is now explicitly described, including details about the transformer layers and their interaction with the codebook.
   • At the same time, we adjusted the structure of the results section to better display.

2. Sample Analysis (Point 8):

   • For the sample, we added caption to help better understand.

Methodological Improvements:

3. Control Dataset (Point 3):

   • The control dataset is generated from the training set by replacing keywords in the target topic dataset with unrelated terms while maintaining the original context. This dataset is only used during the search phase and is entirely derived from the training set, avoiding reliance on any additional or external datasets.
   • We clarified that the test dataset, including novel prompts, is not involved in either the training or unlearning phases. We simulate a practical usage scenario where a user may wish to forget specific information after completing the training with the available training set.

4. Zero-shot Unlearning (Point 4): Your comments are quite correct and we are very sorry for the confusion.

   • We have clarified the concept of "zero-shot" unlearning in our context. The term refers specifically to the forgetting phase, where no additional data or retraining is required. The data for unlearning is drawn exclusively from the training set, and no gradient-based parameter updates are performed during unlearning.
   • While the method still relies on initial training, we emphasize that after this step, the unlearning process operates in a zero-shot manner. This provides a foundation for exploring more advanced zero-shot methods in future work. We have also added a discussion in the appendix regarding potential approaches to omit the initial training step in future research.

Future Work:

• We acknowledge that the initial training phase is still a dependency and limit the "zero shot" range to the forgetting phase. So we added a future work section to illustrate the potential in this direction as well as our shortcomings.

Additional Time for Experiments:

For the remaining 2 and 6, we will reply later. Thank you for your patience and understanding.

We are grateful for your valuable feedback, which has helped refine the paper further.

5. Code Collapse (Point 2)

Thank you for raising this critical point regarding code collapse and the disentanglement of learned codes. Code collapse is indeed a known issue with vector quantization (VQ) techniques, where a small subset of the codebook becomes over-utilized while others remain underutilized or inactive.We acknowledge that the problem still exists. We observed this phenomenon in our experiments—for instance, in a search conducted with a sample size of 500, only 1168 codes out of the total codebook were activated. But fortunately, our unlearning methodology remained effective, as demonstrated in the results. We believe that enhancing the disentanglement of learned codes could further improve the interpretability and effectiveness of the unlearning process.