Composable Interventions for Language Models

Thank you for your thoughtful review and for recognising the contributions of our work in addressing gaps in current evaluation methodologies. We are particularly grateful for your acknowledgment of the practical relevance of our analysis of compositional effects and the insights our findings provide for real-world application.

We hope that our responses and updates address your concerns and encourage you to consider raising your score to calibrate with the other reviewers, who notice a similar weakness but are overall positive. If you still have concerns about our work, we would greatly appreciate further guidance on how we can improve our work to achieve a better score in your review.

W: Examining the underlying mechanisms and causes of composability differences.

We do not believe this is clearly a weakness of the work we do present in the paper—it is a broad suggestion that more work can be done on this topic. We absolutely agree that our work motivates deeper analysis of the underlying mechanisms behind compositional failures. We believe our work can clearly inspire such analyses. But taken together, the breadth of questions our work suggests, the pace of new intervention development, and the pace of model releases suggests that such analysis is best conducted across the community and by researchers during intervention development. Our work is a necessary step towards that goal.

Our findings also already offer some guidance for future research by identifying which techniques exhibit superior composability and under what conditions interventions tend to interact constructively or destructively. For example, our results reveal previously-unknown patterns, such as compression hindering editability and the compatibility of unlearning and editing, which are critical for guiding immediate application and future design.

Overall, our work lays a necessary foundation to showcase that composability is already an issue and to encourage the research community to consider composability as a metric during intervention design. In fact, composability can already guide practitioners as a validation metric during method development and hyperparameter tuning.

Q1: How do different interventions interact with one another? Are there any interference patterns or dependencies that affect their compositional effectiveness?

We study interactions between interventions in terms of resultant model performance. Through this lens, there are many ways interventions interact: Order Dependence: As demonstrated by the Order Sensitivity metric, the sequence in which interventions are applied can significantly impact their effectiveness. For example, compression applied before editing tends to better preserve editability, whereas the reverse often leads to degraded outcomes. Similarly, certain unlearning methods perform more robustly when applied before compression. Category-Specific Interactions: Different categories of interventions exhibit unique interaction dynamics. Compression methods often hinder the success of subsequent editing or unlearning, likely due to changes in how knowledge is encoded. In contrast, editing and unlearning show higher compatibility, suggesting that their objectives and mechanisms are less conflicting. Technique-Specific Interactions: Certain methods, such as AWQ for compression and RMU for unlearning, consistently exhibit superior composability across different setups. These findings highlight the inherent differences in how techniques interfere with or complement one another.

Q2: Why do certain techniques exhibit better composability than others? Are there particular properties or design choices in those techniques that lend themselves to more effective composition?

Understanding why certain techniques exhibit better composability is an exciting direction. While this is future work, we hypothesize that factors such as localized impact, objective alignment, and robustness to interference play a significant role. For example, interventions specifically designed to minimize disruption to unrelated model behavior, like RMU and MEMIT, tend to compose more effectively with other interventions.

Thank you for your thoughtful feedback and for recognising the contributions of our work. We sincerely appreciate your positive assessment of our new metrics, the extensive experimental analysis, and the key findings that highlight the significance of intervention order and composability.

In response to your comments, we have carefully addressed the points you raised. These updates aim to further clarify our contributions and strengthen the paper, building on the insights you highlighted in your review.

We hope that our revisions and detailed responses adequately address your concerns, and we kindly invite you to consider raising your score. If there are additional areas where we could improve, we would greatly value your guidance. Thank you for your constructive feedback and encouragement.

W1: Including more comprehensive insights

We acknowledge that identifying patterns like compression's impact on editing is just a first step. While deeper analysis is an exciting direction for future work, this work intentionally focuses on establishing the foundational evidence for composability as a critical consideration. The rapid pace of model development makes such detailed analysis better suited for broader community investigation. Our findings already provide practical guidance while demonstrating composability's value as a validation metric during intervention development.

W2: Optimal composability may rely on each method

This is not a practical weakness of our work as such but an interesting finding that has implications for researchers and practitioners. It is true that methods within the same intervention class do not always have the same composability as other methods within the same intervention class. The consequence of this is that researchers working on new methods within a given class should test the composability of their new method for completeness. Beyond identifying such findings, our framework provides a standardized way for them to do this.

However, while we acknowledge outliers, there are general trends of composability between intervention classes that give valuable insights: edit before compressing and unlearn before compressing are two such trends.

W3: Low fidelity in results using order-free error and order sensitivity

Our metrics indeed can be similar between methods from within the same class. While we agree they provide valuable insights already, we also agree there may still be more dimensions to examine in the future. However, we also note that we extend beyond the composability metrics to also derive insights from intervention-specific metrics (e.g., Figures 2 and 3). By pairing intervention-specific and composability metrics, we believe our experiments capture sufficient intervention interactions for one paper.

We’d also like to clarify that small differences in the composability metrics successfully reflect small differences in the full interactions (comparing Figures 2 and 3 with their corresponding tables 2 and 3). And the order-free error and order sensitivity successfully capture wide differences in composability where they exist. Two examples of this are: 1) the edit success order sensitivity of LoRA when composed with different unlearning methods (Table 4) ranges from 0.00 to 1.00, using the full fidelity of the metric, and 2) edit success order-free error when composing knowledge editing with compression (Table 2) ranges from 0.10 to 0.59.

Thank you for your thoughtful feedback and for highlighting the clarity of our writing, the strength of our motivation, and the importance of the problem we address. We greatly appreciate your recognition of these aspects of our work. We have thoroughly considered your comments and have conducted additional experiments and clarified key points in the manuscript to address your concerns. We believe these revisions strengthen the paper and provide further insight into the problem and our proposed solution.

We hope that our detailed responses and updates address your concerns, and we kindly encourage you to consider raising your score. If any questions or uncertainties remain, we would greatly appreciate further guidance on how we might improve our work to meet your expectations. Thank you again for your valuable feedback.

(Major Concern) W1: No new technique.

We believe our insights and findings are sufficient to warrant publication—the ICLR call for papers explicitly calls for “datasets and benchmarks” as a relevant topic. Our contributions match comparable papers, which rarely include technical contributions. In fact, we think the scale of our experiments on an interesting and novel problem successfully lays the necessary foundations for future research in this area. So indeed, we propose no new technique (beyond novel measures of intervention composability). But we don’t agree this is a major weakness.

For providing recommendations, this is a great suggestion. The majority of our results section can be viewed as a set of recommendations. To strengthen this point and clarify for future readers, we have distilled our recommendations into the following table which will be displayed predominantly in Section 4 of our paper. Thanks for this suggestion!

W2: Extending to more than two interventions

Our framework absolutely extends to more than two interventions. While we can’t exhaustively consider all combinations of three interventions, we have taken your suggestion and run new experiments composing three interventions. We limit the search space by choosing the most-composable intervention from each category: WANDA 25% sparsity (compression), MEMIT (editing), and RMU (unlearning). Expanding to three interventions allows us to ask the following research questions:

Research Question: Does composing more than two interventions lead to catastrophic regression in overall performance? Across base and instruction-tunes Llama-3 8B, we find that composing these three interventions does not lead to significant variance in overall performance as measured by MMLU. These results are in line with our observations from our initial submission where we find little variance in MMLU performance across compositions in contrast to at times extreme variance in intervention-specific metrics. We also observe similar trends in composability across base and instruct models with triplet compositions.

Llama-3 8B Base

Llama-3 8B Instruct

W3: Inconsistencies in composability patterns across models

We believe this is not inherently a weakness of our work, but a weakness of the popular models and interventions we consider—composability is currently worse than expected and less-predictable than we’d like. This is itself an important finding, not a foundational flaw of our work. We hope our work can especially inspire future intervention developers to at least consider composability as a validation metric to begin rectifying this issue.

W4: Domain-Specific Requirements:

We believe this is not a weakness of our work, but a weakness of the popular models and interventions we consider—composability is currently worse than expected and less-predictable than we’d like. This is itself an important finding. So one of our key contributions is to encourage the community to address this problem, as it opens many new questions (like your listed W1 and W2). By already including results on popular models/interventions, our results can provide some initial guidance. But we do actually hope practitioners and researchers will now know to do such grid searches when necessary and share their results to continue fostering better, more-predictable composability.

W2: Adding comparisons with different generations and sizes of the same model family.

While we already study LMs from different families ranging from 7-9B parameters (Table 8), we agree further insights can be derived from considering model sizes and variations within model families. To address your suggestion, we have extended our experiments to add a new, smaller Llama model: Llama-3.2-1B. Our results are shown in the table below, where we observe that the smaller 1B Llama appears to have similar composability levels to its larger counterparts, as Llama-3.2 1B has an average order sensitivity of 3.60% whereas Llama-3 8B has 2.50%. We also observe that editing after compression is still better, though this effect is bigger in the 8B model. Results across parameter count and model family will be included in Section 4 of the paper.

W3: Quantifying Results Sensitivity:

Thanks for pointing this out—we have already addressed sensitivity but accidentally omitted these results. Thanks for pointing this out. Fortunately, our editing results are actually already averages over 10 data subsets. And all compression methods are deterministic. We will address your comment by adding standard errors into the editing results in Figure 2. For unlearning, we empirically observed they had tiny standard deviations when replicated (e.g., ±0.5% for accuracy), which we will also show in Figure 3. We have added the updated version of this table (Table 2 in the paper) including standard errors below. In this table, we notice the overall trends remain unchanged and not very sensitive to different data subsets.

Thank you for your comments and constructive feedback. We appreciate your recognizing our experimental contributions and that our codebase will be useful to the community. We have carefully addressed your concerns, adding three new sets of experiments and analyses based on your suggestions.

We hope that our response and new experiments address your concerns, and we kindly ask you to consider raising your score. If, after reviewing our responses, you still have concerns about our work, we would greatly appreciate further guidance on how we can improve our work to achieve a better score in your review. Thank you!

W1: Comparing more base models

We agree that understanding how composability varies before and after post-training is an interesting question. While our framework indeed applies here, there are too many open questions here to answer in just one work. But to begin probing this question, we have taken your suggestion and run additional experiments on the instruction-tuned Llama-3-8b-instruct using the most-composable representative interventions from each category: MEMIT (editing), WANDA (compression), and RMU (unlearning). In summary, these preliminary results suggest that our main findings hold between pretraining and instruction tuning. These results suggest that factors such as architecture and pretraining regime are more likely to influence composability than post-training.

Compression → Editing is Best: This observation still holds when pairing MEMIT (editing) and WANDA (compression), though the instruct model is robust to compression when it comes to editing performance. For the instruct model, the edit success for MEMIT→WANDA is 93.93% compared to 99.10% for WANDA→MEMIT.

Unlearning → Compression is Best: This observation still holds when pairing RMU (unlearning) and WANDA (compression), with performance being relatively unchanged between the base and instruct versions. RMU→WANDA WMDP performance (lower is better) is 28.54% and 26.82% for the base and instruct models respectively compared to 31.86% and 31.83% for WANDA→RMU.

Editing←→Unlearning is Composable: We continue to find that MEMIT and RMU are largely composable. However, the delta in performance between pairings is slightly higher for the instruct model for editing metrics.

Thank you for your thorough review! We are glad that you believe we study an important problem, that our experiments capture the problem extensively , and that our framework is flexible to scale and adapt new interventions . We respond to your suggestions as follows:

W1: Generalization to other models:

We’d like to clarify that our experiments already span Llama, Mistral [1], and Yi [2] models (Table 8). We have also extended our experiments to include Llama-3-8b-instruct and Llama-3.2-1b (see general response). We will make this clearer by moving these experiments into the main paper. In these experiments, we find that composability results do generalize across these LMs.

Regarding evaluation datasets, we agree that it is an interesting direction to better understand how far our findings generalize, especially to other datasets. As you note, every dataset has flaws. By studying popular models, datasets, and interventions, our work is a sufficient first step. We will further clarify in the conclusion of the paper that expanding our study to future other datasets is a promising direction for future work.

W2.1: Lack of guidelines

This is a very useful suggestion which will help improve our work by showcasing some key findings. For the interventions we study, we share the best-performing interventions and link to our in-depth analysis in the below table. We will add this concise summary of our key takeaways to Section 4 of our paper.

W2.2: Determining Root Cause of Composability

We agree that better understanding the mechanisms within each intervention that improves or regresses composability is an interesting and important direction. We will expand our conclusions to include some possible factors for composability, building on recent works. For example, recent works in unlearning have highlighted that today’s best approximate unlearning techniques may largely obfuscate knowledge rather than robustly unlearning knowledge from the LM’s weights [5, 6]. While answering these questions is beyond the scope of this current work, further discussion of possible factors in intervention implementations which influence composability can increase our work’s impact as a catalyst for future work on composability.

No code uploaded

We’d like to clarify that our code actually is uploaded in an anonymous repository https://anonymous.4open.science/r/composable-interventions-D005/README.md (see L53 on Page 1). Failing to also upload our code as supplemental materials was an oversight on our part, thank you for pointing this out.

Citations

[1] Young, 0.A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Li, H., Zhu, J., Chen, J., Chang, J., Yu, K., Liu, P., Liu, Q., Yue, S., Yang, S., Yang, S., Yu, T., Xie, W., Huang, W., Hu, X., Ren, X., Niu, X., Nie, P., Xu, Y., Liu, Y., Wang, Y., Cai, Y., Gu, Z., Liu, Z., & Dai, Z. (2024). Yi: Open Foundation Models by 01.AI. ArXiv, abs/2403.04652.

[2] Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L.R., Lachaux, M., Stock, P., Scao, T.L., Lavril, T., Wang, T., Lacroix, T., & Sayed, W.E. (2023). Mistral 7B. ArXiv, abs/2310.06825.

[3] Li, N., Han, Z., Steneker, I., Primack, W., Goodside, R., Zhang, H., Wang, Z., Menghini, C., & Yue, S. (2024). LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet. ArXiv, abs/2408.15221.

[4] Thaker, P., Hu, S., Kale, N., Maurya, Y., Wu, Z.S., & Smith, V. (2024). Position: LLM Unlearning Benchmarks are Weak Measures of Progress. ArXiv, abs/2410.02879.

[5] Deeb, A., & Roger, F. (2024). Do Unlearning Methods Remove Information from Language Model Weights?

[6] Lucki, J., Wei, B., Huang, Y., Henderson, P., Tramèr, F.S., & Rando, J. (2024). An Adversarial Perspective on Machine Unlearning for AI Safety.