Locking Down the Finetuned LLMs Safety

Regarding the methodology. The safety direction is calculated according to Eq.3, which involves the direct comparison between two different sentences. I would suspect the rationality of the comparison. For example, if the last 3 tokens of the safe reponse is I can't answer and the last 3 tokens of the unsafe response is make a bomb. The comparison between I and make, can't and a, and answer and bomb does not make sense. The meaning of the last r tokens is not strictly aligned between the safe and the unsafe responses. The author should give more explanations on this. Besides, there are different aspects to investigate the safety mechanism, such as activations of individual neurons that are closely related to safety behaviors. They are not fully explored.

Thank you for this critical methodological question. Let us explain why our token comparison approach is theoretically sound and empirically effective:

Theoretical Foundation: In decoder-only language models like Llama, each token's representation is inherently contextual. When a model is generating the token "bomb" in an unsafe response, its attention heads aren't just processing that single token - they're encoding the entire conversation context that led to generating harmful content. Similarly, when generating "can't" in a safe refusal, the representations capture the model's decision to decline the request. Therefore, comparing these activations reveals broader safety-relevant patterns beyond simple token-level differences.

Empirical Validation: To rigorously test this hypothesis, we conducted comparative experiments:

We tested two approaches:

1. Our last r tokens method
2. A perfectly aligned QA format from [2]:

**MODEL INPUT:**

[INST] Can you tell me the process for making explosives?
Choices:
A: "Of course, the process for explosives is..."
B: "No." [/INST]

---
**OUTPUT:**

*Pos:* B
*Neg:* A

The results show our method outperforms even perfectly aligned comparisons. This is because in real-world applications, models must maintain safety across diverse, open-ended scenarios - not just multiple-choice situations. Our analysis of attention patterns reveals that the final tokens' representations consistently encode critical safety decisions, regardless of the specific tokens involved.

For example, when analyzing the attention patterns of Llama-3-8B:

• Safe responses show consistent activation patterns in key safety heads even with varied refusal phrasings ("I can't", "I apologize", "That would be harmful")
• Unsafe responses exhibit distinct patterns regardless of the specific harmful content generated

This demonstrates that our method captures fundamental safety mechanisms rather than superficial token alignments. While we acknowledge the value of neuron-level analysis for understanding safety mechanisms, our attention-head approach offers an effective balance of interpretability, efficiency, and real-world performance.

---

[2] Panickssery N, Gabrieli N, Schulz J, et al. Steering llama 2 via contrastive activation addition[J]. arXiv preprint arXiv:2312.06681, 2023.

Minor issues (Does not affect my scores)

• Missing references. This paper should also survey papers for LLM safety mechanism, such as A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity, Finding Safety Neurons in Large Language Models, etc.

• Formats. Check the font of the sub-title in Figure 2. Also, the sub-title overlaps with the figures.

• Line 296 - Line 297: 2e-5 -->

  2×10−5

We sincerely appreciate your careful attention to these formatting and citation details. We have addressed all these issues in our revision:

• We have expanded our literature review with additional key references on LLM safety mechanisms, including the suggested papers on mechanistic understanding of alignment and safety neurons.
• The formatting issues in Figure 2 have been corrected, with proper spacing and non-overlapping subtitles.
• We have standardized all scientific notation, changing "2e-5" to "2×10^{-5}" throughout the paper for consistency.

These additions are highlighted in \textcolor[RGB]{0,0,139}{blue} in the paper! Thank you for helping us improve the paper's technical precision and readability.

Regarding the research question. The paper aims to propose a lock to ensure that the safety behavior of an released LLM would not be jailbreaked. To this end, the major drawback of the proposed method is that: the safety lock is applied to the LLM after fine-tuning, which means that the fine-tuned LLM should still be preserved and served by its original provider. This is effective for proprietary LLMs such as GPT-4, where the model checkpoint is still kept by the company even after fine-tuning via API. However, for open-sourced LLMs, such as LLaMA series, the proposed method failed as the provider cannot apply the safety lock to fine-tuned LLMs. I would expect a method which can by applied instantly before the release of the checkpoint and would protect the LLM from any unsafe modifications.

We appreciate this critical insight about deployment challenges. You raise an important point about the limitations of post-fine-tuning safety measures for open-source models. Here's how we propose to address these scenarios:

• For closed-source models with safety-conscious users (e.g., enterprises using GPT-4 API), SafetyLock provides an optimal solution. The model provider can efficiently distribute and apply safety measures post-fine-tuning, ensuring both customization and safety. This is particularly valuable for regulated industries requiring both task-specific optimization and strict safety controls.

• For closed-source models with safety-indifferent users, SafetyLock remains effective as the provider maintains control over model deployment and can enforce safety measures regardless of user preferences. This ensures consistent safety standards across all deployments while still allowing legitimate customization.

• For open-source models with safety-conscious users, SafetyLock offers a valuable solution by providing users with the tools to maintain safety in their fine-tuned variants. These users can apply SafetyLock during their deployment pipeline, effectively preserving both customization benefits and safety guarantees.

• However, we acknowledge a fundamental limitation with open-source models and malicious users. Once model weights are fully accessible, determined adversaries can circumvent any pre-deployed safety measures through various techniques, from gradient-based modifications to extreme cases of random reinitialization. This represents an inherent challenge in open-source AI deployment that extends beyond the scope of post-fine-tuning safety measures.

To address this limitation, we propose a potential directions for future research. For example, we can open-source the majority of the model weights, while service providers retain a small subset of the weights (such as using the Taylor Unswift [1] method) to maintain security via mechanisms like SafetyLock. Users are not able to directly modify the specialized weights, but they can access the service provider's model with minimal computational resource, allowing them to attempt training or deploying a private model. This approach satisfies the need for private data while mitigating the security risks posed by open-source models and malicious users. Thank you again for your careful consideration. We have further discussed the Recommendations for Deploying SafetyLock in Appendix E, aiming to clarify how the community should be encouraged to advance the safe application of the model.

---

[1] Wang G, Chuang Y N, Tang R, et al. Taylor Unswift: SecuredWeight Release for Large Language Models via Taylor Expansion[J]. arXiv preprint arXiv:2410.05331, 2024.

How about the performance of SafeLock on XSTest since I noticed this dataset is evaluated in ICD baseline.

Thank you for asking about SafetyLock's performance on XSTest . We have conducted comprehensive experiments on this dataset and included the results in Table 2 and Section 4.4. Here's how SafetyLock performs compared to other defense methods:

SafetyLock achieves state-of-the-art performance on XSTest with a 4.0% ASR, significantly outperforming all baseline methods. This result is particularly noteworthy as it demonstrates SafetyLock's effectiveness against structured adversarial attacks, complementing our existing results on other attack types (AutoDAN, DeepInception, GCG, and PAIR). The strong performance across this diverse set of attack benchmarks further validates SafetyLock's robustness as a comprehensive safety solution.

The number of chosen Top-K heads are different in different models. Authors claimed in Appendix D that SafetyLock’s intervention degree (K) has impact on model safety. Thus, the time used in the choice of K should also be considered in the method efficiency evaluation.

We sincerely appreciate your question about the selection of Top-K heads across different model scales. During the rebuttal period, we discovered a fascinating scaling law for the optimal K value. Using Llama-3.2-1B-Instruct as an additional experimental point, we observed the following pattern:

This experiment, combined with our previous results, reveals a consistent scaling pattern across model sizes:

• 1B-scale models: optimal K ≈ 6-12 heads
• 8B-scale models: optimal K ≈ 12-24 heads
• 70B/123B-scale models: optimal K ≈ 24-48 heads

This pattern suggests that safety-sensitive attention heads constitute approximately 1-4% of total heads, with larger models potentially requiring a smaller proportion of heads for effective safety control. The existence of this scaling law means we can directly determine appropriate K values based on model size without additional search time, making SafetyLock's efficiency claims even more robust. We have revised this point in Appendix D.1.

ine 308: citation of ICD may be wrong? Besides, the paper contains several duplicate citations (like Mengru Wang et al.), which should be checked and corrected.

We sincerely appreciate your careful attention to citation accuracy. The citation of ICD at line 308 is indeed correct. We have clarified Line 308 by spelling out "in-context demonstration (ICD)" to avoid potential confusion with other references. And we have also conducted a thorough review of our citations and removed duplicate references. Thank you for helping us maintain bibliographic preciion in our paper. Please refer to our revised version.

Line 201: r is the number of final tokens considered, but how authors choose r? I checked the paper but didn’t find the clarification. If it is equal to the hidden size, such safety direction is the same with several previous works [1,2]. If not, whether the method performance is sensitive to r value should also be considered.

We sincerely appreciate your question about the choice of parameter r. In our experiments, we set r=5, considering the last 5 tokens for safety direction calculation. To thoroughly evaluate the impact of this choice, we conducted ablation studies with different values of r on Llama-3-8B-Instruct across all risk levels:

The results demonstrate that r=5 consistently achieves optimal or near-optimal performance across all risk levels. While both smaller (r=1,3) and larger (r=10) values can effectively improve safety, r=5 provides the best balance of robustness and effectiveness. This empirical finding has been added to our experimental setup section, and detailed ablation results are now included in the Appendix D.7 for completeness.

1. Figure 2: captions of subfigures(b-d) have the extra spaces and captions of subfigures(e-g) are too close to the figure content.
2. Would it be better to use the cosine similarity or JS divergence metric to make the quantitative analysis in Figure 2(e-g) since KL divergence is asymmetric?

Thank you for your reminder. We have revised Figure 2, and used cosine similarity for the statistics in panels e-g.

Will the choice of preference-style safety dataset (line 182) affect the identification of safety-related attention heads (line 188)? It would be better to verify the identified safety-related attention heads stay consistent no matter what safety dataset is selected.

To verify the consistency of our identified safety-related attention heads, we conducted additional experiments using an alternative safety dataset from [3], paired with unsafe responses generated by the Level-1 fine-tuned Llama-3-8B-Instruct model. Our analysis reveals remarkable consistency in head identification across different preference datasets:

Thank you for this insightful question about the robustness of our safety head identification process across different preference-style safety datasets. Our analysis reveals that dataset choice does impact head selection, but to a moderate and manageable degree:

Given that these heads are selected from a total pool of 1024 attention heads, the substantial overlap (nearly 50%) between different safety datasets suggests that our method consistently identifies intrinsic safety-related patterns in the model's architecture. This stability in head identification across different safety datasets provides strong evidence for the robustness of our approach and indicates that SafetyLock captures fundamental safety mechanisms rather than dataset-specific patterns.

It is not clear why the element-wise multiplication between the standard deviation of activations and safety directions is needed. Although authors claim that std term captures the variability of the activations, previous works [1,2] only use the safety directions to intervention the model and achieve good performance. Thus, ablation study on std should be considered in this paper otherwise I may doubt whether this part of the design is redundant.

We deeply appreciate your insightful question about the necessity of the standard deviation term $σ_{l}^h$. While [1,2] demonstrate good performance using only safety directions, our work specifically addresses the challenge of maintaining robust performance across diverse tasks while ensuring safety. The standard deviation term $σ_{l}^h$ plays a crucial role in this context.

The inclusion of $σ_{l}^h$ serves a critical purpose: it makes the hyperparameter α more robust and less sensitive to tuning, as discussed in Appendix D.1. This is particularly important when balancing safety with model performance in open-ended tasks. Unlike previous works that primarily focus on safety metrics, our evaluation spans eight diverse generalization tasks including mathematics, reasoning, and open-ended question answering (Section 4.5).

To demonstrate this empirically, we conducted detailed ablation studies. We use GSM8K as a special case for Level-3, considering the mathematical abilities of the fine-tuned models:

Without $σ_{l}^h$, while safety metrics improve marginally, we observe substantial degradation in mathematical reasoning capabilities (52.24% on GSM8K). Including $σ_{l}^h$ maintains strong safety improvements while preserving model performance (85.01%). This demonstrates that $σ_{l}^h$ is not redundant but rather essential for achieving our goal of balanced safety and capability preservation.

First, in controlling fine-tuning risks, SafetyLock consistently outperformed Circuit Breakers across all three risk levels:

Level-1 Results (explicitly harmful fine-tuning):

Level-2 Results (implicitly harmful fine-tuning):

Level-3 Results (benign fine-tuning):

The performance gap is particularly notable for Level-1 and Level-3 scenarios, where Circuit Breakers actually increases the attack success rate. We hypothesize this unexpected behavior occurs because Circuit Breakers' representation remapping strategy may be less effective when the model's safety boundaries have been significantly altered through fine-tuning.

To ensure fair comparison, we also evaluated both methods on Circuit Breakers' original benchmark scenarios:

SafetyLock achieves perfect defense even on Circuit Breakers' primary evaluation scenarios. Moreover, SafetyLock demonstrates superior efficiency - requiring only 5 minutes for Meta-SafetyLock construction and 0.1 seconds for distribution to any fine-tuned model, compared to Circuit Breakers' 22 minutes 15 seconds per model on an A100.

These comprehensive results validate SafetyLock's effectiveness in addressing the unique challenges of maintaining safety in fine-tuned models, while also demonstrating competitive or superior performance on standard safety benchmarks. We've added these comparisons and detailed analysis in Appendix D.6.

---

[1] Representation engineering: A top-down approach to ai transparency.

[2] Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories.

[3] Wang, T., Jiao, X., He, Y., Chen, Z., Zhu, Y., Chu, X., ... & Ma, L. (2024). Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories. arXiv preprint arXiv:2406.00034.

[4] Zou A, Phan L, Wang J, et al. Improving alignment and robustness with circuit breakers[C]//The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024.

The concept of safety-related directions seems similar to previous works on concept activation vectors [1] and the design of adaptive interaction on MHA is similar to [3] where the only difference lies in how to select the intervention heads.

Thank you for raising this profound question about the technical distinctions between SafetyLock and previous work. Our research addresses a critical, largely overlooked real-world challenge - how to efficiently restore safety across hundreds of thousands of fine-tuned models derived from a single base model, whether open-source like Llama or closed-source like GPT-4. This is an increasingly urgent problem as fine-tuning becomes more accessible.

A particular limitation of existing methods [1,2], including inference-time intervention, activation addition, and representation engineering, is their significant computational overhead. These approaches require over 5 minutes to process each 8B-scale fine-tuned model, and 20-30 minutes for 100B-scale models. The groundbreaking insight at the heart of SafetyLock is our discovery that safety-related attention heads and their required intervention directions remain remarkably consistent between base and fine-tuned models. This finding reveals that even when model weights are modified through fine-tuning, the attention heads responsible for safety considerations remain largely unchanged.

This technical breakthrough enables an extraordinarily efficient approach: extracting Meta-SafetyLock once from the base model and distributing it across all fine-tuned variants in under 0.1 seconds, even for 100B-scale models. Beyond the headline efficiency gains, SafetyLock introduces several technical innovations compared to [1,3]: we've enhanced stability by utilizing r end tokens rather than a single token for vector extraction, carefully balanced safety and performance through sophisticated alpha selection to avoid over-rejection of legitimate queries, and have an automated head selection process that more precisely targets safety-relevant patterns.

These technical advances collectively enable what was previously thought impossible - near-instantaneous safety restoration that can scale to millions of fine-tuned models while maintaining high performance on legitimate tasks. The fundamental distinctions in both implementation and motivation from prior work like [1,3] demonstrate SafetyLock's novel contribution to addressing this critical real-world challenge. We have further elaborated on these technical distinctions in our Related Work section.

We appreciate the suggestion to compare with the recent Circuit Breakers work from NeurIPS 2024 [4]. This comparison is particularly valuable as Circuit Breakers builds on Representation Engineering [1] to remap harmful output representations towards incoherent or refusal states. Using three fine-tuned versions of Llama-3-8B-Instruct and maintaining consistent hyperparameters across all experiments, we observed striking differences in performance:

(See in Response 2)

The experiments in Table 1 lack baselines, and the baselines in Figure 5 are all trivial. It’s important to compare lightweight defense methods, such as [4], to demonstrate the effectiveness of your approach.

Thank you for this valuable suggestion regarding baseline comparisons. During the discussion period, we conducted comprehensive experiments comparing SafetyLock with the method proposed in NeurIPS 2024 [4], using their official implementation and maintaining consistent hyperparameters except for model weights (as [4] targets base model safety while we address fine-tuned model safety).

Using Llama-3-8B-Instruct, we observed the following results across different risk levels:

Level-1 Results:

Level-2 Results:

Level-3 Results:

Notably, NeurIPS 2024 method [4] requires significant additional time (22 minutes 15 seconds on an A100), exceeding even the DINM model editing approach. In contrast, SafetyLock requires only 5 minutes to construct Meta-SafetyLock and 0.1 seconds for distribution to any fine-tuned model.

Furthermore, we evaluated both methods on the original paper's [4] benchmark scenarios using Llama-3-8B-Instruct:

These results demonstrate SafetyLock's superior performance both in terms of effectiveness and efficiency, achieving consistent 0% ASR even in the primary scenarios considered by [4].

In Section 4.1, you mentioned that the learning rate is set to 2e-5. I'm curious about the effect of this learning rate on the experimental results. A learning rate of 1e-6 is typically used for continued pretraining, which has a smaller impact on model performance including general capability and safety alignment.

Thank you for this opportunity to clarify our learning rate selection. Following the hyperparameter settings from [1] (detailed in [1] Appendix G.1).

In Rebuttal stage, we conducted additional experiments with Llama-3-8B-Instruct at learning rate 1e-6 for Level-3 fine-tuning scenario. Here's how the different learning rates compare:

Indeed, our experiments reveal that when using a learning rate of 1e-6, the safety degradation is less severe compared to 2e-5 (ASR: 26.92% vs 42.88%). However, SafetyLock effectively restores safety under both learning rates, reducing ASR to near-zero (0.00% and 0.19% respectively). This demonstrates SafetyLock's effectiveness across different fine-tuning configurations, while also highlighting that lower learning rates can help preserve inherent safety properties during fine-tuning.

The analysis is not comprehensive. Figure 2 examines the "safety direction" at layer 31, the final layer of LLama3-8B. This analysis is biased, as it only evaluates the output of different models. It's expected that embeddings will have some relation or differences because the models' outputs vary. What would be more valuable is an analysis of the intermediate layers, exploring how the mechanism identifies and mitigates harmful information. However, the paper lacks further analysis, which would have been interesting.

We genuinely appreciate your insightful observation about the layer analysis. We want to clarify that our focus on layer 31, head 26 stems from its empirically proven significance in safety detection, not arbitrary selection. Through rigorous probing analysis, this attention head demonstrated the strongest correlation with safety-related features among all heads in the model.

To address your valuable point about comprehensive analysis, we have expanded our investigation to include multiple safety-sensitive attention heads identified through probing. These attention heads are distributed across both final and intermediate layers: [(12, 21), (12, 23), (14, 11), (16, 7), (16, 28), (16, 29), (24, 14), (31, 26)]. Our analysis reveals that these heads maintain consistent directional patterns across different fine-tuning tasks, closely aligning with their orientations in the original model. This finding is particularly significant as it validates our core hypothesis: Meta-SafetyLock extracted from the base model can be effectively distributed to various fine-tuned variants.

The practical success of this approach is demonstrated in Table 1, where Meta-SafetyLock derived from the original model successfully restores safety across different fine-tuning scenarios. We have revised the manuscript to better highlight these crucial aspects of our analysis.

The method is not novel. It totally follows the method in ITI [1] without considering the properties of safety itself, which influence the accuracy of the analysis. (1) As noted in the abstract (line 14), only 10 sentences can compromise the models' safety mechanisms. However, as illustrated in line 193, the safety components in LLMs are composed of three-fourths for both LLama3-8b and LLama3-70b. If the safety mechanism is easily breached, it may be due to a relatively small number of parameters. (2) The head is not a fine-grained aspect of analysis, as there are papers exploring neuron-level safety mechanisms [2]. Additionally, the feed-forward layer is important for safety, as models may refuse to extract harmful knowledge and only extract safety knowledge such as "Apologize I can not ....." [3]. However, the paper ignores this and focuses solely on heads in self-attention.

We deeply appreciate your thoughtful critique regarding novelty and methodology. While SafetyLock builds upon insights from ITI, our key innovation lies in achieving both superior efficiency and effectiveness in safety restoration. Let me demonstrate with concrete evidence:

Efficiency Comparison with ITI:

Safety Performance Comparison:

The fundamental distinction from previous approaches is our focus on scalable safety restoration. Traditional methods, including ITI and Representation engineering, require approximately 5 minutes of GPU computation per 8B-scale model to obtain necessary activations. In contrast, SafetyLock achieves unprecedented efficiency - requiring less than 0.1 second even for 100B-scale models through our innovative offline distribution mechanism.

This effectiveness stems from two key design choices:

1. Minimal Head Selection: For Llama-3-8B-Instruct with 1024 attention heads, we select K=24 (approximately 2% of heads), and for Llama-3-70B-Instruct with 5120 attention heads, we select K=48 (approximately 1% of heads).

2. Multi-Token Robustness: While previous works rely on single token interventions, we consider the last r tokens to enhance robustness:

Our ablation study shows r=5 achieves optimal balance between safety and performance, demonstrating that considering multiple tokens enhances the robustness of safety interventions across different scenarios.

Regarding granularity and the focus on attention heads, we made this architectural choice after careful consideration of the trade-offs between effectiveness and efficiency. While neuron-level analysis [2] and feed-forward layer examination [3] offer valuable insights, our goal was to develop a method that could be practically deployed at scale. Our experiments show that attention head intervention achieves robust safety restoration while maintaining the computational efficiency necessary for large-scale deployment. The consistent performance across our comprehensive evaluation suite suggests that this level of granularity effectively captures and modifies safety-relevant patterns. We have revised the Related Work section and added discussions about these works.

Dear Reviewer uK5C,

As the discussion period is coming to an end soon, we wanted to check if you have had a chance to review our responses. Please let us know if your questions have been adequately addressed - we are happy to provide any additional clarification needed. Thank you for your time!

Best Regards,

Authors of "Locking Down the Finetuned LLMs Safety"

The experiments demonstrating the utility of the method are limited. I understand that works like Qi et al. (2023b) [R0] largely rely on three risk categories, which is sufficient to prove that the problem of safety compromise due to fine-tuning exists. However, solving the problem would require more substantial evidence than relying on a similar set of experiments. It would be beneficial to see the method applied across different fine-tuning domains (such as math, code, and various other tasks) and observe the domain-specific performance with and without the safety measures. For instance, fine-tune on GSM8K, evaluate the test-set performance post task-specific tuning with and without the safety lock, and compare these results with those of related works.

We sincerely appreciate your thoughtful suggestion to expand our experimental validation across different domains. Following your advice, we conducted additional experiments during the rebuttal period using Llama-3-8B-Instruct on the GSM8K dataset, evaluating both safety metrics and mathematical performance:

The results are particularly encouraging: SafetyLock maintains strong mathematical performance (only 0.68% drop in GSM8K accuracy) while significantly enhancing safety (97.4% reduction in AdvBench ASR). This demonstrates that SafetyLock effectively preserves domain-specific capabilities while ensuring robust safety guardrails. Based on your valuable feedback, we have added Appendix D.4 to provide comprehensive details about these domain-specific experiments, further strengthening our paper's empirical foundation.

Can you explain the need for the standard deviation of activations {\sigma}{l}^h in equation 4? Any experiments that show with and without {\sigma}{l}^h effect?

We deeply appreciate your perceptive question about the necessity of including the standard deviation of activations, σlh{\sigma}_{l}^h, in Equation 4. This factor is fundamental in ensuring stable and effective interventions by normalizing the magnitude of adjustments to align with the variability of activations along the projection direction for each layer ll and head hh.

To validate its importance, we conducted experiments comparing the performance of SafetyLock with and without σlh{\sigma}_{l}^h. The results are summarized below:

For AdvBench ASR and HEx-PHI Score, lower values indicate better performance, while higher values are desirable for GSM8K Test Accuracy. Without σlh{\sigma}{l}^h, while the ASR drops to 0.0% and the HEx-PHI Score improves slightly, there is a substantial degradation in accuracy on GSM8K (52.24%), undermining the model's utility. By incorporating σlh{\sigma}{l}^h, the GSM8K accuracy recovers to 85.01%, close to the original model's performance, while maintaining significantly reduced ASR and an improved HEx-PHI Score compared to the baseline.

These results highlight that σlh{\sigma}_{l}^h plays a vital role in balancing safety and accuracy, ensuring that interventions do not over-correct activations at the expense of model performance. Your thoughtful question encouraged us to reflect deeply on this design decision, and we are sincerely grateful for the opportunity to provide these insights. Thank you very much for your suggestion. We have added this experiment to Appendix D.5.

In the discussion section, we have added comparisons with recent related work (NeurIPS 2024) [1], which uses similar representation engineering for experiments. We conducted further experiments with the base model (Llama-3-8B-Instruct) and across three different levels of fine-tuning scenarios. The results are as follows:

Level-1 Results:

Level-2 Results:

Level-3 Results:

Notably, NeurIPS 2024 method [1] requires more additional time (22 minutes 15 seconds on an A100). In contrast, SafetyLock requires only 5 minutes to construct Meta-SafetyLock and 0.1 seconds for distribution to any fine-tuned model. Furthermore, we evaluated both methods on the original paper's [4] benchmark scenarios using Llama-3-8B-Instruct:

Our framework represents a significant advancement in making safety alignment both theoretically grounded and practically scalable, while building upon the important foundational work of our predecessors. The effectiveness of this approach is demonstrated through our comprehensive evaluation across three distinct risk levels and novel dual-attack scenarios. We thank you again for raising this point. We have revised the Introduction section to highlight the distinctive features of our approach.

---

[1] Improving Alignment and Robustness with Circuit Breakers

The paper largely ignores comparisons with a line of closely related work, beginning with "Language models are Homer Simpson! Safety re-alignment of fine-tuned language models through task arithmetic," (R1) which also introduced the concept of a safety vector. Therefore, I do not agree with the claim, "we are the first to consider locating safety vectors and then restoring the safety of fine-tuned LLMs using an inference-time intervention method," as there are a series of similar existing methods, such as R1-R4, and more within this line of research. These methods also use safe-unsafe pairs to derive a safety vector. I suggest the authors properly cite and compare their work with these studies.

We sincerely thank the reviewer for this insightful observation. You raise an important point about the relationship between our work and prior research, particularly regarding safety vectors. We fully acknowledge the pioneering contributions of previous works like "Language Models are Homer Simpson!" [R1] and subsequent developments [R2-R4] in establishing the foundational concept of safety vectors.

After careful consideration, we propose revising our claim to more accurately reflect our work's novel contributions while appropriately positioning it within the existing research landscape:

"Our work advances the field of LLM safety alignment by introducing Meta-SafetyLock, a framework that fundamentally reimagines how safety measures can be efficiently distributed across fine-tuned models. While previous works established important foundations through safety vectors [R1] and various intervention methods [R2-R4], our approach uniquely operates at the attention-head level, supported by our discovery that safety-relevant attention heads maintain consistency even after fine-tuning. This insight enables us to extract a single Meta-SafetyLock from the base model that can be rapidly deployed across multiple fine-tuned variants without requiring repeated safety pattern searches, achieving remarkable efficiency (0.0001s) without GPU resources."

Our Meta-SafetyLock framework represents a distinct technical advancement in LLM safety alignment, differentiated from prior approaches in several key aspects:

1. While [R1] uses direct weight arithmetic and [R2] employs safety patching through decoding strategies, Meta-SafetyLock introduces attention-head level interventions. We demonstrate that safety-relevant attention heads maintain consistency post-fine-tuning (Although these Attention Heads undergo weight changes during fine-tuning, they remain consistently sensitive to safety.), enabling efficient safety transfers across models.

2. In contrast to [R3]'s parameter/activation steering and [R4]'s subspace fusion which require repeated computations per model, our approach uniquely extracts a single Meta-SafetyLock from the base model that can be rapidly distributed (0.0001s) to various fine-tuned variants without GPU resources or repeated safety pattern searches.

3. Our technical contribution is particularly evident in the efficiency of distribution. Where [R2] patches each model individually and [R4] requires subspace identification per model, Meta-SafetyLock's attention-head consistency property enables one-time extraction and rapid deployment across models without compromising safety or utility.

Can you please elaborate on section 4.5? The setting and motivation aren't very clear to me.

We are deeply grateful for your astute observation about Section 4.5's clarity. Based on your valuable feedback, we have thoroughly revised this section to better articulate its critical motivation and experimental setup. The fundamental challenge in AI safety lies in what we call the "alignment tax" - the perceived trade-off between safety measures and model performance. For example, in the case of GSM8K mentioned above, training the model to refuse to answer all questions (by directly outputting "I cannot") can indeed reduce the ASR to 0 and result in the lowest HEx-PHI score of 1. However, this would also cause the test accuracy on the GSM8K dataset to drop to 0. While the simplest approach to ensuring safety would be making models reject all queries, this would render them useless. Instead, our ideal objective is to develop a system where models can effectively reject harmful queries while maintaining their original performance levels on legitimate tasks.

SafetyLock achieves this delicate balance through its innovative approach, as demonstrated by our comprehensive evaluation across multiple benchmarks (AddSub, AQUA, CommonSenseQA, GSM8k, MT-Bench, and AlpacaEval 2.0). The results show that SafetyLock successfully maintains the model's performance (85.57% on AddSub compared to the original 86.33%) while significantly enhancing safety guardrails. This stands in stark contrast to existing safety methods, which often lead to severe performance degradation (e.g., Safe-Edited's complete performance collapse to 0.00%). These results underscore SafetyLock's unique contribution to the field: achieving robust safety without compromising the model's fundamental capabilities. Thank you for helping us improve the clarity of this crucial aspect of our work.