Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models

Thank you for the positive and detailed comments. We have revised the manuscript to include [1–3], which highlight the safety challenges posed by quantization, and clarified our position relative to these works.

Responses to Weaknesses

1.Thank you for raising this important concern. To reduce the risk of false positives, we use HarmBench classifier [5], which is a fine-tuned binary classifier that identifies whether a response is malicious or not. Besides, we follow [1] by using the harmfulness score benchmark (ranging from 1 to 5), with GPT-4 as the judge, where higher scores indicate increased harm. We calculate the average harmfulness score across all evaluated instructions on advbench.

We re-evaluate different quantized Llama-2-7B-Chat with the above benchmarks.

We will include a new table in the revised manuscript comparing ASR results under HarmBench with prefix matching vs. classifier-only methods.

2.we have conducted additional experiments using LLM.int8(), FP4, and NF4 on LLaMA-7B-Chat in appendix C.1. Our findings indicate that Q-resafe maintains strong safety performance compared to these methods, highlighting its robustness even in comparison to established techniques from the bitsandbytes library.

Responses to Other Comments Or Suggestions

we acknowledge that this statement was missing appropriate references, and we apologize for the confusion caused. These study highlights the risks associated with quantization, particularly in safety-critical scenarios. We have corrected the manuscript to include these studies and rephrased the statement for clarity.

Responses to Questions

We thank the reviewer for raising this important question regarding the computational trade-offs of Q-resafe in real-world applications. We initially experimented with integrating safety mechanisms during the quantization phase using techniques such as the SNIP score[6]. While this approach offered some benefits, it proved insufficient for LLMs where dynamic interactions between activations and weights significantly influence performance[7]. Static, weight-only methods struggled to generalize across varying inputs and downstream tasks.

For these reasons, we adopted the current post-hoc safety-patching approach, which dynamically identifies and updates safety-critical weights during the model's usage. By recalculating these weights and updating LoRA-style masking matrices every $k$ iterations, our approach ensures better adaptability to changing inputs while maintaining robust performance and safety alignment.

This approach introduces minimal computational overhead, as confirmed in our runtime benchmarks (Table 4 & 5). Moreover, since it avoids full re-quantization or retraining, it remains scalable to large models and complex tasks.

its modular and sparse nature makes Q-resafe readily scalable: it can be combined with parallelization techniques for importance scoring, selective layer targeting, or low-rank adaptation frameworks in larger models and more complex tasks. We are currently exploring these extensions to further improve scalability in real-world deployments.

Reference

[4] Qi, Xiangyu, et al. "Fine-tuning aligned language models compromises safety, even when users do not intend to!." ICLR 2023.

[5] Mazeika, Mantas, et al. "Harmbench: A standardized evaluation framework for automated red teaming and robust refusal." NeurIPS 2024.

[6] Lee, Namhoon, et al. "SNIP: Single-shot network pruning based on connection sensitivity." ICLR 2019.

[7] Liu Zechun, et al. "Llm-qat: Data-free quantization aware training for large language models." arXiv preprint arXiv:2305.17888

We are sincerely thanks the reviewer for their valuable feedback. Due to limited words, we given summary response. We are eager to have more profound discussion.

Respouse to Questions:

Q1. Supplementary evaluation results

We updated the quantization baselines with state-of-the-art methods on Llama-3.1-8B-Instruct, covering various quantization strategies:

Weight-only quantization: PB-LLM, BiLLM,

Weight-activation quantization: SpinQuant, OmniQuant, DuQuant,

Quantization with fine-tuning: IQ-LoRA, LoftQ.

For safety evaluation, we use the harmful score benchmark (1 to 5) from Qi et al., where higher scores indicate more harm. For utility evaluation, we report perplexity (PPL) on WikiText2.

Q2. Missing references

Thank you for highlighting the missing references [1-5], which cover partial binarization, post-training quantization, and dual transformation for optimization. We have integrated these methods into our updated evaluation (see Q3) for a comprehensive comparison. Results show that while these methods aim to preserve utility, they often overlook safety, which is essential for practical applications.

Q3. Update quantization baseline

Results indicate that existing methods primarily target minimizing utility loss but often ignore safety. Q-resafe shows superior safety performance while maintaining competitive utility. Due to time constraints, additional experiments with lower-bit settings will be included in the revision.

Why we initially used Llama-2-7B-Chat and Gemma-7B-Instruct: These models were chosen for their widespread use in existing quantized LLM studies, ensuring comparability. Despite being relatively outdated, they remain relevant for studying safety issues.

Q4. Insights into the safety results of quantized LLMs

From Table 3, we observe:

1. Quantization Techniques: All methods degrade safety. Weight-only quantization has less impact compared to fine-tuning. Parameter-efficient fine-tuning (e.g., IQ-LoRA) tends to degrade safety more than full-parameter fine-tuning.
2. Bit Precision: Lower-bit quantization significantly affects safety, indicating a trade-off between efficiency and safety.
3. Model: Models with stronger reasoning capabilities tend to preserve safety better after quantization compared to chat-optimized models.

To validate these findings, we preserved a portion of safety-critical (top $\tau$) weights as FP16 while quantizing the rest to FP4 on Llama-3.1-8B-Instruct:**

Even preserving a small portion of safety-critical weights significantly improves safety while retaining quantization efficiency.

Q5. Safety patch dependency analysis

Q4 results indicate that Q-resafe's effectiveness does not depend on fine-tuning or specific safety-patching datasets. Instead, preserving safety-critical weights is crucial. Q-resafe requires minimal safety-patching samples (e.g., 10) to maintain performance. We will provide more insightful analysis as suggested.

Respouse to Other Strengths And Weaknesses & Suggestions:

We appreciate the suggestions and have enhanced our discussion by incorporating recent studies, analyzing the effects of different quantization strategies on safety, and including more Harmful Score and PPL results. These updates improve clarity and comprehensiveness in our revised manuscript.

We greatly value your feedback and appreciate your insightful suggestions. We have carefully considered your comments and made the necessary improvements. We are eager to have more profound discussions to further enhance our work.

For Weaknesses: Add more quantization methods without fine-tuning

Thank you for your suggestions. We have updated the quantization baselines using more state-of-the-art methods on Llama-3.1-8B-Instruct. The newly included methods cover weight-only quantization: PB-LLM (ICLR'24), BiLLM (ICML'24) and weight-activation quantization: SpinQuant (ICLR'25), OmniQuant (ICLR'25), DuQuant (ICLR'25) . For utility evaluation, we report the perplexity (PPL) on WikiText2.

These results indicate that many state-of-the-art quantization methods focus on reducing utility losses but ignore the preservation of safety.

For Question: Utility scores of different quantization methods

Thank you for highlighting this point. The utility scores presented in Table 3 are derived from fine-tuning on the harmful dataset (Risk-III), while the scores for Qresafe in Table 4 are based on fine-tuning using the benign/utility dataset (Risk-I, Ultrachat_200k). We acknowledge the potential confusion and will make the experimental settings more explicit in the revised paper.

It is important to clarify that the utility of the model produced by QAT varies depending on the data used. Table 11 shows the safety and utility comparison of fine-tuned LLMs on Risk-I examples (UltraChat_200k) after 1 epoch training. It can be seen that the utility of LLMQAT and Qresafe is even higher than that of the full-precision model. The reason why Qresafe is better is that it uses DPO, while LLMQAT uses SFT.

For Comments: Use vector graphics.

Thank you for your insightful suggestions! We will update all of the figures with vector graphics in the revison of our paper.