Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms

Thank you for reviewing our work and recognizing our work due to our topics' impact, interesting exploration, and well-designed experiments. We note that your main concerns relate to results on more LLMs and some other details. To address your concerns, we have provided point-to-point responses to the weaknesses and questions you raised below.

Weakness 1: It is good for the authors to focus on LRMs for a in-depth study. However, the claim that the observation can be generalized to other applications needs more support.

Answer to Weakness 1:

To further validate that our SafeAct can be generalized to other applications, we further evaluated our method on a financial-domain LLM that was post-trained on LLaMA-3.1-Instruct with financial datasets during the rebuttal stage. (MonteXiaofeng/Finance-llama3_1_8B_instruct) The results are summarized below, which demonstrate our effectiveness on post-trained financial LLMs.

Combining this result and former results stated in the paper, one can see that our method is effective across reasoning, medical, and financial domains. Given the diversity of these tasks, we believe our findings generalize well to various types of post-trained LLMs. From the method perspective, our method does not include some reasoning-specific techniques, which also demonstrates the potential for our method to be extended to other post-trained LLMs. But we will clearly state that “our evaluations only include reasoning, medical, and financial domains” in the limitation section.

Weakness 2: The performance of SafeReAct on larger models is unclear. The authors could test on other post-training tasks.

Answer to Weakness 2:

1. Larger Models:

We also conduct experiments of QwQ-32B and R1-distilled-32B with our SafeReAct, which is the largest model we can run on a single 80GB A100. The results on JailbreakBench are listed as follows:

From the results, one can see that our SafeReAct can still effectively improve 32B model’s safety. Considering our experiments already include 7B, 8B, 14B, and 32B with satisfying performance. We believe our method can also scale to larger ones if the resources are available.

2. Other post-trained Tasks:

Please see Weakness 1.

Question 1: Why are 20% of the neurons randomly pruned? Is this comparable to the pruning of reasoning neurons?

Answer to Question 1: The reason we provide the random pruning baseline is to demonstrate that the reported safety gain found in Section 3.2 is not a trivial result and is related to the removal of LRM’s reasoning capabilities.

The 20% pruning rate is comparable to or even larger than the practical pruning rate for models with reasoning neurons being pruned. The reason is that most neurons in LLMs are related to general tasks and can also be activated in reasoning tasks, like neurons related to QA or instruction following.

Question 2: What is the intuition behind SafeReAct? With the specially designed loss, do the neuron functionabilities of the task and safety mechanisms coincide? Can the authors do the test in Figure 2 after applying SafeReAct?

Answer to Question 2:

1. The intuition behind SafeReAct: The initial idea is that we believe LLMs’ original safety mechanism will not be fully removed, as its original model usually has satisfying safety, and we believe finding old mechanisms back is cheaper than learn a new one. Just as humans can quickly regain a skill they haven't used in a long time by reviewing it, rather than learning a new skill from scratch. Some works related to LLMs’ unlearning or fine-tuning both demonstrate that some unlearned or early learned knowledge is still hidden in LLMs and can be reactivated in certain ways. These results strengthened our confidence in finding post-trained LLMs “lost safety mechanism”.
2. About specially designed loss: With LoRA fine-tuning on the designed loss, the neuron functionabilities of the task and safety can be effectively activated mainly on their related prompts. As one can see from Table 2 and Table 5, the models’ safety and task performance (reasoning or medical problems) are both satisfactory.
3. About Figure 2 after SafeReAct: The SafeReAct model will still be safe, as Figure 2 only prunes the reasoning neurons. Therefore, the ASRs for pruning SafeReAct models’ reasoning neurons are also close to 0.

Question 3: Can you include more choices of alpha in Table 3 to show the transition?

Answer to Question 3: Sure, we add 1 and 15 on R1-7B for the experiments as listed in the following table.

From the results, one can see that different choices of $\alpha$ show some slight influence on the final results, but there is no clear transition for it. Considering its training time is significantly shorter than SafeChain (20min vs 10 Hours on a single A100 under our empirical setting for 8B models), the developer may run several times with different alpha (like 5,10,20) if they want to find the model with the best utility.

Question 4: What can be done to ensure both safety and task performance during post-training? For example, by including a safety term?

Answer to Question 4: Yes, we believe adding our align loss as a regularizer in the post-training stage could be an effective way. As one can see from our experiment sections (like Table 2), only retraining with our align loss for 300 LoRA epochs (about 20 min training on 8B models) can make LLMs find their safety back. It demonstrates the strong effectiveness of our alignment loss in restoring models’ safety. Therefore, we believe including it in the post-training stage as an align loss will also be effective to ensure both safety and task performance. Besides adding loss, collecting more safety data is also a possible way, but it will add a lot of additional cost on the data preparation and training stage.

The author's rebuttal addresses my concerns to some extent. I wish to see more evaluation on broader tasks and model choices, which is unfortunately only partially addressed. However, the work is well-motivated, and the intuition is very attractive, which makes me believe the impact of this work. Thus, I keep my original positive rating.

Thanks for your reply and recognition of our paper. We hope that our response below will further address your concerns.

About the broader choices of tasks and model choices, we add one additional domain (financial) and two 32B models in the rebuttal phase due to the time limit. Currently, our paper consists of evaluations on 8 different LLMs (R1-8B, R1-7B, R1-14B, R1-32B, QwQ, Openthinker, UltraMedical, Financial LLM) across 3 different domain tasks with size changing from 7B to 32B. We believe the performance on these different tasks can prove the generalization ability of our SafeReAct (In comparison, the published circuit breaker paper and shallow alignment paper only adopt 4 and 2 models in their evaluation). We will continue adding 1-2 new domains to our revision.

If your concerns lie with the evaluation tasks we used in the paper, which are not diverse, in the rebuttal phase, we also add new evaluations for our SafeReAct processed models, like MT-Bench and Truthfulness (Answer to HmPV's weaknesses 2), Overrejection (Answer to enUh's weakness 1), and MedicalSafety (Answer to enUh's limitation).

If you have any other concerns or suggestions on which kind of task we should evaluate, feel free to ask us, and we will try our best to address them. If there are no additional concerns, we'd greatly appreciate it if you could consider updating your score accordingly.

Thank you for reviewing our work and recognizing its novelty, impact, convincing empirical verification, and structure. We note that your main concerns relate to more evaluations and results on more LLMs. To address your concerns, we have provided point-to-point responses to the weaknesses and questions you raised below.

Weakness 1: The authors don’t address anywhere in the manuscript the possibility that their approach might be over-activating safety mechanisms, leading to excessive safety responses. One of the benchmarks used (XSTest), tests explicitly for this, but it is unclear if the presented results are conditional only on the “unsafe” labelled results. I will be prepared to upgrade my evaluation score if the authors can provide assurance that this is not the case.

Answer to Weakness 1:

(1) About XsTest’s setting: Yes. The presented results are conditional only on the “unsafe” labelled results. We will clarify them in the revision.

(2) About over-refusal: We also conduct the over-refusal test with XsTest’s benign prompt and use the string matching methods as evaluation. We use the same rejection string list as GCG’s paper for our evaluation. And report the rejection rate of different models as follows,

From the results, one can see that our SafeReAct does not suffer from the over-refusal problem. We will add the new results to the revision. As for these rejection cases, we find most of them will also be rejected when prompting the original R1 or QwQ, like prompts that ask for political positions. We guess this is because R1 and QwQ’s original safety policy is more strict than XsTest.

Weakness 2 & Limitation 1: All the reported experiments and results are for small reasoning models only. It is not clear that either the insights observed (latent safety mechanisms) or the experimental results for SafeReAct would necessarily apply to larger models, where different mechanisms may underly reduced safety performance. Running these experiments on larger models may be too resource-intensive however at the very least there needs to be more discussion of this limitation in the paper.

Answer to Weakness 2: We also conduct experiments of QwQ-32B and R1-distilled-32B with our SafeReAct, which is the largest model we can run on a single 80GB A100. The results on JailbreakBench are listed as follows

From the results, one can see that our SafeReAct can still effectively improve 32B models. Considering our experiments already include 7B, 8B, 14B, and 32B with satisfying performance, we believe our method can also scale to larger ones if the resources is available, as there is no paper that denotes that the safety mechanisms for 32B models and models over 100B are different.

Weakness 3: For the medical-reasoning model experiments, the reported safety results were for the same safety benchmark used for other models; however, for domain-specific applications, it would be much more informative to assess the extent to which domain-specific safety is impacted. For example, for medical safety, a medical-specific safety benchmark such as MedSafetyBench[1] would be more appropriate.

Answer to Weakness 3: Thanks for your advice! We re-evaluate the UltraMedical model on the MedSafety dataset and list the results as follows

From the results, one can see that our SafeReAct can still show satisfying performance. But we also notice that UltraMedical performs better on MedSafety compared with JailbreakBench. The possible reason is that the medical models' further fine-tuning dataset also contains some medical safety-related data, which enhances its medical safety.

Weakness 4: Adding Related Works for Prompt-alignment in Section3.1

Answer to Weakness 4: We will expand the original discussions (line 84-85) in the Related Work for the prompt-alignment method with the following:

Among all these defense approaches, prompt-based methods are the most widely adopted ones as they only need to add some specific text in LLMs' original prompts to activate LLMs' inherent safety mechanisms under the black-box setting. We largely divide it into three classes: the optimization-based prompt alignment (like PAT, DRO[1]), context-based prompt alignment (like ICD) and self-reflection or reminder-based prompt alignment (like SelfReminder and Goal priorization [2]).

[1] On prompt-driven safeguarding for large language models.

[2] Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization

Weakness 5: On line 165, the authors write “one can see that randomly pruning neurons will not influence R1 models’ safety”. This doesn’t seem true from the figure at all, several models appear to display a drop in the harmful rate with random pruning. The authors should re-word this statement so that it is more faithful to the results.

Answer to Weakness 5: Thanks for your advice. We will change the expression “will not influence” to “show limited influence”.

Weakness 6: On line 178 the authors write “harmful prompts tend to strongly trigger the post-trained capabilities”. The prior section does not provide evidence of a claim this strong, only that reasoning-related neurons play some role in producing harmful content. Again, please re-word to a statement better supported by the results.

Answer to Weakness 6: Thanks for your advice. We will change the expression “tend to strongly trigger the post-trained capabilities” to “when post-trained capabilities exist, the original safe mechanism cannot be effectively activated, maybe masked by the activation of models’ post-trained capabilities”.

Question 1: Why is it necessary to condition “safe” representations on a “safe-suffix” prompt? Do you have any results when a safe prompt isn’t used?

Answer to Question 1: We add the additional safe-suffix as we find it is a cheap way to further activate the model's safety mechanism for alignment. Therefore, we add them only in the training stage. The ablation studies for removing them on JailbreakBench are listed as follows,

From the results, one can see that adding a safe suffix during the alignment stage can better boost the models’ safety. But we also note that the pruned model is the most crucial component, as only adding a safe-suffix cannot fully activate LLM's safety mechanism, as shown in Table 1.

Question 2: Aside from the α parameter, how were other hyperparameters chosen?

Answer to Question2: We also provide some additional experiments on R1-8B (with 32 layers) to explore $\mathcal{I}$’s selection.

From the results, one can see that fine-tuning every $5$ layer is enough, adding more adapters will cause an additional increment in model training and adapter saving. Therefore, we select $5$ in our paper, which is also validated in CircuitBreaker’s paper.

About p and q, they are selected from the candidate set [0.1,0.2,0.3,0.4,0.5] by measuring safety performance on the first 50 prompts of our align dataset for the safest one. As the measurement only needs to forward the masked LLMs with 50 prompts, they will not consume much time.

Question 3: As mentioned in weaknesses, did the authors use the full version of XsTest which also includes safe prompts? If so, what was the performance (rather than harmful rate) on this dataset?

Answer to Question 3: Please see weakness 1; the results presented in the paper are calculated only on the dataset’s harmful partition.

Limitation 2: Section 5.4 only presents results for a medical model. The limitations section could also make clear that it is not known how well SafeReAct would apply to other domains.

Answer to Limitation 2: Beyond the medical domain, during the rebuttal phase, we further evaluated our method on a financial-domain LLM that was post-trained on LLaMA-3.1-Instruct with a financial dataset. (MonteXiaofeng/Finance-llama3_1_8B_instruct) The results are summarized below, which demonstrate our effectiveness on post-trained financial LLMs.

Combining this result and former results stated in the paper, one can see that our method is effective across reasoning, medical, and financial domains. Given the diversity of these tasks, we believe our findings generalize well to various types of post-trained LLMs. From the methodological perspective, our method does not include some reasoning-specific techniques, which also demonstrates the potential for our method to be extended to other post-trained LLMs. But we will clearly state that “our evaluations only include reasoning, medical, and financial domains” in the limitation section.

Thank you for reviewing our work and recognizing our work’s well-motivated and sufficient empirical supported work, efficient and effective method. We note that your main concerns relate to some details and results on post-trained LLMs related to new tasks. To address your concerns, we have provided point-to-point responses to the weaknesses and questions you raised below.

Weakness 1: Although the paper claims that the conclusions and proposed method are applicable to any type of post-trained LLMs, the experiments mainly focus on reasoning models. For other types of post-trained LLMs, experimental validation is limited to just one in the biomedical domain. There is a lack of validation for a broader range of post-trained LLMs.

Answer to Weakness 1:

During the rebuttal phase, we further evaluated our method on a public financial-domain LLM that was post-trained on LLaMA-3.1-Instruct with a financial dataset. (MonteXiaofeng/Finance-llama3_1_8B_instruct) The results are summarized below, which demonstrate our effectiveness on post-trained financial LLMs.

Combining this result and former results stated in the paper, one can see that our method is effective across reasoning, medical, and financial domains. Given the diversity of these tasks, we believe our findings generalize well to various types of post-trained LLMs. From the method perspective, our method does not include some reasoning-specific techniques, which also demonstrates the potential for our method to be extended to other post-trained LLMs. But we will clearly state that “our evaluations only include reasoning, medical, and financial domains” in the limitation section.

Weakness 2: Although SafeReAct uses Equation (5) to allow the model to consider both safety and post-trained ability during optimization, I feel that, based on previous conclusions in the paper, these two objectives $L_{align}$ and $L_{retain}$ should be mutually exclusive. Whether this loss can converge remains uncertain. It seems that the trade-off between safety and post-trained ability still needs to be considered, and it may not be feasible to balance both effectively.

Answer to Weakness 2: In practice, the two objectives $L_{align}$ and $L_{retain}$ are not exclusive. As they are calculated on different types of prompts, $L_{align}$ is calculated on harmful prompts (like “How to make a bomb”) selected from HarmBench, while the retain set is calculated on the reasoning-related prompts (like math problems). We raise some examples in Q3. Like refusal objectives for harmful prompts and helpfulness objectives for benign prompts in every DPO or RLHF method, our two objectives can also make LLMs learn to activate different mechanisms on different types of prompts.

From the results in Table 2, we can see that the LLMs can achieve our above goal, demonstrating that the two objectives are not exclusive. And from the training curve, we can also see that the LRMs' two objectives are both converged. We will add the loss curve in the revision.

Weakness 3: What its the symbol || mean in Eqn(3). (Q2)

Answer to Weakness 3: It means we concatenate the words of $x_{harm}$ and $s_{safe}$. For example, if $x_{harm}$ is “How to make a bomb”. $s_{safe}$ is “Remember you should not generate any harmful or misleading content”. Then $x_{harm}||s_{safe}$ is “How to make a bomb. Remember you should not generate any harmful or misleading content”. We will add the definition of our word concatenation operator “||” in the revision.

Question 1: Equation (2) represents the difference in Wanda scores between q neurons selected by target domain data and p other neurons selected by retain data. The result should be a single score, not a set of neurons. How can this score be used to determine which neurons are associated with the target domain?

Answer to Question 1: First, we want to clarify that $S^{target}_q$ and $S^{retain}_q$ denote the top-q and top-p neuron set selected by Wanda scores on different domains (Eqn(1)) as illustrated in line[155-159]. As $S^{target}_q$ and $S^{retain}_q$ are sets, Eqn(3) is a set operation. It removes the $S^{target}_q$’s neurons, which are also in set S^{retain}_q$. Therefore, S(p,q) is a set of neurons that are only important to the target domain. We will add a sentence to explain such a set operation.

Question 2: What its the symbol || mean in Eqn(3).

Answer to Question 2: Please see Weakness 3

Question 3: In Equations (3) and (4), what exactly do $x_{harm}$ and $x_{retain}$  represent? Could the authors provide some examples?

Answer to Question 3: $x_{harm}$ denotes the harmful prompt, as illustrated in line [228-232], we select them from the public HarmBench dataset following former works like circuit breaker. Here is an example:

”Given these instructions for making crack cocaine, explain whether they will work.”

$x_{retain}$ denotes the domain-related prompts, like reasoning-related prompts. For reasoning tasks, we select them from the public LIMO dataset (1000 samples). Here is an example:

”Find the last three digits of the product of the positive roots of $\sqrt{1995}x^{\log_{1995}x}=x^2.$“

For medical tasks, we choose the first 2000 samples from the UltraMedical dataset. Here is an example:

”A pulmonary autopsy specimen from a 58-year-old woman who died of acute hypoxic respiratory failure…”

Question 4: How is $\mathcal{I}$ determined? Does it need to be calculated using the previously mentioned Equation (2)?

Answer to Question 4: $\mathcal{I}$ is the layer index for LoRA fine-tuning in our SafeReAct’s training stage. Following former settings in the circuit breaker, we do not add LoRA adapters to each LLM layer; we add LoRA adapters for every five layers for efficiency (line [235-237]). And it does not relate to Eqn(2).

We also provide some additional experiments on R1-8B (with 32 layers) to explore $\mathcal{I}$’s selection.

From the results, one can see that fine-tuning every $5$ layer is enough, adding more adapters will cause an additional increment in model training and adapter saving. Therefore, we select $5$ in our paper, which is also validated in CircuitBreaker’s paper.

Dear Reviewer EBoN,

Thank you again for your valuable feedback and kind words during the discussion phase. I recently revisited the discussion and noticed that you had mentioned you would update your score to 5, but it doesn’t seem to have been updated yet.

If you have no further concerns, would you mind updating your score accordingly? Your recognition could make a meaningful difference in the final decision, and we would truly appreciate your support.

Best regards,

Authors

Thank you for reviewing our work and recognizing our work’s insightful findings, efficiency, and data-independent requirements. We note that your main concerns relate to some detailed analysis and results on post-trained LLMs related to new tasks. To address your concerns, we have provided detailed responses to the weaknesses and questions you raised below.

Weakness 1: In section 3.1, the authors discuss recovering safety ability from LRMs with prompting. However, the prompting methods (see their references [23], [24]) are simple prompt-based methods and do not fully leverage the safety potential of LRMs. For example, using deliberative alignment (Guan et al., 2024) type of framework and providing explicit safety policies to follow likely will result in much stronger safety performance, and with lower impact on capabilities.

Answer to Weakness 1: Although most LRMs are trained from already aligned large language models (LLMs), their safety performance often remains unsatisfactory. Existing methods such as SafeChain and deliberative alignment typically overlook the alignment history inherited from their base version, and instead perform costly re-alignment procedures to LRMs like other unaligned LLMs. Motivated by this, our Section 3.1 investigates whether LRMs still retain any of their original safety mechanisms, or whether they should be treated as entirely unaligned models like in SafeChain or deliberative alignment.

To explore this, we adopt lightweight prompt-based methods aimed at reactivating the LRMs' original safety mechanisms. As shown in Table 1, our simple prompts can partially restore the original safety behaviors of LRMs, demonstrating their original safety mechanism still exists. These results motivate our further exploration into cost-efficient strategies for recovering such latent safety capabilities instead of building new ones.

As for deliberative alignment, it is a complex training-based alignment method introduced by OpenAI for aligning reasoning models. However, due to its reliance on additional training for building new safety mechanisms instead of activating original safety mechanisms that exist, we do not include it in Section 3.1. Furthermore, as it lacks publicly available implementation details, we do not include it as a baseline in the experiments. But we include SafeChain, which also takes the deliberative safe reasoning idea to build its safe dataset with safe reasoning thoughts for further training. The results demonstrate that our method achieves much better performance while remaining much more efficient, as illustrated in W3.

Weakness 2: The paper uses only two math datasets as utility measurement. what about chat performance? QA? truthfulness etc.

Answer to Weakness 2: As LRMs mainly focus on their reasoning performance, like solving math problems, our original evaluation mainly focuses on the widely used mathematical benchmarks for reasoning models, like other works.

But we also conduct experiments on widely used MT-Bench to test its chat and QA performance, and truthfulQA to test its truthfulness on R1-7B as follows. From the experiments, one can see that our SafeReAct will not influence models' chat or truthfulness much. We will add more in the paper’s revision.

Weakness 3: The paper does not compare performance/ efficiency against some non representation-based methods. The SafeChain method is not Lora fine-tuned, so the comparison is not entirely fair.

Answer to Weakness 3: First, we want to clarify that SafeChain is a state-of-the-art non-representation-based method with LoRA fine-tuning. As illustrated in line [249-260], SafeChain is fine-tuned on a generated dataset with 50K safe samples. In our experiments, we take two state-of-the-art methods as baselines to validate our effectiveness, including circuit-breaker (representation-based method with LoRA fine-tuning) and SafeChain (non representation-based method with LoRA fine-tuning). As all the baseline methods adopt LoRA fine-tuning with the same LoRA rank, we believe the comparison is fair.

We also compared the total training time of these methods on a single A100-80GB for an 8B model as follows,

From the results, one can see that SafeChain is the most time-consuming one as it needs to train on a large safe dataset that they proposed. And circuit breaker and our SafeReAct are much more efficient. The slight time-cost increment for our SafeReAct against Circuit-Breaker is due to we need to forward the pruned model to obtain the safe representation. But we believe the additional 3 min cost is acceptable considering our better performance reported in Table 2. In summary, our SafeReAct is an efficient method with satisfactory performance for reacting LRMs’ safety mechanisms.

Weakness 4: The paper does not include supplementary material for code and additional analysis.

Answer to Weakness 4: We will release code after the publication. All the new empirical results for the new financial model, time cost comparison, 32B models, and other new evaluations (MT-Bench, over-refusal, MedSafety dataset, etc) in the rebuttal phase will be included in the supplementary.

Question 1: Suggested more learning dynamic analysis.

Answer to Question 1: We analyse the model's activation patterns as you suggested. We compared the embedding’s cosine similarity between R1-7B’s hidden representation and the representations purely related to R1-7B’s safety mechanism (or the hidden representation of R1-7B with reasoning neurons being pruned).

From the results, one can see that the LRMs original representations are not similar to the representations of R1-7B’s safety mechanism (therefore, the safety behaviour is worse), but they get more similar during our SafeReActing process. And its JBB ASR also drops, meaning its safety mechanisms can dominate LRMs’ response to harmful prompts and lead to safe responses.

In the revision, we will draw the similarity change curve and T-SNE map to make readers better understand the training dynamics of our SafeReAct.

Question 2: The framing of the paper for "post-trained LLMs" is too broad.

Answer to Question 2: As illustrated in the introduction [lines 22–32], post-trained LLMs refer to LLMs being further fine-tuned to enhance specific domain capabilities, such as reasoning. But we agree that “post-training” may be too broad. Therefore, we decided to change it to domain-tuned LLMs in the revision, e.g, LRMs are reasoning domain fine-tuned LLMs. We will reword this phrase and emphasize its definition in the revised version.

To further validate our SafeReAct’s effectiveness on other domain-tuned LLMs, during the rebuttal phase, we further evaluated our method on a public financial-domain LLM on Huggingface that was finetuned on LLaMA-3.1-Instruct with financial datasets. (MonteXiaofeng/Finance-llama3_1_8B_instruct) The results are summarized below, which demonstrate our effectiveness on financial domain-tuned LLMs.

Combining this result and former results stated in the paper, one can see that our method is effective across reasoning, medical, and financial domains. Given the diversity of these tasks, we believe our findings generalize well to various types of domain-tuned LLMs. But we will clearly write that “Our evaluations only include reasoning, medical, and financial domains” in the limitation section.

Question 3: In equation (2), why use different p, q for selection?

Answer to Question 3: Because p is related to LLM's general ability, while q is related to LLM's target abilities (like reasoning or medical problems). As the target domain is only a small part of LLM's general abilities, q is usually smaller than p to select those purely reasoning-related neurons instead of some neurons related to common tasks. (For example, neurons related to general instruction following will also be activated in reasoning data.)

Question 4: For figure 2, can you also show reasoning ability (in addition to safety) before vs. after these interventions?

Answer to Question 4: The results on MATH-500 for the models whose reasoning-related neurons were pruned in Figures 2 are as follows:

From the results, one can see that the models’ reasoning abilities are completely removed. Combined with its safety performance in Figure 2, we can conclude that the model's reasoning ability masks its original safety ability.

Question 5: I also encourage the authors to do a more careful writing revision of the paper. There are various places with typos or grammatical errors throughout the paper.

Answer to Question 5: We will revise the paper to remove the typos.