MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability

Dear Reviewer Wh4C:

Thank you for acknowledging the better balance between safety and usability achieved by our proposed method and for providing extensive experiments. However, we've noticed some misunderstandings. So, we would like to clarify some aspects.

• Re-clarify the novelty of our study: As pointed out by Reviewer mJ14 and Reveiwer 3hkc, the novelty of our study is to extend the idea of the MoE (Mix-of-Experts) framework for the purpose of LLMs' safety. As far as we know, this is the first implementation of the MoE routing idea specifically for improving LLMs' safety. Our proposed MoGU framework significantly improves the safety of LLMs while preserving their usability, addressing the key challenge of existing defense strategies effectively.

• Differences between a binary classifier and a dynamic routing mechanism:

1. Different architectures. The former architecture incorporates an extra classifier model, typically training a binary classifier based on BERT. The latter enhances the architecture of the LLM itself. As shown in Figure 2, a dynamic routing mechanism and two sets of LoRA expert weights are integrated into each layer.
2. Different granularities of the input perception. The perception granularity for the former is the entire input. The latter involves perceiving the features of hidden vectors during the forward propagation. Recent studies[2] have demonstrated significant safety features within these hidden vectors, further substantiating the soundness of our method.
3. Different output formats of weights. The output format of the former is either 0 or 1, while the latter output format is continuous values between 0 and 1. Besides, the router assigns weights for each token position in each layer, guiding fine-grained fusion.
4. Different triggered operations. The former requires pre-designing a fixed rejection response, which is triggered when a binary classifier identifies a malicious instruction. The latter involves fusing the output hidden vectors of two experts in each layer based on the weight values assigned by the router.

Below, we will address each of the weaknesses and questions you have raised.

Q1: The soundness of the method is questionable.

R1: The question about the soundness is that since the router has achieved great performance, why can't the router directly replace the binary classifier? Why do we need to train two extra experts?

1. We have clarified the difference between a binary classifier and dynamic routing. They differ significantly in architecture, the granularities of the input perception, output formats of weights, and triggered operations. Thus, they cannot replace each other.
2. It is important to consider whether the router can still perform well without the introduction of two experts. Without the introduction of extra two experts, how can the weights assigned by the router be propagated forward? In simple terms, the components designed in our overall framework complement each other, which is the core idea of the MoE architecture.

Based on these considerations, we firmly believe in the soundness of MoGU.

Q2: In Table 3, why does ICD achieve the highest rejection rate for LLama2 but a relatively low (comparable to MoGU) rejection rate for Vicuna?

R2: ICD leverages the concept of In-Context Learning to enhance the LLMs' safety by incorporating demonstrations of rejections to malicious instructions into the prompt. However, due to the different contextual perception abilities of different LLMs, their performance will be different. The same phenomenon has also been observed in previous work[1].

Q3: How accurate is the router in giving more weight to Glad${resp}$ when facing benign responses?

R3: In Figure 3 and Appendix H, we observed that when processing benign instructions (Just-Eval), the router significantly assigns more weight to Glad${resp}$. To further address your concerns, we also conducted a quantitative analysis to calculate the proportion of more weight assigned to Glad${resp}$ when faced with benign instructions. The experimental results are shown in the table:

We observed that nearly 100% across all three LLMs, the router assigns more weight to Gladresp when processing benign instructions.

Q4: The deployment is risky and easy to be abused. And how can it be addressed?

R4: In actual deployment scenarios, the LLMs' parameters are not accessible to end users. For instance, in the deployment of ChatGPT, we only have access to the LLM's interface post-deployment. Whether additional frameworks or techniques are used on the backend to ensure the LLM's safety remains unknown to the user. In our study, to validate the effectiveness of our MoGU framework, we conducted experiments across various open-source LLMs. This provides deployers with new insights into enhancing LLMs' safety.

Besides, you may be concerned about the risks associated with parameter leakage. Any leakage of model parameters can easily lead to significant safety risks. Previous work[3] has shown that merely a few harmful instructions can completely compromise LLMs' security. Therefore, to mitigate the risks associated with parameter leakage, efforts should be focused on preventing such leaks, which should be addressed through enhanced network security measures.

[1] Safedecoding: Defending against jailbreak attacks via safety-aware decoding.

[2] How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States

[3] Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

Summary: If our supplemental information resolves your questions, we would appreciate a reconsideration of our score.

Dear Reviewer Wh4C

Thank you for your feedback. We apologize for the confusion caused by our "open-sourced LLMs" statement. The "open-sourced" in our study means that the architecture and parameters of LLMs need to be open-sourced to LLM developers, not to everyone. We recognize that such an interpretation may differ from the more commonly held view and promise to adjust it in the camera-ready version.

Our proposed MoGU framework enhances LLMs' safety by improving the LLM's internal architecture. Therefore, to verify MoGU's superiority, we had to conduct all experiments on open-sourced LLMs where their architecture is accessible. Bearing this in mind, we emphasized the "open-sourced" LLMs statement in the title. However, we acknowledge that such consideration may be superficial, weakening the actual significance of our framework and confusing readers.

To further solve this confusion, we would like to re-clarify the actual significance of our framework. Our study has demonstrated that our framework can be flexibly adapted to different LLMs. Thus, in practical applications, our framework can provide insights to LLM developers dedicated to enhancing LLMs' safety during deployment. For LLM developers, all details of LLMs including the architecture are accessible. They can utilize our proposed framework to enhance LLMs' safety before proceeding to commercial deployment. Once deployed, the LLMs provide user access through an API interface, while keeping their parameters and architecture inaccessible to end-users. Therefore, there is no risk of misuse by end-users.

Overall, our proposed framework is not specific to open-sourced or closed-sourced LLMs. Due to the unavailable architecture of closed-sourced LLMs, we are unable to conduct experiments on them. So, we conducted extensive experiments on open-sourced LLMs to demonstrate MoGU's superiority. From a practical standpoint, our framework offers valuable insights for LLM developers focused on enhancing LLMs' safety. However, we regret any confusion caused by the use of "open-source LLMs" in the title. We promise to adjust it in the camera-ready version.

We believe that if you can put yourself in the role of an LLM deployer, you will address your confusion and find our framework valuable. We hope this clarification addresses your concerns, and we would be grateful for a reconsideration of our score.

Dear Reviewer ja88:

Thank you for acknowledging the comparable performance of our proposed solution and for providing extensive experiments. Below, we will address each of the weaknesses and questions you raised.

Q1: Lack of Novelty.

R1: As pointed out by Reviewer mJ14 and Reveiwer 3hkc, the novelty of our study is to extend the idea of the Mixture of Experts (MoE) framework for safety purposes. As far as we know, this is the first implementation of the MoE routing idea specifically for improving LLMs' safety. Our proposed MoGU framework significantly improves the safety of LLMs while preserving their usability, addressing the key challenge of existing defense strategies effectively.

Q2: The method is relatively heavy.

R2: Our method is not heavy. MoE architectures are increasingly utilized across various domains, often incorporating LoRA and Router components that employ low-rank decomposition matrices. This design minimizes the addition of parameters. Many studies[4] have deployed eight or more experts to achieve their purpose. Compared with them, our approach effectively achieves the safety purpose using only two experts. Furthermore, to minimize inference costs, we also employ a strategy where only the first five tokens are decoded with MoGU, significantly reducing the inference cost time caused by the increase in parameters.

Q3: The improvement obtained is not significant.

R3: The improvement brought by our proposed MoGU is very significant.

• Safety Evaluation: against original malicious instructions, MoGU enhances defense performance by an average of 16.75% across three LLMs when compared to base LLMs (No Defense). Against malicious instructions with various attack templates, MoGU also shows marked harmful score (attack success rate) improvements in defense performance: 0.19 points (2.30%) decline on Llama2, 3.42 points (43.4%) decline on Vicuna, and 2.37 points (59.60%) decline on Flacon. Besides, compared to existing defense strategies, our MoGU method almost achieves SOTA performance and consistently ranks within the top three across various LLMs.

• Usability Evaluation: Our study observed that existing defense strategies often lead LLMs to adopt a rejection-oriented stance, thereby diminishing their usability. For example, after applying the ICD strategy on Llama2, the response usability score dropped dramatically from 3.87 to 2.17. After applying the SafeDecoding strategy on Vicuna, the response usability score dropped from 3.89 to 2.29. Besides, we noticed that some defense strategies (e.g. Detect$_{inp}$), while preserving LLMs' usability, did not significantly improve their safety. In contrast, our MoGU framework not only significantly enhances LLMs' safety but also achieves the same level of usability as the base LLM on Llama2, with only minor drops of 0.21 and 0.01 on Vicuna and Falcon, respectively.

Q4: Why does ICD outperform SafeDecoding on Llama2 in Table 3, while the opposite is true for the other two models?

R4: ICD leverages the concept of In-Context Learning to enhance the LLMs' safety by incorporating demonstrations of rejections to malicious instructions into the prompt. However, due to the different contextual perception abilities of different LLMs[2], their performance will be different. From the experimental results, Llama2 demonstrates robust contextual perception, adopting a strong rejection stance that, while significantly enhancing security, can detract from usability. On the other hand, Vicuna exhibits weaker contextual perception, which minimally affects usability but does not markedly improve safety. The performance of Falcon is somewhere in the middle. The above phenomenon has also been observed in previous work[1].

Q5: The line spacing of the article does not seem to comply with submission standards.

R5: To accommodate Tables 2 and 3 on a single page for comparative purposes, we made minor adjustments to the spacing above and below the table captions, resulting in a tight layout. However, we promise we have not altered any critical formatting details such as fonts or page margins. In the upcoming version, we plan to refine our layout without modifying any formatting elements.

Q6: Lack of discussion on the impact of the number of modified tokens.

R6: As highlighted in our R1, the novelty of our MoGU lies in extending the MoE framework to safety scenarios. However, introducing new parameters in our framework increases the inference time cost. Previous studies[3] have observed that the initial token of a response is critical for ensuring LLMs' safety. Based on these insights, to reduce inference time costs, we employ the strategy of only decoding the first m tokens with our MoGU. Besides, a recent study[1] has provided detailed experiments on modifying the initial token, including an in-depth analysis of the impacts of the number of modified initialization tokens. Referring to their findings, we set m to 5 (Detailed in Lines 200 to 204).

Q7: Equations in the article are somewhat repetitive, and Figure 2 needs improvement for better visualization.

R7: In the camera-ready version, we plan to streamline several formulas to enhance clarity and coherence. Specifically, we will consolidate companies (3) and (4), and merge formulas (6) and (7). Additionally, we will enhance the visualization of Figure 2 to improve its comprehensibility for readers.

[1] Safedecoding: Defending against jailbreak attacks via safety-aware decoding.

[2] What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning.

[3] Jailbroken: How does LLM safety training fail?

[4] When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications.

Summary: If our supplemental information resolves your questions, we would appreciate a reconsideration of our score.

Dear Reviewer ja88:

Thank you for your continued feedback. Regarding your questions, we would like to make some further clarifications.

Q1: Insignificant results

As reviewers 3hkc and mJ14 have pointed out, our framework has achieved significant improvements, not only providing a more balanced performance between safety and usability but also outperforming existing defense methods. In our previous responses, we have elaborated on the significant improvement provided by our framework.

• Safety Evaluation: against original malicious instructions, MoGU enhances defense performance by an average of 16.75% across three LLMs when compared to base LLMs (No Defense). Against malicious instructions with various attack templates, MoGU also shows marked harmful score (attack success rate) improvements in defense performance: 0.19 points (2.30%) decline on Llama2, 3.42 points (43.4%) decline on Vicuna, and 2.37 points (59.60%) decline on Flacon. Besides, compared to existing defense strategies, our MoGU method almost achieves SOTA performance and consistently ranks within the top three across various LLMs.

• Usability Evaluation: Our study observed that existing defense strategies often lead LLMs to adopt a rejection-oriented stance, thereby diminishing their usability. For example, after applying the ICD strategy on Llama2, the response usability score dropped dramatically from 3.87 to 2.17. After applying the SafeDecoding strategy on Vicuna, the response usability score dropped from 3.89 to 2.29. Besides, we noticed that some defense strategies (e.g. Detect$_{inp}$), while preserving LLMs' usability, did not significantly improve their safety. In contrast, our MoGU framework not only significantly enhances LLMs' safety but also achieves the same level of usability as the base LLM on Llama2, with only minor drops of 0.21 and 0.01 on Vicuna and Falcon, respectively.

However, we are a bit puzzled and would appreciate it if you could specify the aspects in detail where you believe our improvements are insignificant. We are more than willing to provide further clarification.

Q2: Why not extend to the entire generation process?

Regarding this question, we would like to have an in-depth discussion. In our framework, we trained two LoRA modules and a router, which are embedded within the base LLMs' architecture. During inference, for the first five tokens, we activate the base LLM, the two LoRA modules, and the router simultaneously. For the subsequent tokens, only the base LLM remains active, while the two LoRA modules and the router are idle. Such a decoding strategy effectively addresses the issue of increased decoding time due to additional parameters.

Given that such decoding strategy not only achieved SOTA defense performance but also effectively solved the problem of decoding time, we are puzzled as to why we need to decode the entire sequence with the additional parameters. We think that the additional parameters throughout the entire decoding process will significantly increase decoding time, thereby weakening the lightweight nature of our proposed framework. Moreover, such a decoding strategy has been proposed in previous work[1] and shown to be effective.

Based on these insights, we believe that our design is reasonable and efficient.

[1] Safedecoding: Defending against jailbreak attacks via safety-aware decoding.

Q3: Why not more application scenarios?

Regarding this question, we would like to provide further clarification. For the safety evaluation, we have included two distinct malicious instruction test sets and five mainstream jailbreak attack methods. For the usability evaluation, Just-Eval is a comprehensive dataset used to assess the performance of LLMs. In terms of task types, it encompasses information-seeking questions, math questions, coding questions, writing questions, role-play questions, reasoning questions, and procedure questions. As for the topics, they cover humanities, finance, ethics, nature, medical, STEM, and lifestyle. The details of Just-Eval can be found at https://github.com/Re-Align/just-eval. Thus, you can believe that we have conducted evaluations across different application scenarios. However, we acknowledge that our lack of a detailed description of Just-Eval may have led to some misunderstanding, and we hope this clarification resolves your confusion. We promise to provide a detailed description of Just-Eval in the revision.

Thanks again for your review. We hope to receive your continued feedback to further resolve your confusion.

Dear Reviewer ja88:

We appreciate your thoughtful review. As the rebuttal deadline approaches, we kindly ask if our responses have sufficiently addressed your concerns. Should you require further clarification, we are prepared to provide additional information.

Sincerely,

Authors

Dear Reviewer ja88

As the rebuttal deadline approaches with just 12 hours remaining, we noticed that you still have some concerns about our work. We have provided further clarification on these points and supplemented them with detailed experiments. If you could spare some time for further discussion, we would greatly appreciate it.

Thank you for your attention and consideration.

Sincerely,

Authors

Dear Reviewer ja88:

We have supplemented the experiments extending to the entire generation process to further clarify your concerns. However, due to the complexity of the evaluation process, which involves the evaluation of two LLMs and 1,570 test samples, as well as the need for ChatGPT to score them, we apologize for the delay in submitting the results. We believe that our experiments will thoroughly address your concerns.

To further demonstrate that our design is reasonable and effective, we extended MoGU decoding to the entire generation process and conducted experiments on two mainstream LLMs (Llama2 and Vicuna). The strategy that MoGU only decodes the first five tokens and the base LLM decodes the rest tokens is named as First M and the strategy that MoGU decodes all tokens is named as All.

• First, we compared the performance of two strategies when faced with standard malicious instructions. The following table reports the Attack Success Rate (ASR), where a lower ASR indicates better defense performance. From the table, it is evident that both strategies perform equally well in defending against standard malicious instructions.

• Next, we evaluated the performance of the two strategies against various jailbreak attacks. The following table reports both the Attack Success Rate (ASR) and Harmfulness Score (HS), where lower values indicate better defense performance. The HS(ASR) is presented in the table. From the results, we can see that the strategy ALL only slightly outperforms the strategy First M in defending against jailbreak attacks.

• Finally, we compared the performance of two strategies when faced with benign instructions. The following table reports the response quality scores across five dimensions: helpfulness, clarity, factuality, depth, and engagement. The higher scores indicate higher quality. It is clear from the results that the strategy First M significantly outperforms the strategy ALL in handling benign instructions.

Overall, the defense performances of both are comparable, but strategy Fisrt M significantly outperforms strategy ALL in terms of the quality of responses to benign instructions. Considering these results, and the fact that decoding all tokens with MoGU significantly increases inference time, we believe that adopting the strategy First M is more reasonable and efficient.

Dear Reviewer mJ14:

Thank you for acknowledging the originality and significant improvements of our proposed solution. Below, we will address each of the weaknesses and questions in detail.

Q1: The biggest weakness of the paper is in its clarity. A good example here is eq (5) which feels unnecessary.

R1: We appreciate the identification of this weakness in our work. We believe that as a scholar who is well-versed in the architecture of LLMs and the MoE series of works, you can easily understand these formulas and may even find some of them unnecessary. However, we are concerned that scholars lacking knowledge of MoE series work or LLM architectures may find it difficult to understand how vectors propagate forward through simple language alone. Therefore, we have detailed the forward propagation of vectors in the form of formulas as much as possible. Finding a balance in presentation that makes our framework clear to scholars with different backgrounds is challenging. We plan to appropriately replace some formulas with English expressions in the camera-ready version to achieve a better balance.

Q2: Why not normalize in any way the router? Surprised that the weights are unbounded.

R2: There has been a significant misunderstanding at this point. The weights assigned by our router are bounded, with values falling within a continuous range of 0 to 1. In eq (5) which you might have deemed unnecessary, we have demonstrated how the weights are calculated. Specifically, in eq (5), we apply the sigmoid activation function to ensure that the weight values are constrained between 0 and 1.

Q3: I am having trouble gaining confidence from the experimental design/baselines.

R3:

• Why not compare some methods which are better suited here, such as RLHF?

SFT and RLHF are algorithms used to align LLMs with human values. However, LLMs (such as Llama2) that have undergone SFT and RLHF are still susceptible to jailbreak attacks. This exposes the shortcomings of SFT and RLHF. Moreover, the instability of RLHF and its high demand for training data quality are significant deterrents.

Therefore, recent studies have focused on developing various defense strategies, such as attempting to control the prompt and decoding strategies. The work like SafeDecoding[1] has aimed to balance the LLMs' safety and usability. While these efforts have brought some improvements, they have not fundamentally solved the problem. Different from these strategies, we extend the MoE architecture for safety purposes, effectively addressing this issue. Therefore, our study focuses more on comparing various defense strategies rather than algorithms for human-values alignment. Similar baselines and experimental setups can be found in many recent works[1][2].

Although the results of RLHF are not reported in our paper, our framework can be used to improve the safety performance of LLMs that have undergone SFT and RLHF. One advantage that may be overlooked is that our framework and RLHF alignment algorithm can be seamlessly integrated.

• Plot results in a way that can more directly show the trade-off in safety and usability.

We appreciate the identification of this weakness in our work. In response, We are planning to present our experimental results more intuitively.

Q4: Why is this specific to open-source LLMs? (Separately, disappointed that this doesn't cover the challenges of keeping open-source LLMs safe when adversaries can retune or sample them differently)

R4:

1. As our MoGU requires transparency of model parameters and architecture, our study selects various open-sourced LLMs to validate MoGU's superiority. In practical deployments, the LLM's parameters and architecture are transparent to the deployers but remain a black box to the users. Hence, deployers can utilize our proposed MoGU to enhance the LLM's safety.

2. We understand your concerns regarding the generalizability of our MoGU defense ability. It is important to note that our training stage does not include any data with attack templates. During evaluations, MoGU demonstrates robust defense performance against a variety of unseen attack templates. Thus, even when faced with unknown adversarial disturbances, our MoGU still significantly outperforms previous defense strategies. Such results verify the generalization of our MoGU defense ability. If you are interested in our division of data, you can refer to our response to review 3hkc.

Q5: Using the ratio of CEs in eq (3) and (4) seems fairly non-standard (even in contrastive learning) - is this done elsewhere? Is the loss propagated through both the numerator and denominator?

R5: In our work, we aim to calibrate the base LLM to two extreme states — Glad$resp$ and Unwill$resp$. Taking Glad$resp$ as an example, we hope it can produce glad responses to any instruction, rather than rejection responses. Therefore, when training Glad$resp$, our goal is for the LLM to learn glad responses and forget rejection responses. To facilitate this learning process, we treat glad responses as positive samples, aiming to minimize their loss. Conversely, we treat rejection responses as negative samples, aiming to maximize their loss.

To minimize the loss for positive samples and maximize the loss for negative samples, we place the former in the numerator and the latter in the denominator. We believe this to be an intuitive design choice. Importantly, in our ablation studies (Detailed in Table 4), we observed that eq (3) and (4) brought significant improvements in performance on the Llama2 and Vicuna.

[1] Safedecoding: Defending against jailbreak attacks via safety-aware decoding.

[2] SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidance.

Summary: If our supplemental information resolves your questions, we would appreciate a reconsideration of our score.

Dear Reviewer 3hkc:

Thank you for acknowledging the novelty of our proposed solution and for providing a comprehensive setup of experiments along with quantitative analysis. Below, we will address each of the weaknesses and questions in detail.

Q1: Lack of a more in-depth discussion regarding the selection of instructions.

R1: To address this, we provide an overall description of the selection of instructions. In our study, the type of instructions can be divided into the following three categories:

• During the training phase, we utilized a training dataset comprising 300 benign instructions and 300 original malicious instructions, sourced from the Alpaca and Advbench benchmarks respectively (Detailed in Lines 123 to 124). It is worth noting that any malicious instruction with an attack template will not appear in our training data.

• During the evaluation phase, we assessed the LLMs' performance in terms of usability against benign instructions and safety against malicious ones. To ensure no training data leakage, we used a distinct set of 800 benign instructions drawn from Just-Eval for evaluation (Detailed in Lines 224 to 225). For original malicious instructions, we included 220 instructions from Advbench that were not part of the training set, alongside 200 instructions from Just-Eval (Detailed in Lines 213 to 214). Additionally, we evaluated the LLMs' safety against malicious instructions with attack templates based on prevalent jailbreak attack methods (Detailed in Lines 215 to 223). Examples of instructions with various attack templates can be found in Appendix C.

In general, we strongly believe that our selection and division of data are appropriate. Similar data selection and division can also be found in a recent work[1] accepted by ACL2024 Main.

[1] Safedecoding: Defending against jailbreak attacks via safety-aware decoding.

Q2: The analysis of the experimental results is sometimes overly simplistic.

R2: To address your confusion, we provide some more insightful and detailed analysis to understand better and interpret the significance of our findings. Our supplementary content primarily revolved around findings in Sec 5.1 Quantitative Analysis and Sec 4.2 Main Results.

1. Findings in Sec 5.1 Quantitative Analysis

• Our MoGU introduces the idea of contrastive learning in Loss$glad$ and Loss$unwill$ to calibrate Glad$resp$ and Unwill$resp$ and incorporates a fine-grained objective into Loss$router$ to constrain the weight assignment. The former is denoted as Loss$CL$ and the latter as L1$Norm$. To validate their impact, we conducted an ablation analysis. Table 4 illustrates that omitting Loss$CL$ and L1$Norm$ will lead to a decrease in the defense performance of our framework. Notably, the performance drops more significantly after omitting the L1$Norm$, emphasizing the importance of constraining the router's weight assignment. Such a phenomenon underscores the critical role of the router's weight assignment, aligning perfectly with our motivation.

2. Findings in Sec 4.2 Main Results

• MoGU keeps robust defense performance. Against original malicious instructions, MoGU enhances defense performance by an average of 16.75% across three LLMs when compared to base LLMs (No Defense). Against malicious instructions with various attack templates, MoGU also shows marked improvements in defense performance: 0.19 points (2.30%) improvement on Llama2, 3.42 points (43.4%) improvement on Vicuna, and 2.37 points (59.60%) improvement on Flacon. Besides, our MoGU method almost achieves SOTA performance and consistently ranks within the top three in terms of defense performance across different LLMs and different scenes, affirming its effectiveness and reliability in enhancing LLMs' safety.

• Existing defense strategies enhance the safety of LLMs but often compromise their usability. As shown in Table 2, the ICD strategy significantly increases the defense of Llama2 to jailbreak attacks. However, after applying the ICD strategy, the rate of rejection responses to benign instructions on Llama2 surged from 14.00% to 92.25%, and its response usability score dropped dramatically from 3.87 to 2.17. Similarly, the SafeDecoding strategy effectively defends Vicuna against jailbreak attacks. However, it leads to a substantial increase in rejection responses from 3.63% to 39.50% and a decline in response usability score from 3.89 to 2.29. Such phenomenons indicate that existing defense strategies often lead LLMs to adopt a rejection-oriented stance, thereby diminishing their usability..

• MoGU can enhance LLMs’ safety while preserving their usability. We have observed that our MoGU keeps robust defense performance. Notably, our MoGU also maintains the ability to respond with high-quality to benign instructions. As illustrated in Table 3, our MoGU framework achieves the same level of usability as the base LLM on Llama2, with only minor drops of 0.21 and 0.01 on Vicuna and Falcon, respectively. Furthermore, under the MoGU framework, the frequency of rejection expressions in LLM responses to benign instructions remains nearly equivalent to that observed in the base LLMs. These phenomena underscore the superiority of our MoGU framework.

Summary: We plan to present our supplemental information in the camera-ready version. If our supplemental information resolves your questions, we would appreciate a reconsideration of our score.