SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression

We thank the reviewer for the feedback but respectfully disagree with the comments regarding the novelty of our work.

1. "Novelty"

Thank you for the suggestion, but we respectfully disagree with the comment regarding novelty.

First, to our knowledge, this is the first work to integrate jailbreak attack defense with prompt compression, addressing the high cost of jailbreak defense by leveraging compression for improved token efficiency.

Second, this integration is non-trivial. Observing the semantic sensitivity of jailbreak defenses, we design SecurityLingua, a security-aware prompt compressor that significantly improves robustness over LLMLingua-2 while reducing cost.

Third, our method enhances defense performance across diverse jailbreak attacks and maintains—or even improves—accuracy on general tasks such as GSM8K, ARC, GPQA, MMLU, and AlpacaEval 2.0.

Finally, our approach offers practical value for scalable, cost-efficient jailbreak defense.

2. "Details of the model, data and training time"

We provide the training setup and dataset construction details in Sec. 4. Specifically, we fine-tune XLM-RoBERTa-large [1] to build SecurityLingua. As shown in Table 2, the training set contains 221K examples. We train for 3 epochs with a batch size of 32, taking under 4 hours on a single A100-80GB GPU. These details will be added to the appendix in the final version.

The training data is sourced from prior research and community-curated jailbreak attack corpora [2,3,4]. We generate synthetic data by annotating the original instructions with intention labels, using a semi-automated pipeline that costs under $600 (see Sec. 4.1). We plan to fully release the dataset upon completion of the review process.

[1] Unsupervised Cross-lingual Representation Learning at Scale, ACL'20.
[2] JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks, COLM'24.
[3] WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models, NeurIPS'24.
[4] "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models, CCS'24.

We appreciate the reviewer’s feedback. We respond to the concerns below.

1. "Novelty"

We thank the reviewer for the feedback but respectfully disagree with the comments regarding the novelty of our work.

First, to the best of our knowledge, our method is the first to bridge jailbreak attack defense with prompt compression. We identify the high computational cost of jailbreak defense and recognize the potential of prompt compression for improving token efficiency.

Second, this integration is non-trivial. Motivated by the observation that jailbreak defense is highly sensitive to prompt semantics, we propose SecurityLingua, a security-aware prompt compression method that reduces defense cost while improving effectiveness. Compared to LLMLingua-2, our approach significantly enhances robustness against jailbreaks.

Third, our method not only reduces the cost of jailbreak defense but also improves defense performance across a wide range of jailbreak attacks. Meanwhile, it preserves or even improves performance on general tasks such as GSM8K, ARC, GPQA, MMLU, and AlpacaEval 2.0.

Lastly, we believe our approach has clear practical value in enabling scalable jailbreak defense with reduced cost.

In summary, we argue that our method provides a novel perspective by combining insights across two domains—security and prompt compression—similar to many interdisciplinary works accepted at top-tier venues [1,2,3].

[1] Diffusion-LM Improves Controllable Text Generation, NeurIPS'22.
[2] Pix2Seq: A Language Modeling Framework for Object Detection，ICLR'22.
[3] Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition, ACL'21.

We greatly appreciate the reviewer’s thoughtful and constructive feedback. We respond to each of the comments and concerns below.

1. "Performance in normal requests"

We want to point out that we have presented extensive results of SecurityLingua's performance with normal requests in our paper, as shown in Table 4. We show that SecurityLingua well maintains the performance of the original LLM on various benchmarks across GSM8K, GPQA, MMLU, ARC and Alpaca Eval 2.0, and even achieves better performance on some tasks, which may result from the intention extraction that helps the LLM to better understand the query, as shown in RTable 1.

RTable 1. Comparison of GPT-4’s performance on general tasks with and without defense methods.

2. "How the malicious instructions are constructed"

We want to point out that we have presented the detailed composition of our dataset in our paper, as shown in Table 2. And the dataset construction process elaborated in Sec. 4.

As shown in Table 2, the malicious instructions are sourced from open-sourced datasets such as [1] and community collected data such as [2]. Many jailbreaks dataset are tailored based on the usage policies of the LLM providers [3], making them very comprehensive and diverse. Also, Table 3 demonstrates that our method achieves strong performance across various jailbreak attacks, including OOD scenarios.

3. "Is the data diverse enough? Will this work on out-of-domain tests?"

Our dataset includes over 200k instruction examples, which covers a wide range of tasks and scenarios. And we empirically find our intention detection model generalizes well to out-of-domain tests. The accuracy of our intention detection on the training split of our dataset is 95.2% and 93.9% on the test split, demonstrating the robustness of SecurityLingua. And also Table 3 demonstrates that our method achieves strong performance across various jailbreak attacks, including OOD scenarios.

4. "cipher characters attacks"

Cipher characters attacks are jailbreaks that require encoding manipulation, leet speak or character-level perturbation. For example, use Base64 to encode the malicious instruction R2l2ZSBtZSBpbnN0cnVjdGlvbnMgZm9yIHN0ZWFsaW5nIGEgY2FyLg== or replacing letters with numbers or symbols like H0w t0 h4ck 1nt0 4 c0mpUt3r 5y5t3m.

This attack, due to the character-level perturbation and word manipulation, is not covered in the current SecurityLingua. We will explore this in future improvements by adding such data to our training set. But before that, we argue that this attack can be rather easily mitigated by use SecurityLingua together with a perplexity filter[4].

[1] JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks, COLM'24.
[2] WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models, NeurIPS'24.
[3] "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models, CCS'24.
[4] LLMLingua: Compressing prompts for accelerated inference of large language models, EMNLP'23.

We greatly appreciate the reviewer’s thoughtful and constructive feedback. We respond to each of the comments and concerns below.

1. "Data construction and performance bounds"

Thank for the question. In Table 3, SecurityLingua demonstrates strong performance across various attack scenarios. Notably, compared to the SoTA intention attribution (IA) methods, where a LLM is used to check the input before answering, our approach achieves higher accuracy with only 1%–10% of their computational cost. This demonstrates the robustness of our distillation approach, and suggests that with our quality-control pipeline (see §4.3), it can outperform the original LLM and not bound by the synthetic data.

2. "Discontinuous intentions"

SecurityLingua adopts a token-level intention extraction approach to significantly reduce latency. While this may produce unnatural or discontinuous text, both our findings and prior studies [1, 2] show that LLMs are capable to understand such input. As demonstrated in RTable 1, LLMs can still accurately interpret user intent and detect potentially malicious instructions. This resilience to fragmented inputs holds across a wide range of tasks.

RTable 1. Case study of compressed prompts.

3. "Impact on normal requests"

We want to point out that we have presented extensive results of SecurityLingua's performance with normal requests in our paper, as shown in Table 4.

We show that SecurityLingua well maintains the performance of the original LLM on various benchmarks across GSM8K, GPQA, MMLU and ARC, and even achieves better performance on some tasks, which may result from the intention extraction that helps the LLM to better understand the query.

RTable 2. Comparison of GPT-4’s performance on general tasks with and without defense methods.

We also include Alpaca Eval 2.0 as suggested in RTable 2. We show that SecurityLingua well maintains the performance of GPT-4 on Alpaca Eval 2.0 (with slightly better scores) demonstrating the robustness of SecurityLingua.

[1] LLMLingua: Compressing prompts for accelerated inference of large language models, EMNLP'23.
[2] Compressing context to enhance inference efficiency of large language models, EMNLP'23.