Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models

Thanks for your reviews.

Desipte the performance reported by the authors, the current contribution is incremental because jailbreaking attacks have been heatedly studied by experts in this domain. The proposed method mainly improves the defense method in the original setting.

We first build a framework that adaptively defends jailbreak attacks in an online manner, instead of static defense, which differs from previous works a lot. Moreover, we build an efficient detector and a dataset to train it, making our defense more practical.

We sincerely thank you for your kind reviews.

Thanks for your kind reviews. We provide additional experimental results in the link https://anonymous.4open.science/r/ICML109sda-E0E4/tab_and_fig.pdf. We will address your concerns one by one.

If the proposed method cannot detect a sophisticated jailbreak attack, then it might never be able to capture it. Conversely, if there is too much data in the initial dataset, the system might become overly sensitive to benign prompts. Additional ablation on the choice of the initial dataset D_d could help address this concern.

We designed the D_d to detect harmful responses instead of the jailbreak instruction. Although the attacker's jailbreak attack may be complex, the answer is not particularly difficult to identify. Moreover, as long as a small number of samples can be identified, we can use these small samples to implement our defense effectively according to our experiments.

The number of jailbreak attacks tested is limited (one for MLLM and one for LLM), raising questions about the method's effectiveness against more advanced jailbreak techniques.

We supplemented the results of the white-box attack, GCG, in Table 3 in the link for reference. TIM still shows its effectiveness on GCG. After that, we evaluate two jailbreaks for LLM (i.e., GCG, I-FSJ) and two for MLLM (i.e., Figstep, MM-SafetyBench).

The ASR calculation method using prefix matching is known to lack accuracy. It is suggested to implement LLM-based ASR calculation methods to validate the results more reliably.

We report the ASR evaluated by LLM in Table 3. Indeed, prefix match is more accurate than LLM, according to our observation. For example, VLGuard refused to answer the jailbreak question successfully, but most samples were still judged to have generated harmful content under LLM evaluation. Moreover, the attack success rates of the other three methods are lower than the actual situation. Methods based on large model evaluation are subject to fluctuations in many factors, whether it is the large model used, the template, or the decoding way.

How does the pipeline respond when it encounters a previously unseen jailbreak attack?

All of our experiments are conducted under the unseen jailbreak attack, the response pipeline can be seen in Algorithm 1 in the appendix. Or if you want to ask the static transferability, like Reviewer 77PY, you can refer to the rebuttal for him.

Rewriting of Section 4.

Thanks for your advice. We will rewrite Section 4 in our manuscript to make it clearer and more readable.

We are grateful for your valuable suggestions. The figures and tables of the additional results are provided in the link https://anonymous.4open.science/r/ICML109sda-E0E4/tab_and_fig.pdf. We will do our best to address your concerns.

The concern about the efficiency and effectiveness of our detector. And the training cost of our defense training.

To demonstrate the efficiency of our methods, we report the time cost in Table 1 (see the link). TIM takes a short time for detection (about 0.3% of inference cost), and the latency caused by detection is extremely low. For effectiveness, we compare metrics such as Accuracy, True Positive Rate (TPR), and False Positive Rate (FPR) with detection methods like Self-Defense and LVLM-LP in Table 3 of the original paper. Additionally, we report the detection metrics in Figures 3, 4, and 5b of the original paper. Our method shows effective detection performance (90+% TPR, and FPR less than 1%) compared to other methods and our variants. We also add the accumulated TPR and FPR during training in Figures 1 and 2 (in the link) for further analysis. According to the results in Table 1, the training time accounts for 12.2% of the total time used. The testing process of TIM is more efficient than the vanilla model because fine-tuned model can generate short rejection instead of long harmful responses for jailbreak instructions. For computation requirements, we only use one RTX A6000 for training. In practical applications, defense training can be conducted on the training GPU instead of the inference device.

The paper does not sufficiently address the likely increase in false positive refusals, both in what is classified as harmful and in the subsequent outputs of the re-finetuned model.

We report the​ Over Defense Rate (ODR) to assess false positive refusals. Our method exhibits relatively few false rejections compared to other methods, as shown in our paper. In the experiment results, there are usually two types of samples that are judged as harmful: malicious instructions with harmful responses or rejection responses. Normal samples are rarely judged as harmful. Based on the ODR of our experiment, most samples can still be answered normally by our fine-tuned model.

Manual verification and testing on held-out examples.

We think the detector performance can be validated by Acc, TPR, and FPR. For manual verification, the false positive examples are more like to be the samples metioned above and normal samples are rarely misjudged as jailbreak samples (see FPR of the TIM-NA). Moreover, we including the held-out accuracy of the validation set from detector training (99+% for all experiments) in the Table 6 (in the link).

Our method against more complex jailbreaks.

The jailbreak method (i.e. PAIR, TAP) performs poorly (<10% ASR) on llama7b-chat, according to original paper and [1]. MSJ is a similar attack to the I-FSJ (which we used). Indeed, I-FSJ is a complex method including techniques like in-context jailbreak, greedy searching, and token replacement. I hope the explanation can address your concerns. Furthermore, we supplemented the results of the white-box attack, GCG in Table 3. [1] https://www.harmbench.org/results

How well your fine-tuned models perform on new jailbreaks (do they get more robust even to held-out jailbreaks when fine-tuned against?).

It's worth noting that our method is an online adaptive defense method. New types of jailbreaks will be adaptively defended against as they emerge. Nevertheless, we demonstrate the static transferability of the fine-tuned model in Table 5 in the link. It is effective when migrating from a more complex attack (MM) to a simpler one (Figstep), but its effectiveness is limited in the reverse direction.

How practical do you think your test-time training approach would be on frontier models.

The inference latency is extremely low, as stated in the first response. The training cost is only 12.2% of the total computing cost. Training occurs only when jailbreak activities are successful. Moreover, the defense training can be submitted as a background task. As a result, the latency is just the detection latency.

Do you have any evidence or theoretical basis for how your approach might perform on much larger models?

We have conducted additional experiments using LLaVA1.6-Vicuna-13B, as presented in Table 4. TIM remains effective on the 13B model. However, due to computational resource limitations, we are unable to apply our method to 70B+ models (we only have access to an RTX A6000).

How fine-tuning process affect the model's ability to handle legitimate requests? Do you observe an increase in false positive refusals?

We report ODR in our paper. TIM shows the fewest false positive refusals compared to other methods. For more details, you can refer to the change in ODR shown in Figures 1 and 2 in the link. There are increases, but acceptable.

Morevoer, We will discuss Anthropic's paper in the manuscript.

We sincerely appreciate the constructive feedback. Below we provide point-by-point responses with methodological clarifications and supplementary experimental evidence. All referenced figures/tables are available in our link https://anonymous.4open.science/r/ICML109sda-E0E4/tab_and_fig.pdf.

Regarding adaptive attack against our method.

We acknowledge the reviewer's concern about possible adaptive attacks targeting our defense approach. Indeed, adversaries could attempt to craft adversarial samples to mislead our detector and degrade its performance. However, in practical deployment scenarios, our detector operates as a black box—its internal decisions are not directly exposed to users. This inherent opacity increases the difficulty of mounting successful attacks, as adversaries lack direct feedback to refine their adversarial inputs. However, we are considering designing a targeted attack in future work to verify whether our method is robust enough.

Concerned about performance during finetuning and the catastrophic forgetting.

In Figures 1 and 2 in the provided link, we report key metrics, including accumulated ASR, ODR (Over Defense Rate), TPR, and FPR, for our method during the fine-tuning process. If the reviewer is referring to the model's ability to answer normal questions, the changes in ODR provide a clear indication of its performance in this regard. While our method did cause the model to reject some normal answers during testing, this effect was manageable and did not lead to significant degradation in overall performance. If you mean the ability to reject the jailbreak attack encountered before, we reported this result in Figure 4 of the original paper. Despite experiencing other attacks, we still maintained our defense against the previous jailbreak attack. No obvious catastrophic forgetting of defense capability against previous jailbreaks is observed.

Scalability of gist token.

For multi-turn scenarios, the purpose of gist token is to detect whether the model generates malicious answers, so in multi-turn conversations, we only need to insert gist token after each answer generated by the model, and remove it from the KV cache after the detection is completed. For detector training, we can also build a multi-turn conversation detection dataset to ensure that we can still maintain effective detection when facing multi-turn data. For generalizations for longer context, in our experiments, I-FSJ is a in-context jailbreak attack that uses multiple demonstration to induce our model to generate malicious answers. The average length of its jailbreak instructions is 1720 tokens, while in our training set, the average length of questions and answers is only 15.2 and 281.4 tokens. However, we can still effectively detect jailbreak samples in I-FSJ, hoping that this can address your concern about the generalization of gist token.

Training parameter of the detection loss.

We solely train the gist token and the binary classifier. During the training of the detector, the LLM is frozen to prevent any impairment to the model's question-answering ability.

Question about the defense training.

The LoRA is continuously trained. We have attached the training and detection costs in Table 1, which can be found via the provided link. The results indicate that our method is highly efficient and thus practical for application.

Modern architecture.

We have added the results of LLaVA-v1.6-Vicuna-13B in Table 3, which can be found in the link. LLaVA 1.6 is among the most state-of-the-art MLLMs. Currently, we are attempting to conduct experiments on LLaMA3-8B-Instruct, and we will update the results soon if we finish.

Related works and writing.

Thank you for your valuable advice. In the final version, we will carefully consider and discuss the differences between the works you have mentioned. Additionally, we will rewrite Section 4.3 to enhance its clarity and comprehensibility.

We sincerely hope that the above rebuttal effectively addresses your concerns.

If you have additional question or we midunderstand your review, feel free to respond.