Response to {Reasons To Reject}:

A1: Major concern

We do not ONLY evaluate the unsafe textual prompts. All the jailbreak samples in our benchmark involve text and image inputs. We also included 8000 unique attack queries based on two attack methods (QR and Figstep), where the images are malicious and the corresponding text is benign. We believe our benchmark comprehensively evaluates MLLM jailbreak safety from BOTH textual and image attack surfaces.

A2: Methodological weakness

We want to clarify that we ask LLM to generate 100 unsafe QUESTIONs at once. This is because it will generate more diverse questions. We empirically observe that when data are generated one at a time, LLM tends to produce highly similar data even with varying seeds. We generated 20 questions with GPT4, and only find 5 unique ones. However, ours will increase the diversity. We compared the similar across the different questions. The similar score is 0.18, which is lower than commonly used question sets, e.g., Advbench (0.33). We think the reason is LLMs are autoregressive models so that each question is generated condition on the previously generated questions. We used Llama Guard to evaluate the toxicity of the GPT-generated data. The results are shown below:

A3 : Frozen image encoder

Our benchmark is designed to evaluate the jailbreak robustness of MLLMs at the inference stage, so the entire model, including the image encoder, is frozen. We believe the image input is fully leveraged to design the jailbreak sample in our benchmark: In our dataset construction process, we have included 8000 unique attack queries based on two state-of-the-art image attack (QR and Figstep), where the images are malicious and the text is benign. These methods ensure a broad and comprehensive evaluation of MLLMs' vulnerabilities.

Response to {Questions To Authors}:

A4: See A2

A5: See A2

Thank you for your feedback and for raising your score based on our rebuttal. In our benchmark, we have included the most "interacting" jailbreak methods that were proposed by the time we submitted (Figstep and QR), which leverage both image and text modality simultaneously to jailbreak MLLMs. We will continue to monitor the community for any new multimodal jailbreak methods focusing on interaction in the future. Thanks for your suggestions. Best regards, Authors

Response to {Reasons To Reject}

A1: Biased Evaluation Method

Thanks for the suggestion, in order to make a more comprehensive evaluation, we have added experiments on 2 models for 4 additional evaluation metrics: (i) L means Llama-guard evaluation with default configuration, (ii) H means evaluation metric in Harmbench[1], (iii) LR means Llama-guard evaluation combined with relevant evaluation which instructs Mistral-7B-Instruct-v0.2 to return the relevance of response from MLLMs and the harmful query. and (iv) HR means Harmbench evaluation metric with relevant evaluation. We find that the results are aligned with the conclusions our paper claimed. We will give details of the settings and results in the revised version.

[1]: MAZEIKA, M., PHAN, L., YIN, X., ZOU, A., WANG, Z., MU, N., SAKHAEE, E., LI, N., BASART, S., LI, B., FORSYTH, D., AND HENDRYCKS, D. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024.

Response to {Questions To Authors}

A2: Generation Time and Cost Evaluation

We will provide an off-the-shelf dataset for jailbreak robustness evaluation, so it will not be a problem for the community to take generation costs on evaluation. In terms of our construction cost, to generate malicious questions, we manually instruct GPT to generate questions, which is not costly. To generate jailbreak prompts, our average cost for each prompt is about 1 second for 5 sample. We generate the jailbreak prompt on 4 NVIDIA A100 80GB.

Response to {Reasons To Reject}

A1: Potential Lack of Originality

Thank you for your feedback. However, we believe our benchmark has unique advantages in the comprehensiveness of malicious questions and the originality of MLLMs jailbreak benchmark. Specifically: Table 1 compares RedTeam-2K with other malicious question datasets used in various benchmarks. RedTeam-2K stands out as the largest and most diverse dataset considering both the size and diversity of the data involved. It's essential to note that all the datasets listed in Table 1 primarily focus on LLM jailbreaks, as they serve as benchmarks for MLLM jailbreaks. Regarding the originality of our work, we propose two datasets: RedTeam-2K and JailBreakV-28K. RedTeam-2K is indeed the most diverse and high-quality malicious question dataset available to date. JailBreakV-28K is the first benchmark specifically designed for evaluating MLLM jailbreak protocols, built upon RedTeam-2K. Our contribution lies in introducing the first MLLM jailbreak benchmark to address the growing need for assessing MLLM vulnerabilities. We hope this clarification addresses your concerns regarding the originality of our benchmark.

A2: Responsibility Release of Dataset

Thank you for your concern. We plan to make the dataset and evaluation code publicly available as an open-source resource. We will take serious action to address safety concerns. Specifically, we will adopt a safety approach to obtain the whole benchmark by conducting a questionnaire and granting permission to researchers who wish to use our benchmark. This will ensure that our benchmark can be used safely.

Response to {Questions To Authors}

A3: Comparative Analysis of MLLM Technical Approaches

We have done evaluations on two types of MLLM solutions: finetuning full LLM backbone and using an adapter module. Unfortunately, the results show that both of the solutions are not robust to LLM’s jailbreak techniques. We will include more analysis in the future.

Response to {Reasons To Reject}

A1: Closed-source models

We add experiments on Gemini. The results show that our benchmark is still effective on Gemini (QR with 30.30 ASR, FigStep with 31.80 ASR, LLM transfer attack with 68.43 ASR).

A2: Transferability of different backbone LLM models

We observe that jailbreak prompts trained on a specific LLM don't lead to a higher ASR across various MLLMs, even if they share the same LLM as their backbone. For instance, the ASR of the jailbreak generated by attacking Vicuna-7B is 78.89 evaluated on llava-1.5-7b with Vicuna-7B as backbone, which is lower than the ASR evaluated on Qwen-VL-Chat with Qwen1.5-7B as LLM backbone (91.40).

A3: Image perturbation

We discussed them only in the related work section since they do not have strong transferability across models which is important for benchmark. We conducted an experiment on VisualAdv and found the transferability of this method is poor (the jailbreak generated by minigpt4 can only achieve 12.9 ASR on Gemini and 1.6 ASR Qwen ).

A4: Dataset publication

We will release the dataset and code

Response to {Questions To Authors}

A5: Variance of ASR

Our new experiments show the variance of ASR for OmniLMM-12B, which is low (0.456 on average among different attacks)

A6: Evaluation criteria for harmful but unrelated generated output

We added experiments about the topic correlation, and the results are consistent with the conclusions in our paper. We use Mixtral-7b to test the topic relevant and judge whether the attack was successful. The ASR is 7.74, 22.2, 22.52, 32.30, 46.98 for different attacks (QR)SD, (QR)SD_Typo, (QR)Typo, Figstep, and LLM transfer attack respectively, which is consistent with the tendency revealed by Llamaguard score in our paper.

A7: High ASR Across MLLM

As analyzed in Sec.4, fine-tuning backbone LLMs in MLLMs will break the safety alignment of the Backbone LLMs.

A8: Safety Policy Configuration of Llama-guard

Yes, it can be extended to new policies in our evaluations. We configure the safety policy to strictly align with our safety policies. We evaluate Llama-guard with and without configuration. To test the detection accuracy, we manually crafted a malicious content dataset containing 200 samples in different policies. The accuracy rises from 88.5% to 93% after we configure the system prompt, which means the model effectively adapts to new policies, aligning with the conclusion of the Llama-guard official paper.