EFFICIENT JAILBREAK ATTACK SEQUENCES ON LARGE LANGUAGE MODELS VIA MULTI-ARMED BANDIT-BASED CONTEXT SWITCHING

We sincerely thank you for your valuable comments, observations, and the time you have dedicated to evaluating our paper. We have carefully addressed your questions and the weaknesses you highlighted in a comprehensive manner. Our response is structured by presenting each weakness or question posed, followed by our detailed reply.

Weakness 1: It is a natural question: why do you not compare your work with other methods such as GCG[1] , AutoDAN[2], PAIR[3], RENELLM[4] and so on?

Response-to-Weakness-1:

In response to reviewer GXrd we compare our proposed Jailbreak attack method to the current SOTA jailbreak methods. We request the reviewer to refer to Table RT2 in Response-to-Weakness-3 to the reviewer GXrd.

Weakness 2: Since your work requires selecting a sequence of context-switching queries, I am curious about it time complexity.

Response-to-Weakness-2:

• In the proposed jailbreak attack method, at each step the MAB decision making framework chooses a context-switching-action using an $\epsilon$-greedy strategy by balancing exploration of new context-switching-actions and exploitation of existing context-switching-actions. (See Line 13 in Algorithm-1 on Page 4 of the main paper). This can be done in constant time $O(1)$ by using efficient data structures such as priority-queues at each step. Thus, for the entire multi-turn attack of $T$ rounds, the time-complexity is $O(T)$.

• We request the reviewer to refer to the response Response-to-Weakness-3 for the reviewer GXrd, where we compare the computational cost of our methods with various SOTA jailbreak attack strategies.

Weakness-3: In my opinion, testing only on Mistral and Llama is not sufficient to demonstrate the advantages of your work. Moreover, you have only chosen small LLMs (up to 8B), which is not convincing. As far as I know, Llama has a 13B version. What's more, CHATGPT is necessary to be choose. In conclusion, without comparisons, it is difficult for me to fully assess the contributions of this paper, especially considering that there are many papers on LLM jailbreaks. If you address my concerns, I will consider giving a higher score.

Response-to-Weakness-3:

We agree that evaluation of our method on proprietary models such chatGPT and larger open-source models such as Llama-13B will further help us assert our claim of high ASR of our proposed method. In this regard, we have evaluated our method on Llama-2-13B-chat and Vicuna-13B- v1.5.

Table RT 5: Performance of our attack on Llama-2-13B and Vicuna-13B- v1.5.

Our method is able to successfully jailbreak chatGPT from our preliminary experiments. However, the harmful responses generated by chatGPT is triggering the output content filter in the chatGPT-API and this blocks us from asking further questions. We are in the process of generating results for chatGPT and will add the performance on chatGPT shortly.

In summary the our contributions to address the reviewer's concerns include :

• Comparison of the proposed jailbreak method with other state-of-the-art (SOTA) methods in terms of Attack Success Rate (ASR) and Cost Per Prompt (CPP).
• Clarifications regarding the time complexity of the proposed method.
• Experiments on larger LLMs using our method to validate our claim of achieving a high Attack Success Rate (ASR).
• Enhancements to improve the presentation and readability of the paper.

We hope the technical clarifications, additional experiments, and justifications we provided have addressed your concerns. If you find the responses satisfactory, we kindly request you to consider raising the score above the acceptance threshold. Please let us know if there are any remaining issues or additional details you would like us to address, which we would do it right away.

We would like to respectfully clarify to the reviewer that our baselines were evaluated on a different dataset compared to the one referenced on the website [1]. This difference can significantly impact the ASR results for various methods. Our dataset has 2700 harmful questions (See section 5.1 in the main draft) compared to the dataset mentioned in the website which has only 100 questions. Additionally, the language models used on the website are older versions of the LLaMA models, specifically the LLaMA-2 model.

It is important to note that LLaMA models are continuously updated with improved safety fine-tuning, making newer versions more resilient to previously proposed jailbreak attacks. This results in a significant decrease in ASR for the latest model iterations. In our work we mainly used Llama-3 models which have better safety fine tuning compared to the older versions of the model.

We kindly request the reviewer to refer to Table 2 in [2] and Table 1 (No defence, LLaMA-2 scores) in [3], which present results for the same methods we have compared against. These tables demonstrate that the ASRs for the referenced methods are either lower than or comparable to those in our work.

[2] https://arxiv.org/pdf/2311.08268#page=5.65 (ReNeLLM)

[3] https://aclanthology.org/2024.acl-long.303.pdf#page=6.62 (Safe Decoding)

For Poor Readability

The author needs revise their manuscript, there are too many issues which the manuscript should not be published.

For Lack of Comparison

The CPP is a new concepts you mentioned during rebuttal. If you believe that the CPP can prove your method effectiveness, please revise in the pdf, instead of review response. I am still confused about what CPP is, as the authors only described it in an informal reply.

For Few experiments

The question is the same as above. The informal reply and the author's confusing expression make it difficult for me to understand what the authors did. Please modify it in the PDF. Thank you.

For Question 2

Please add citations to prove your point, which API uses filters, I don't think your statement is correct.

For Question 3

Thanks to the author for explaining this to me. Please add your clarification in the manuscript.

---

Before all my concerns addressed in revised pdf, I will not imporve my rating.

In response, the author rearranged my review, making it very difficult for me to read.

We sincerely thank you for your valuable comments, observations, and the time you have dedicated to evaluating our paper. We have carefully addressed your questions and the weaknesses you highlighted in a comprehensive manner. Our response is structured by presenting each weakness or question posed, followed by our detailed reply.

Weakness-1: In the abstract and in the overview (Figure 1) , the authors mention "multi-armed bandit (MAB)", but they do not explain what is MAB in the introduction. I suggest the authors revise this, it confuses me until I have read related works.

Response-to-Weakness-1:

We sincerely thank the reviewer for their insightful comment regarding the introduction of the MAB concept in our manuscript. We appreciate your feedback, and we will revise the draft to provide a clear and concise introduction to MAB in the suggested sections. This addition should make the concept and its relevance to our work more accessible from the outset.

Weakness-2: In addition to MAB, the introduction has unreasonable content. With four paragraphs, more than half introduction is irrelevant to the contribution of this paper. And in the third paragraph, there are too many concepts are proposed but lack of details. I suggest that the authors adjust their introduction, current version has poor readability.

Response-to-Weakness-2:

Thank you for your valuable feedback on our paper's introduction. We appreciate your insights regarding the relevance and readability of the introductory section. We understand the importance of maintaining a focused and clear introduction that directly supports our contributions. Based on your suggestion, we will adjust the introduction to ensure that all content aligns more closely with the paper's main contributions and reduces any extraneous information. Additionally, we will refine the third paragraph to provide clearer explanations and avoid introducing too many concepts without adequate detail.

Weakness-3: I think it is not appropriate to introduce how this paper uses MAB in related work. And why is T both rounds and sequence length (total reward over T rounds: line133; attack sequence length T:line137)? What does T represent in the subsequent content?

Response-to-Weakness-3:

Thank you for pointing out the notation ambiguity regarding $T$ in our description of the MAB framework and its use in the training algorithm. We acknowledge that our use of $T$ as both the total number of episodes and as a reference to the context length during each episode may cause confusion.

To clarify, $T$ in our setting represents the number of episodes over which we want to maximize the total reward obtained during the course of the SoC attack, each corresponding to one interaction in the training phase. Within each episode, the "length" of the context window dynamically increases, as shown in line 16 of the training algorithm. This context window accumulates as more context-switching queries (CSQs) are appended to it.

Weakness-4: There are a lot of abrupt concepts introduced without much explanation in context. In line 215, I can not find any information or reference about policy $\pi$ and action-value Q (including cost C in line 240). I can only speculate that this has something to do with reinforcement learning. And in Algorithm 1, what does $E_{explore}$ stand for and what does $E_{exploit}$ stand for? I can not find any explanation.

Response-to-Weakness-4:

Thank you for your valuable feedback. We appreciate the opportunity to clarify the concepts introduced in our paper and address the points you've raised.

1. Explanation of $E_{explore}$ and $E_{exploit}$:

   • In the training algorithm, $E_{explore}$ and $E_{exploit}$ represent the counts of exploration and exploitation actions taken, respectively. These counters track how often we engage in exploration (choosing a random action) versus exploitation (selecting the best-known action based on the current Q-values). They are used to compute the cost function which updates the action values across episodes, which is a key aspect of the epsilon-greedy strategy (See Line 18 in Algorithm 1 on Page 4 in the main paper).

2. Relation between Policy $\pi$ and Q-value $Q(a)$:

   • The policy $\pi$ in this context is essentially a decision rule that dictates which action (CSQ category $a$) to select at any given step. The Q-value $Q(a)$ represents the expected reward for choosing action $a$ (a particular CSQ category) given the current knowledge.

   • The training algorithm iteratively updates the Q-value for each action using the observed rewards, enabling the model to learn the optimal action-value function $Q(a)$ for each DMQ category.

   • The optimal policy $\pi^*$ is a function of the Q-values and is defined as $\pi^{*} = \arg\max_{a}(Q(a))$. This policy selects the action that maximizes the expected reward according to the learned Q-values at each step of the SoC attack, ensuring that the algorithm converges to a strategy that maximizes harmful or unsafe responses over time.

3. Clarification of Training Algorithm’s Purpose:

   • The training algorithm is designed to learn these Q-values $Q(a)$ by balancing exploration and exploitation. Over multiple episodes, the Q-values for each CSQ category get refined based on the observed rewards, allowing the model to converge toward an optimal policy for SoC -based jailbreak attacks.

We understand that the introduction of certain concepts may have seemed abrupt without additional context, and we apologize for any confusion caused. We will make sure to revise the draft to clearly define $E_{explore}$, $E_{exploit}$, the relationship between policy $\pi$ and Q-value $Q(a)$, and the role of the training algorithm in learning an optimal policy.

Your insights have been incredibly helpful in improving the clarity of our paper. Thank you once again for helping us enhance the readability and precision of our work.

In summary our contributions to address the reviewers concerns include,

• Comparison of the proposed jailbreak method with other state-of-the-art (SOTA) methods in terms of Attack Success Rate (ASR) and Cost Per Prompt (CPP).
• Enhancements to improve the presentation and readability of the paper.
• Clarifications regarding the binary reward function used in our work and a through analysis of its comparison PAIR.

We hope the technical clarifications, additional experiments, and justifications we provided have addressed your concerns. If you find the responses satisfactory, we kindly request you to consider raising the score above the acceptance threshold. Please let us know if there are any remaining issues or additional details you would like us to address, which we would do it right away.

Weakness-5: This paper only demonstrate SoC can jailbreak LLMs, but has no comparison with prior works. I believe that there are many excellent baseline jailbreak methods. The SoC attack has similar format with the In-Context Attack[8] which also jailbreak LLMs based on context. Besides, the authors optimize their method with reinforcement learning, but the binary reward is very similar to PAIR, which use LLM optimize jailbreak prompt[9].

Response-to-Weakness-5:

In response to reviewer GXrd, we detail and interpret the comparison with existing SOTA jailbreak methods. We request the reviewer to refer to the Table RT 2 in Weakness-3 and the corresponding response Response-to-Weakness-3. We additionally also detail and compare the existing jailbreak methods based on their computational costs.

Previous multi-turn jailbreak attacks, such as the In-Context Attack, have manipulated the LLM into a jailbroken state by leveraging the context. However, we argue that these attacks fall under the category of templatized approaches and face limitations that we address in our work.

For instance, as noted in Section 3.2 of the In-Context Attack paper, the method relies on sequences of handcrafted prompts, which are often manually created to attempt to jailbreak the LLM. These sequences may include unnecessary or irrelevant steps, leading to longer attack sequences. In contrast, our approach learns a policy within a multi-armed bandit framework, balancing exploration of new context-switching actions and exploitation of existing ones using an $\epsilon$-greedy strategy. This allows us to minimize the attack sequence length by intelligently selecting context-switching actions.

We agree with the reviwer’s comment on the similarities of our binary reward with PAIR. However, we would like to emphasize the following issues with PAIR that we have addressed in our work.

While PAIR employs a binary reward function to indicate whether the LLM has been successfully jailbroken and does not optimize to acheive jailbreak in a minimum number of steps, our approach leverages this function to estimate the expected average reward for each context-switching action. This allows us to learn an optimal policy that minimizes the number of steps required to achieve a jailbreak.

Additionally, PAIR relies on an LLM to refine input prompts based on conversation and reward history, which can increase the number of steps needed for a successful jailbreak. This approach may lead to misaligned prompts without a clear objective, further hindering its efficiency.

We request the reviewer to refer to Table RT2 in our reponse Response-to-Weakness-3: to reviewer GXrd where, we demonstrate the gain acheived by our method in terms of CPP compared to other SOTA jailbreak methods.

Weakness-6: In the entire paper, only four figures in Figure 3 prove that SoC is effective, and there is a lack of comprehensive experiments from multiple angles. For example, with different hyperparameters such as J and K, I am not sure whether this method can be generalized or only performs well under certain specific parameters.

Response-to-Weakness-6:

In response to reviewer feedback and to further assert our claims, we have carried out additional experiments focusing on

The generalization capabilities of our model, wherein we perform 5 fold cross validation by performing policy training on a subset of the original DMQ categories and test on unseen DMQ categories. We find that in terms of ASR our methods achieve’s performance, comparable to training with all the DMQ categories, with the drop in performance being marginal. We request the reviewer to please refer to Table RT1 in our response to Weakness-2) of the reviewer GXrd.

A comparison of the judge function used in our work with the current SOTA judge functions from the Llama-Guard Family. We request that the reviewer refer to Table RT3 in our response Response-to-Question-2: to the reviewer GXrd.

In our method the 3 main hyperparameters are the weights asSoC iated with the cost function which guides the MAB policy learning $w1$, $w2$ and the exploration parameter $\epsilon$ which controls the how our our method balances exploration and exploitation of the context-switching-actions. To choose the best values of the aforementioned hyperparameters, we performed a grid search over the hyperparameters by using a binary search approach to arrive at the hyperparameter values that yield the best performance in terms of ASR in a short amount of time.

Question-1: I believe that in the White-Box Attack, the authors should not cite a lot of attack but is irrelevant to jailbreak. Besides GCG, there are many white-box attack, such as fine-tuning attack[1, 2], improved GCG[3], interpretability-based[4, 5]. Nevertheless, I am only making a suggestion. Whether the author makes modifications or not will not change my rating.

Response-to-Question-1:

Thank you for your feedback and for suggesting additional relevant literature on white-box attacks, including fine-tuning and interpretability-based attacks. We appreciate your suggestion and will incorporate these references into our draft to ensure comprehensive coverage of existing methodologies.

Regarding the references cited in our paper, we believe all cited works contribute to the broader understanding of jailbreak attacks, including relevant aspects of the context and methodology we explored. However, we welcome any specific feedback on particular references that you feel may be irrelevant. This will help us refine our citations to enhance clarity and relevance further.

Question-2: In Section 3.1 line 155-line157, the authors mention "In most instances, such queries fail to produce harmful responses and can be guarded using straightforward strategies, such as employing a classifier to flag harmful words and phrases". Current LLMs do not use those filter, but are fine-tuned to align with human values[6, 7].

Response to Question-2:

To address the reviewer’s query, we clarify that the statement refers to the common practice of wrapping LLM-APIs with guardrails that perform content filtering to prevent responses to prompts containing harmful phrases. This serves as an additional safety layer alongside the safety fine-tuning achieved through RLHF.

Question-3: Algorithm 1 aims to optimize a policy $\pi$, however, where is update for $\pi$ or $\pi^{\star}$. Why can Algorithm 1 obtain a optimized policy $\pi^{\star}$? This confuses me as to how SoC Attack works.

Response-to-Question-3:

Thank you for your valuable feedback and for highlighting this point of potential confusion.

1. Clarification on Policy Optimization: In the Multi-Armed Bandit (MAB) setting we are using, the policy $\pi$ itself is not directly updated. Instead, our algorithm uses a standard $\epsilon$-greedy approach, where the Q-values $Q(a)$ (the expected reward for each action $a$, or CSQ category in this case) are iteratively updated based on the rewards obtained during exploration and exploitation. The policy $\pi$ in this context is defined implicitly by the learned Q-values. Once training is complete, the optimal policy $\pi^*$ is obtained by selecting the action $a$ that maximizes $Q(a)$, i.e., $\pi^* = \arg\max_{a} Q(a)$. This means that the optimal policy $\pi^*$ is derived from the Q-values, not directly updated as in traditional reinforcement learning.

2. Acknowledgment of Ambiguity: We respectfully agree with your observation that the algorithm is presented ambiguously, which could lead to misunderstandings about how the policy $\pi$ is derived. We will revise the draft to clarify that in the MAB framework, the Q-values are updated, and the optimal policy $\pi^*$ is implicitly derived from these Q-values rather than being directly optimized.

Typos:

1. Incorrect use of quotation marks: line 153-155,
2. Do B and C refer to the appendix? line 201 & 202 & line 318 & line 321 & line 365
3. Do you mean harmful or harmless? line 213-214: which assigns a binary reward indicating whether the response is harmful or unsafe.
4. see-D.0.1, see-D.0.2, see-D.0.3 and see-D.0.4 mean Appendix D.0.1, Appendix D.0.2, Appendix D.0.3 and Appendix D.0.4?

Response-to-Typos:

1. We thank the reviewer for pointing this out, we will fix this in the draft.
2. Yes, B and C refer to the appendix sections.
3. To clarify, we aim to state that the judge function $J$ assigns a reward of 0 when the input to it is safe/harmless and assigns a reward of 1 when the input to it is unsafe/harmless. We request the reviewer to look at section B.4 in the appendix for more details.
4. Yes they refer to the corresponding sections in the appendix.

We sincerely thank the reviewer for their constructive feedback in improving our manuscript. Below, we address each of the concerns and highlight the corresponding updates made to the manuscript. For ease of reference, we provide the section and line numbers where the revisions have been implemented. All new revisions to the manuscript are highlighted in red font.

1. Poor Readability

Reviewer Concern: The introduction lacks clarity and relevance. Key concepts such as MAB, policy $\pi$, and action-value $Q$ are introduced abruptly without explanation. Algorithm 1 is not well explained.

Response:

• Explanation of MAB: We have added a detailed explanation of the multi-armed bandit (MAB) problem in the

  Preliminaries Section (Section 3.1, Lines 175-184). This ensures readers are familiar with the concept before encountering its application in our work.

• Streamlined Introduction: We have revised the manuscript to include a detailed section on preliminaries (Section 3.1), where we thoroughly explain all the components necessary to understand our proposed method. Additionally, we have rewritten the introduction to improve its overall flow. This revision enhances clarity by clearly defining the problem we aim to solve and introducing the various components through toy examples, making the key concepts more intuitive and accessible to the reader.

• Clarification of Symbols: Explanations for policy $\pi$ (Section 3.1, Lines 192-204), action-value $Q$ (Section 3.1, Lines 192-204), and cost $C$ (Section 3.1, Lines 203-213) have been added in the Preliminaries section.

• Algorithm 1 Clarification: We have clarified $E_{explore}$ and $E_{exploit}$ in the Preliminaries section (Section 3.1, Lines 203–213), detailing their role in computing the cost function within our work. Additionally, we have included explicit explanations for these terms in Algorithm 1 (Lines 6 and 7). Furthermore, the SoC Attack Generation section has been rewritten to provide a clear, step-by-step explanation of both Algorithm 1 and Algorithm

2. Lack of Comparison

Reviewer Concern: Insufficient comparison with prior works and baseline jailbreak methods. CPP was introduced in the rebuttal but not in the manuscript.

Response:

• Comparative Analysis: We have included comparisons with state-of-the-art (SOTA) jailbreak methods in Section 5.5, evaluating our approach in terms of Attack Success Rate (ASR) and Cost Per Prompt (CPP). These metrics allow us to assess both the jailbreak performance and the time required to execute a jailbreak.

• CPP Definition: CPP (Cost Per Prompt) is now defined and described in Section 5.5 (Lines 491-493). Its role in demonstrating the effectiveness of our method is explicitly stated.

3. Few Experiments

Reviewer Concern: Limited experimental analysis and lack of generalization tests.

Response:

• We have added additional experiments validating the generalization capability of our proposed method. We request the reviewer to see section 5.6 for the corresponding results.
• We have also added experiments that compare the performance of the proposed judge function $J$ with SOTA judge functions from the Llama Guard Family in Section 5.1 Lines 411-412.
• We further provide details about the hyperparameter setting of our proposed method in Appendix. C
• We have also validated our method on LLMs of larger parameter size in Appendix. B

4. Question 2: Misstatement on LLM Safeguards

Reviewer Concern: The claim regarding classifiers in modern LLMs lacks citation.

Response:

• Citations Added: We have clarified our statement and added citations to justify the discussion of classifiers in LLM APIs in Section 3.1 (Lines 161-163).

5. Question 3: Policy Optimization in Algorithm 1

Reviewer Concern: The policy update mechanism for $\pi$ and $\pi^\star$ is unclear.

Response:

• Policy Update Explanation: We introduce $\pi$ and $\pi^{*}$ and explain their connection to the other key components of our proposed method in Section 3.1 (Lines 192–204). Furthermore, in Section 3.2, we elaborate on the relationship between $\pi$ and $\pi^{*}$. Finally, we have provided the mathematical equation representing the relation between $\pi$ and $\pi^{*}$ in Line 26 of Algorithm 1.

Dear Reviewer rSU9,

Thank you for your response and for increasing the score. We greatly appreciate your consideration.

In addition to the experiments already included in the manuscript, we have conducted an additional experiment using Cohere, a closed-source LLM. For more details, you can refer to our reply to reviewer pzyz's reply titled Response to Reviewer Suggestion linked here : https://openreview.net/forum?id=jCDF7G3LpF&noteId=dLVoAFtkQV

We would appreciate it if you could let us know if there are any further clarifications, experiments, or adjustments you would like to see in order to raise the score further, above the acceptance threshold. We are happy to make any additional improvements needed.

Thank you once again for your valuable feedback and support.

We sincerely thank you for your valuable comments, observations, and the time you have dedicated to evaluating our paper. We have carefully addressed your questions and the weaknesses you highlighted in a comprehensive manner. Our response is structured by presenting each weakness or question posed, followed by our detailed reply.

Weakness-1 This paper does not compare the proposed jailbreak attack with the existing methods.

Response-to-Weakness-1

In response to reviewer GXrd, we detail and interpret the comparison with existing SOTA jailbreak methods. We request the reviewer to refer to the Table RT 2 in Weakness-3 and the corresponding response Response-to-Weakness-3. We additionally also detail and compare the existing jailbreak methods based on their computational costs.

Weakness-2 The only theorem in this paper is almost trivial, and more importantly, the assumptions and limitations of this theorem are not discussed. It is encouraging to include theoretical analysis in LLM research. However, the results are not strong enough to serve as "one of the main contributions" (as stated in Line 253) of an ICLR paper.

Response-to-Weakness-2

Thank you very much for taking the time to review our work. Your feedback is invaluable in helping us refine and strengthen our research.

We acknowledge that our theoretical results are derived within a standard Multi-Armed Bandit (MAB) framework, and some results, such as Lemma 1, leverage established techniques in MAB theory. We have cited the relevant literature in sections where these results are presented to provide context and clarity.

We emphasize that the results in this paper are original and tailored to our unique setting, drawing inspiration from MAB literature but enriched with new insights to suit our framework.

Unlike much of the existing ad hoc jailbreak literature in black-box settings, which lacks theoretical analysis, our work establishes a formal decision-making framework grounded in MAB theory. This principled approach provides predictive and explainable insights into the success of context-switching strategies, advancing beyond mere empirical observation.

To the best of our knowledge, this is the first application of MAB to a multi-step attack scenario using context-switching strategies in LLMs. This novel approach introduces a structured decision-making framework for conducting multi-step SoC attacks, systematically exploring and exploiting context-switching queries (CSQs) to maximize jailbreak success in minimal steps, addressing a critical gap in the field.

Non-Triviality of Theorem 1 in the Context of SoC Attack Strategy

In our Multi-Armed Bandit (MAB) framework with $\epsilon$-greedy exploration, deriving the bound $\mathbb{E}[N_i] \leq \frac{2 \log T}{\Delta_i^2}$ in the context of SoC attacks introduces unique challenges that differ from traditional applications. Specifically:

• Sequential Dependency in Rewards: Although, we adopt a standard MAB setting, the SoC attack framework inherently introduces sequential dependencies in the rewards across CSQs. Each CSQ in the prompt sequence interacts with prior context switches, influencing the language model's responses. This sequential reward dependency deviates from the usual MAB assumption where each action’s reward distribution is independent of prior actions, making it non-trivial to derive tight bounds on the expected number of selections of suboptimal CSQs.

• Evolving Reward Gaps $\Delta_i$: In SoC-based attacks, each selected CSQ subtly modifies the LLM's context, potentially altering its reward patterns over time. As a result, the gap $\Delta_i$ between the optimal and suboptimal CSQs may experience slight shifts throughout the sequence. This dynamic aspect introduces additional complexity in bounding the expected number of times suboptimal CSQs are selected, as it requires accommodating potential variations in $\Delta_i$ rather than assuming it remains constant, as in traditional MAB problems.

Therefore, bounding $\mathbb{E}[N_i]$ in this context is non-trivial, as it extends the standard MAB theory by accounting for the cumulative effects of sequential context switching. This result is particularly important for optimizing SoC attack efficiency, ensuring minimal time is spent on ineffective CSQs.

Limitations for Theorem-1

Limitation 1: Rapid Convergence in Low Variance Settings

Suppose all CSQs $i$ have reward variances close to zero, i.e., $\text{Var}(R_i) \approx 0$, with small reward gaps $\Delta_i$ between them. In such cases, the convergence to the optimal CSQ could be artificially accelerated.

For a given $\Delta_i$, the bound $\mathbb{E}[N_i] \approx \frac{\log T}{\Delta_i^2}$ could become loose if $\Delta_i$ is very small, resulting in excessive early exploitation of suboptimal CSQs. This Limitation reveals that our bound depends heavily on the assumption of a sufficiently large reward gap $\Delta_i$, which may not hold in low-variance reward settings. This suggests a need for adaptive bounds that tighten based on observed variances.

Limitation 2: Suboptimal CSQs with Near-Optimal Rewards

Consider suboptimal CSQs $i$ for which $\Delta_i \to 0$ as $i$ approaches the optimal CSQ reward $\mu^*$. Here, the probability of selecting a suboptimal CSQ due to its close reward $\mu_i$ increases significantly. Mathematically, the probability of selecting a suboptimal CSQ is:

$\mathbb{P}(\hat{\mu}_i \geq \hat{\mu}^*) \leq 2 \exp\left(-\frac{n_i \Delta_i^2}{2}\right)$

As $\Delta_i \to 0$, $\exp\left(-\frac{n_i \Delta_i^2}{2}\right) \approx 1$, suggesting that suboptimal choices might persist even with large sample sizes. This reveals a limitation: for very small reward gaps, the theorem’s exponential convergence rate weakens, and the attacker may struggle to differentiate between near-optimal CSQs.

Limitation 3: Misclassification in the Judge Function

The judge function provides a binary reward $r_t \in \{0, 1\}$ based on whether a response is harmful. However, suppose the judge function misclassifies with an error rate $\delta$, giving the wrong binary classification with probability $\delta$. This can be modeled as an added noise term in the reward:

$\tilde{R}_t = R_t + \epsilon_t, \quad \epsilon_t \sim \text{Bernoulli}(\delta)$

The expected reward now has an error-adjusted bound:

$\mathbb{E} \left[ \sum_{t=1}^T \tilde{R}_t \right] \approx T \mu^* - O \left( \frac{K \log T}{\Delta_{\min}} + \delta T \right)$

where $\delta T$ represents the added regret due to judge misclassifications. This indicates that errors in the judge function directly affects the convergence and optimality of the CSQ selection process, requiring the attacker to account for these potential misclassifications when evaluating rewards.

Non-Triviality and Limitations of the Corollary in the SoC Attack Framework

This corollary provides critical insight into the efficiency of the SoC attack generation process, specifically in terms of minimizing the number of selections needed to identify and consistently exploit the optimal CSQ for each Direct Malicious Query (DMQ). The Non-Triviality arguments and Limitations for this corollary are same as that of Theorem-1 as it is directly derived from it.

Non-Triviality and Limitations of the Lemma-2 in the SoC Attack Framework

This lemma is non-trivial and significant for the following reason:

• Effect of Judge Misclassification on Cumulative Reward: In SoC attack setting, the judge function can introduce misclassifications, leading to noisy rewards. Lemma 2 incorporates the judge misclassification rate $\delta$, accounting for the accumulated impact of such noise on the expected reward. This feature is crucial in our setting, as it formalizes how errors in reward signals impact the efficiency of the attack over multiple iterations. To the best of our knowledge, in LLM Jailbreaking setup, there does not exist any literature that studies the effect of misclassification of Judge function. This part makes it non trivial and an important contribution from a theoretical perspective

Limitations for Lemma-2

Limitation 1: High Judge Misclassification Rate ($\delta$ is Large)

When the judge function’s misclassification rate $\delta$ is high, the correction term $\delta T$ increases, leading to a substantial deviation from the optimal reward:

$\mathbb{E} \left[ \sum_{t=1}^{T} \mu_{a_t} \right] \geq T \mu^* - O \left( \frac{K \log T}{\Delta_{\min}} + \delta T \right)$

In this scenario, the lemma highlights that a high misclassification rate by the judge function significantly lowers the expected total reward. This limitation underscores the importance of a highly accurate judge function to maintain effective SoC attacks, as increased noise in the reward signal can undermine the effectiveness of the algorithm.

Limitation 2: Small Reward Gap ($\Delta_{\min} \to 0$)

If the smallest reward gap $\Delta_{\min}$ approaches zero, the regret term $\frac{K \log T}{\Delta_{\min}}$ grows large:

$O \left( \frac{K \log T}{\Delta_{\min}} \right) \to \infty \quad \text{as} \quad \Delta_{\min} \to 0$

In this case, the lemma reveals that the algorithm may accumulate substantial regret, leading to a lower expected total reward. This limitation implies that the convergence of the total reward to the optimal reward is most effective when there is a significant reward gap between the optimal and suboptimal CSQs.

Limitation 3: Early Stopping and Limited Episodes ($T$ is Small)

For a small number of episodes $T$, the bound on the expected reward may not tightly approximate the optimal reward:

$\mathbb{E} \left[ \sum_{t=1}^{T} \mu_{a_t} \right] \geq T \mu^* - O \left( \frac{K \log T}{\Delta_{\min}} + \delta T \right)$

In this case, the bound shows that the algorithm requires a sufficient number of episodes to approach the optimal reward reliably. For short episodes or early stopping, the cumulative effect of suboptimal choices and judge misclassifications may result in a lower reward, affecting the efficiency of SoC attacks in limited settings.

Weakness-3: In brief, the authors claim three main contributions: creating a dataset, proposing a novel jailbreaking attack strategy, and providing a theoretical analysis. However, I think contributions (ii) and (iii) are slightly overstated. That is why I gave a score of 5 for this paper.

Response-to-Weakness-3:

Thank you for your valuable feedback. We appreciate the opportunity to clarify the significance of our contributions, particularly contributions (ii) and (iii), which focus on the novel attack strategy and its theoretical underpinnings.

While we understand that many existing black-box jailbreaking methods focus primarily on the practical success of prompting techniques, we believe our approach provides a structured, explainable framework that addresses a critical gap in current research. Here are key points that highlight the unique advantages and value of our MAB-based SoC attack methodology:

1. Explainability and Insight into Attack Dynamics: Our work frames the problem within a MAB setup, enabling a theoretically grounded analysis of context-switching decisions. Unlike existing methods, that rely on trial-and-error prompt engineering, our theoretical results explain why certain sequences succeed in jailbreaking, providing insights that are valuable for both refining attacks and designing effective defenses.

2. Efficiency and Theoretical Rigor in Strategy Optimization: By leveraging the $\epsilon$-greedy MAB framework, our approach quickly converges to optimal context-switching strategies with minimal exploratory overhead. The derived bounds, such as limiting suboptimal CSQ selections, enhance the attack's efficiency and effectiveness. These theoretical contributions provide a rigorous foundation to explain the empirical success of our method, addressing a gap in the largely ad-hoc approaches dominating current jailbreaking literature. See Response, Response-2 to reviewer UGpw for more details.

Weakness-4: This paper contains too many acronyms. I suggest adding a list of acronyms in the appendix. Besides, the authors do not provide a detailed explanation for some of the terms (e.g., sequence of context attack, and context switching queries) at their first appearance. It would make this paper more easy to follow if the authors add some explanations for the terms and reference them (e.g., see Section xxx for detailed discussions) when the term is mentioned for the first time.

Response-to-Weakness-4)

We thank the reviewer for their constructive suggestion to improve the readability and presentation of the paper! We agree that, this will improve the paper. We will add the following Table RT4 in the appendix section and will modify the text of the main paper to incorporate the reviewer’s suggestion for providing references to tie the various sections of the paper together.

Table RT4 : Table of Acronyms

Weakness-5 The citation style (i.e., the green boxes) is dazzling.

Response-to-Weakness-5) Thanks for the suggestion! We will modify the color of the citations in the main paper and appendix.

In response to the reviewer’s concerns, our contributions include:

• A comparison of the proposed jailbreak method with SOTA methods in terms of ASR and CPP.
• Clarification of the significance of Theorem 1, emphasizing the complexities involved in its derivation, and a discussion of its potential limitations to provide a comprehensive understanding of our theoretical contributions.
• Presentation of Lemma 2 as a critical result in SoC attacks, offering a bound on expected cumulative reward while accounting for exploration costs and judge misclassification errors. This analysis highlights practical limitations such as high misclassification rates, small reward gaps, and limited episode lengths, essential for understanding the efficiency of SoC attacks.
• Enhancements to improve the presentation and readability of the paper.

We hope the technical clarifications, additional experiments, and justifications we provided have addressed your concerns. If you find the responses satisfactory, we kindly request you to consider raising the score above the acceptance threshold. Please let us know if there are any remaining issues or additional details you would like us to address, which we would do it right away.

We sincerely thank you for your valuable comments, observations, and the time you have dedicated to evaluating our paper. We have carefully addressed your questions and the weaknesses you highlighted in a comprehensive manner. Our response is structured by presenting each weakness or question posed, followed by our detailed reply.

Weakness-1 : Compared to other automatic jailbreak attacks (e.g., GCG [4], PAIR [2]), this method requires dataset collection and policy model training, making it more resource-intensive and time-consuming

Response-to-Weakness-1 :

• The proposed method reformulates the multi-turn jailbreak problem as a sequence of context-switching actions aimed at steering an LLM into a jailbroken state with minimal steps using a multi-arm-bandit framework.
• A policy is learned to determine the optimal action at each step by evaluating the action values based on binary rewards from previous steps. The policy uses an ε-greedy strategy to balance exploiting known actions with maximum value and exploring new actions for potentially better outcomes.
• Our proposed method jailbreaks the LLM in lesser time than state-of-the-art (SOTA) methods due to the policy training stage, as evidenced by average time taken for jailbreak (cost-per-prompt, CPP) reported in Table RT2.
• To enable fair comparisons across LLM models, a dataset of context-switching prompts is created for evaluation. Alternatively, context-switching prompts can be dynamically generated at each step, at the cost of an additional LLM API call per step.
• Unlike white-box methods, which rely on accessing LLM parameters and gradients to search for adversarial inputs prompts, the proposed method requires only API calls, making it computationally less expensive. This results in improved performance w.r.t to CPP, as demonstrated in Table RT2.
• The method uses Multi-Armed Bandits (MAB) to efficiently learn optimal context-switching sequences, requiring an initial training phase but enabling generalization across malicious query categories. Unlike GCG or PAIR, it is mathematically robust (Theorem 1), minimizes unnecessary steps, and ensures scalable, efficient jailbreaks with significant computational benefits.

Weakeness-2 : The proposed method is limited to pre-defined harmful query categories, and its extensibility to unseen categories is not thoroughly investigated

To evaluate the generalizability of our approach, we perform 5-fold cross-validation using 18 harmful query categories (DMQs) which are mentioned in the main paper in section 3.1. In each cross validation fold, the proposed method is trained on a subset of these DMQ (harmful-query-categories) categories and tested on unseen ones. The Attack Success Rate (ASR) for each fold are reported in Table RT1.

The cross-validation results demonstrate the robustness and generalizability of the approach, with the Attack Success Rate (ASR) consistently high (0.905 to 0.933, average 0.9214), showing effective generalization to unseen harmful query categories.