GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs

We appreciate Reviewer U7Y9 for taking the time to provide feedback on our submission. However, we found that the review's content appears to be entirely unaligned with the actual content of our paper. Specifically, the points raised do not reflect the problem setting, methodology, or contributions we presented, and no part of the review directly references any section, experiment, or claim from the submission.

To briefly reiterate, our paper introduces GASP, a novel black-box framework for generating coherent and highly effective adversarial suffixes using Latent Bayesian Optimization. We evaluate GASP across various LLMs and provide thorough analyses, including ablations, baselines, and generalization studies. None of these aspects was mentioned or reflected in the current review.

We kindly ask the reviewer to revisit the paper and consider updating both the review content and the score to reflect an evaluation of the actual submission. Should there be any confusion or ambiguity in our writing, we are more than happy to provide additional clarification. Thank you again for your time and consideration.

We thank the reviewer for their feedback and for recognizing the uniqueness of GASP. Below, we respond to each concern raised:

W1: Details in the appendix

We defer most of our implementation and evaluation details to the appendix, mainly due to space constraints. The main paper outlines key design decisions and intuitions behind each component. Still, we recognize the need for clarity and will include more technical details in the revised paper.

Regarding the readability metric: we follow a JudgeLLM-style setup, using a system prompt instructing a language model to score fluency on a 0–1 scale. While simplified here, our actual prompt is longer to ensure nuanced and consistent judgments.

---

W2 (Q1): Inconsistent attack setups & potential unfair comparisons

Below, we address concerns raised by the reviewer to clarify why our comparisons with baseline methods are both fair and consistent.

Baseline attacks. All compared optimization-based methods, PAIR, AutoDAN, TAP, and GCG, are evaluated in their prompt-specific modes, consistent with their default usage stated in their respective papers, since our goal is to evaluate prompt-wise attack quality. Although these methods support universal settings, performance degrades, and universal prompts limit adaptability to specific prompts.

AdvPrompter's configuration. AdvPrompter is evaluated using its warm-start config in both white-box and gray-box settings. As noted in our response to Reviewer NJDJ under W3: AdvPrompter's performance & evaluations, we use a transfer-based config only in the black-box transfer attack setup, consistent with the capabilities available in those scenarios. This ensures fair comparison with GASP under consistent assumptions.

AdvSuffixes vs. AdvBench. AdvPrompter is trained on AdvBench (as per the original paper's implementation), not AdvSuffixes. While AdvSuffixes includes the same 520 harmful prompts from AdvBench, its purpose is entirely different: it provides syntactically plausible suffixes to help SuffixLLM learn the structure of potential suffixes, not to inject harmful content. This dataset is designed to initialize SuffixLLM with a structural prior. As shown in our ablation (Figure 4a and W2 Table provided to Reviewer NJDJ), using AdvSuffixes alone yields low ASR, confirming it provides a weak prior, not an attack advantage.

Inference time comparisons (Figure 2). We clarify that our reported inference times are on a per-query basis. Since GASP is designed to generate suffixes tailored to each prompt, we report the time per suffix generation. This is consistent with how other methods are also evaluated, as all are used in prompt-specific settings, so the per-query timing comparison remains fair.

---

W3: Important missing comparisons (best-of-N jailbreaks)

We thank the reviewers for pointing out the two recent black-box jailbreak frameworks. We discuss the two methods and clarify below why direct comparison is often non-trivial due to fundamental differences.

Best-of-N Jailbreaking. This brute-force method relies on repeated random augmentations. While it can eventually succeed, it may require up to 10,000 queries per input, making it impractical under realistic query budgets. It also lacks readability, a core goal of GASP.

LIAR. LIAR uses Best-of-N sampling, generating many suffixes scored by an unsafe reward. While per-sample inference is fast, achieving high ASRs often requires thousands of generations per prompt, leading to high time-to-attack and query costs—especially problematic in black-box settings with limited budgets. Besides, LIAR’s effectiveness depends on the quality of reference LLM samples, which can also be a limiting factor.

Empirical comparisons. Due to the lack of an open-source LIAR implementation and time constraints during the rebuttal, we cannot fully reproduce its results. We are actively reimplementing LIAR and will include results in the final version if available during rebuttal. In all our experiments, we limit the number of iterations to 10 when evaluating GASP. Regardless, even with a larger number of augmentations, Best-of-N struggles to match the performance of GASP.

We evaluate all methods on 100 test prompts from AdvSuffixes, reporting ASR@1, @10, and @100 to reflect different query budgets. Due to time constraints, GASP’s ASR@100 is omitted for now but will be included in the final revision. As expected, Best-of-N underperforms in low-budget settings due to unguided, chance-based sampling. Notably, GASP exceeds Best-of-N’s ASR@100 using only ASR@10, highlighting its efficiency and effectiveness.

---

W4 (Q2): Lack of key ablation studies

We would like first to clarify that key ablations have already been provided in the main paper (Section 4.3) and in Appendices F.3 and F.4. Below, we conduct additional experiments to address each concern:

Impact of AdvSuffixes. While our two-shot prompting guides SuffixLLM to produce plausible suffixes, we also construct four AdvSuffixes variants:

• zero-shot,
• one-shot DAN,
• one-shot role-play
• one-shot STAN: using STAN (Strive To Avoid Norms) jailbreak to test beyond DAN and role-play examples.

We finetune SuffixLLM on each of these variants and report GASP's performance to evaluate the effectiveness of these settings.

GASP performs consistently well across all AdvSuffixes variants, with only slight ASR differences, showing AdvSuffixes acts mainly as a weak prior. LBO remains the core strength, adaptively guiding the search. Notably, the zero-shot variant produces slightly more diverse suffixes, possibly explaining its marginally higher ASR in some cases. We will include this ablation in the revision.

GASPEval's effectiveness. Regarding GASPEval, we already include evidence in Appendix E.1 and Figures 11–14 showing common failure cases in existing evaluation models. These examples underscore the limitations of other judges. More importantly, Figure 4b shows GASPEval outperforming StrongREJECT, providing more reliable feedback that enables LBO to achieve significantly higher ASR, underscoring its crucial role in guiding adversarial suffix search.

To assess each question’s impact on GASPEval’s scoring, we perform a leave-one-out ablation by removing one prompt at a time during LBO. For each variant, we rerun GASP on 10 prompts (on Mistral-7B-v0.3) and evaluate adversarial suffixes with ASR@10 and full GASPEval.

Removing any single GASPEval question generally lowers ASR, confirming that each aids optimization. Moderate drops show the set is robust and balanced, enabling LBO to address multiple harmfulness aspects for reliable attacks. Future work will identify correlated questions to enhance coverage and efficiency.

Perplexity comparisons. We conducted a new experiment comparing the perplexity of generated suffixes across leading frameworks, further confirming that GASP produces more fluent, human-like outputs consistent with our readability claims.

---

W5 (Q3): Difference in train/test splits of AdvSuffixes

The creation of train and test datasets is provided in Appendix B of our supplementary materials. We use 100 out-of-distribution harmful prompts for testing GASP’s generalization to diverse, unseen prompts. We acknowledge that syntactic differences don’t guarantee semantic novelty, so we conduct additional experiments to clarify.

Distribution differences. We measure token distribution divergence using KL and JS metrics computed via a pretrained tokenizer (Llama-3.1-8B). These moderate divergence values confirm that test prompts are distinct from training ones, indicating limited harmful content overlap and supporting evaluation robustness.

Jensen-Shannon (JS) divergence: 0.368

KL divergence: 0.155

Ablation study with AdvBench. Additionally, we run a targeted ablation study on AdvBench by training SuffixLLM using only 420 prompts and evaluating GASP on the held-out 100 prompts. GASP still achieves strong ASR, demonstrating that our method generalizes well to unseen harmful intents and does not rely on memorized training prompts.

---

We again thank the reviewer for their thoughtful feedback and detailed review. We will also revise our figures to be clearer and more readable. If our responses resolve your concerns, we kindly ask you to consider increasing your score. Otherwise, we are happy to offer any additional explanations you may need.

We would like to thank the reviewer NJDJ for acknowledging our work's clear presentation, the novelty in using LBO for efficient latent space exploration, and the comprehensiveness of our experiments. Below, we respond to each concern raised in the weakness section.

W1: GASP is complex and has many subcomponents

While our method does consist of multiple components, each was deliberately chosen to address distinct challenges inherent to adaptive black-box jailbreak generation, a problem setting that is both hard and underexplored. Importantly, GASP is modular, and all components are plug-and-play, requiring no gradient access or end-to-end fine-tuning. In practice, we found that each module requires minimal hyperparameter tuning and generalizes well across TargetLLMs, as shown by our ablations in Figure 4, where removing any component significantly reduces performance.

We therefore argue that the complexity is not only justified but necessary to achieve state-of-the-art black-box jailbreak performance, and GASP remains practical, scalable, and reproducible under realistic attacker assumptions.

W2: AdvSuffixes and its impact on performance

To clarify, the AdvSuffixes dataset is not responsible for the high ASR of our method. As shown in our ablation in Figure 4(a) and the table below (where we additionally evaluated Llama-2-7B), using AdvSuffixes alone without the LBO & ORPO results in very low ASR improvements, indicating that the dataset does not drive the attack performance. It only serves to initialize SuffixLLM with a lightweight prior on suffix structure and improve its ability to produce syntactically and semantically meaningful suffixes.

We report ASR@1 and ASR@10 in the table above. The full GASP pipeline significantly boosts ASR, confirming the core methodology, not the dataset, drives performance.

Moreover, we want to emphasize the construction process of AdvSuffixes is both inexpensive and lightweight. It does not require human annotations or model-specific jailbreaks; we generate it automatically to produce syntactically plausible but harmless suffixes that loosely resemble adversarial ones. The setup is one-time and reusable across target models, requiring only a few minutes of compute on a modest GPU.

As no existing dataset captures adversarial suffix generation in our setting, AdvSuffixes is also a valuable contribution we hope can serve as a standard benchmark and initialization tool for future work in black-box LLM jailbreak research.

W3: AdvPrompter's performance & evaluations

We agree with the reviewer that AdvPrompter is a relevant baseline. However, it is important to note that AdvPrompter operates in a grey-box setting, relying on access to model internals during training, whereas our method is strictly black-box. Since AdvPrompter cannot operate in a fully black-box setting, we follow the transfer attack protocol: we train AdvPrompter on a white-box model (Mistral-7B-v0.3) and evaluate the generated suffixes on GPT-4o and Claude-3.7-Sonnet, reflecting the same transferability principle used in the original paper.

The results in the table report ASR@1 and ASR@10 for both methods. GASP demonstrates stronger and overall higher success in the black-box setting.

For the evaluation dataset, we clarify that all experimental configurations, including dataset details, are specified in Section 4 under the Dataset heading. The experiments in Figure 3(a) utilize 100 out-of-distribution prompts from AdvSuffixes, created explicitly for evaluation purposes (same setup used in our other experiments). Further details on their construction can be found in Appendix B of our supplementary materials. This setup is intentionally designed to rigorously test generalization performance in adversarial settings, making it particularly suitable for evaluating jailbreak methods that could potentially overfit to its training data.

W4: AmpleGCG's performance

We appreciate the reviewer's suggestion to include a comparison with AmpleGCG. While both GASP and AmpleGCG employ generative mechanisms, a direct comparison is non-trivial and potentially misleading due to key methodological differences. AmpleGCG relies heavily on collecting and filtering many successful adversarial suffixes via GCG from an LLM. These suffixes are then used to train a separate generative model, which is subsequently used to sample candidate suffixes, making it a sampling-based approach to red-teaming. In contrast, GASP is an optimization-based framework utilizing LBO and black-box feedback via GASPEval. Besides, the adversarial suffixes generated by AmpleGCG are GCG-like, meaning that their readability scores are much lower than ours.

In particular, the key challenge in a fair comparison lies in aligning attack cost and evaluation granularity:

• AmpleGCG's overall query cost includes (i) the thousands of queries made during GCG to "overgenerate and filter" (OTF) training data and (ii) additional decoding-time queries for sampling and evaluation. The original paper does not detail how to partition the query budget across these phases, making comparing methods on a unified query-efficiency axis difficult.
• On the other hand, GASP's query cost is controllable, as we explicitly fix the number of queries made to the TargetLLM. This is governed by the number of training epochs multiplied by the number of times LBO selects a new candidate suffix. This design makes GASP more transparent and adaptable in query-limited settings, crucial in realistic attack scenarios where access to the target model is restricted or metered.

Furthermore, AmpleGCG's evaluation is not directly comparable to GASP, as it is not optimization-based and does not use ASR@1 or ASR@10. Instead, AmpleGCG generates a large number of suffixes via sampling (via Group Beam Search to find successful diverse suffixes) and reports overall ASR across multiple prompts. In contrast, GASP explicitly optimizes suffixes and can report ASR@1 or ASR@10, reflecting the number of harmful prompts that can lead to successful jailbreaks. These fundamental differences make it difficult to perform a fair, apples-to-apples comparison.

To address the reviewer’s request, we present AmpleGCG’s results trained over 64000 GCG-generated suffixes from 50 prompts (via LLaMA-3.1-8B), along with total time taken and queries to the TargetLLM, including suffix generation, while conducting inference on 100 prompts from AdvSuffixes.

While we report AmpleGCG's results here for completeness, we emphasize that this is not a fair comparison. AmpleGCG benefits from extensive offline training on 64000 GCG-generated suffixes from a powerful open-source model, a setting that assumes access to strong white-box generators and large computational budgets. In contrast, GASP operates in a fully black-box, online setting, generating adversarial suffixes with around 30x fewer queries and 2x less time. This makes GASP significantly more query-efficient, practical, and deployable under realistic attacker constraints. Moreover, AmpleGCG shows weaker transferability to black-box models like GPT-4o compared to GASP. We encourage future work to establish common benchmarks that normalize for TargetLLM query cost, which we believe is the most faithful metric for comparing attack efficiency in black-box settings. We will add this discussion to our revised paper.

W5: Creation details of AdvSuffixes

We provide the full creation details of AdvSuffixes in Appendix B. The dataset itself is also included in the supplementary materials zip and the anonymous code repository linked in the submission. To briefly summarize, AdvSuffixes consists of syntactically plausible but harmless suffixes generated using an uncensored language model guided by templated prompts. These suffixes are designed to loosely resemble adversarial completions in structure and serve only to provide SuffixLLM with a minimal prior over the style of suffixes, not to inject harmful behavior or leak task-specific information. Please refer to the table and our response to Reviewer sW9s under the heading Impact of AdvSuffixes corresponding to W4: Lack of key ablation studies for concrete evidence and detailed discussions. The construction procedure of AdvSuffixes is lightweight, reproducible and task-agnostic, making it a robust and reusable component of our framework.

---

Q2: Dashes in Tables 2 and 3

The dashes in Tables 2 and 3 indicate frameworks where ASR@10 was not reported due to the high computational cost of running multiple attacks per prompt. For these optimization-based baselines, generating a single adversarial suffix is already expensive, making ASR@10 computationally costly. We report ASR@1 instead, which still provides a fair and informative comparison.

---

We again thank the reviewer for their thoughtful feedback and detailed review. If your concerns have been addressed adequately, we kindly ask you to consider increasing your score and evaluation; otherwise, we would be happy to provide further explanation.