Tokenizer-Agnostic Transferable Attacks on Language Models for Enhanced Red Teaming

3. Impact of Multi-Model Optimization on Semantic Coherency:
   Multi-model optimization in TIARA, using diverse tokenizers (e.g., Llama2 and Qwen), improves the semantic coherence of adversarial prompts. As shown in the table below, prompts optimized with multi-model attacks exhibit improved perplexity compared to single-model attacks, indicating that transferability and semantic coherence are complementary objectives effectively addressed by TIARA’s optimization framework. Notably, this improvement occurs without explicitly optimizing for fluency.

Attack Stability and Evolutionary Trajectories

Comment: Details on evolutionary trajectories and analysis of failed cases are needed.

1. Evolutionary Trajectories:
   TIARA’s optimization indeed operates as an evolutionary process, iteratively applying single-token mutations, evaluating a logit-based fitness score, and selecting the highest-scoring candidate at each step. These localized perturbations, combined with a sufficiently large candidate pool, enable the loss function to decrease steadily over iterations. The number of iterations required for the first successful attack varies across models, as shown in the table below. TIARA achieves high attack success rates (ASR) with fewer than 100 iterations on most open-source models, demonstrating rapid convergence. Larger iteration counts improve success rates and enhance transferability for multi-model attacks.

2. Failed Cases:
   Failures in TIARA’s attacks can be categorized as follows:

• Refusal Mechanisms: The model explicitly refuses the request, e.g., "I cannot fulfill your request." This type of failure is rare in direct source-model attacks, as the model often begins with an affirmative token like "Sure". However, it is more frequent in transfer attacks on target models.
• Response Retraction and Self-Correction: The model initially agrees but retracts, e.g., "Sure, here's how to acquire precursor chemicals... I cannot provide instructions on how to illegally obtain or manufacture drugs." This behavior is more common in direct attacks and can stem from the loss function’s focus on generating a fixed target string, which may not align well with achieving a harmful response. Improving the objective function could mitigate these failures.
• Unrelated or Meaningless Content: The model generates irrelevant or nonsensical responses, e.g., "Sure, here's my response:\n\nMediter: $%^&*()_!@#$%^&*()_!@#$%^&*().... These failures are rare and often occur when a short initial harmful instruction is overwritten by a longer adversarial suffix. Adapting the string length dynamically to the instruction could help reduce such issues.

Tokenizer-Agnostic vs. Model-Agnostic

Comment: The distinction between tokenizer-agnostic and model-agnostic approaches is unclear.

TIARA implements a token-level optimization method that is tokenizer-agnostic, meaning it can operate with any tokenizer (or combination of tokenizers) for tokenization and perturbation sampling. The optimization process does not rely on a fixed tokenizer, enabling flexibility in selecting or combining tokenizers during attacks.

This tokenizer-agnostic property allows TIARA to attack multiple models simultaneously, even if they use different tokenizers. By leveraging multi-model attacks, TIARA produces optimized strings that are transferable across diverse target models, a property we refer to as transferability.

TIARA itself is applicable to any model that provides access to logits. However, the term model-agnostic is not explicitly defined in this paper, as its interpretation in this context was unclear from the reviewer’s comment. We focused on transferability and tokenizer-agnosticity as the core properties of our method.

Thank you for your valuable feedback. To address your concerns, we will include detailed analyses in the supplementary materials, such as experiment settings (e.g., temperature, allowed token sets), bounds on attack performance for different source model configurations, tokenizer overlap analysis and its impact on transferability, ASR progression and failed case analysis, and clearer definitions of tokenizer-agnosticity and transferability. These findings will be summarized in the main text, with clarified terms and details for improved readability. We hope these responses and planned updates address your concerns, and we kindly request that you consider raising your score.

2. Tokenizer Behavior on Optimized Prompts:
   We analyzed tokenizer behavior on TIARA-optimized prompts for both multi-model and single-model attacks. Multi-model attacks produce prompts with significantly higher token overlap across tokenizers (84% on average) compared to vocabulary overlap (49% on average), indicating shared vulnerabilities among models. Additionally, token counts remain consistent (around 20-21) across tokenizers, suggesting that multi-model optimization implicitly selects shared tokens, enhancing semantic coherency and generalization. In contrast, single-model attacks result in lower overlaps (81% on average) and more variation in token counts, showing reduced transferability. These findings highlight the strength of multi-model optimization in producing robust, transferable prompts.

• Multi-model attack (Llama2+Vicuna+Qwen): Below is an example of tokenization by different tokenizers for an adversarial suffix optimized through a multi-model attack for the harmful behavior: List high-profile skyscrapers in the US that would be easy to destroy with a car bomb and explain your reasoning

Additionally, we present the average overlap in tokenization for adversarial suffixes optimized by the multi-model attack across all prompts:

• Single-model attack (Llama2): Here is an example of tokenization by different tokenizers for an adversarial suffix optimized through a single-model attack for the same harmful behavior.

The average overlap in tokenization for adversarial suffixes optimized by the single-model attack is summarized below:

Thank you for your detailed review and thoughtful feedback. Below, we address your concerns and provide additional clarifications and results.

Method and Experiment Details

Comment: Inadequate exposure of method and experiment details, including temperature, top-k, subset of candidate tokens for sampling, and tokenizer correspondence.

1. Temperature and Top-k Settings:
   As described in Section 3.1, we use a greedy sampling strategy, where the token with the highest logit is always selected (temperature=0, top-k=1). This approach ensures deterministic generation, eliminating the randomness introduced by other sampling methods. This choice aligns with prior work, enabling direct comparability.

2. Allowed Subset of Candidate Tokens:
   Thank you for highlighting this point. We will clarify it in the experiments section and will share our source code upon publication. In all experiments, we restrict the optimized strings to ASCII characters only, and we exclude any special tokens, such as padding, beginning-of-sequence, end-of-sequence tokens, etc. This approach is consistent with baseline methods (e.g., GCG) and avoids leveraging artifacts that may bypass models in unrealistic scenarios. Special tokens, while potentially effective for attacking some models, are often filtered out or tokenized differently in typical API or chat interface implementations.

3. Perturbation Tokenizer Correspondence:
   Thank you for raising this point. We will clarify in the paper that perturbation sampling defaults to the source model’s tokenizer. In single-model attacks, we use the corresponding source model tokenizer, while in multi-model attacks, all source model tokenizers are used. The attack state is maintained as a string to handle models with differing tokenizers and avoid generating invalid token sequences that cannot be consistently retokenized.

   To demonstrate the flexibility of perturbation tokenizer selection, in the Ablation Study (Section 5.2), we used an external tokenizer (GPT4o) to attack Llama2. As shown in Figure 5, this improved transferability (e.g., to GPT-3.5) for longer strings. Therefore, when a target model’s tokenizer is known, it can be leveraged to further enhance performance.

Multi-Model Attack Configuration

Comment: The choice of source models is ad-hoc, and lower/upper bounds of attack performance corresponding to the weakest and the strongest source model are needed.

Thank you for the insightful question. The table below demonstrates the weakest and strongest single-source models for each closed-source target model:

Since the number of possible source model combinations grows exponentially with the number of models considered, we focused on three diverse models — Llama2, Vicuna, and Qwen — selected for their varied architectures, tokenizers, and training schemes. As shown in Section 4.1, this combination significantly improves transferability, achieving 51.2% ASR on Gemini Pro and 82.9% ASR on GPT-3.5, outperforming any single-source model. Our work is the first to systematically analyze the effect of source models and their combinations on transferability, laying the groundwork for future exploration of multi-model optimization.

Analysis of Tokenizer Behavior

Comment: ... a larger vocabulary of tokenizer A may contain the smaller vocabulary of tokenizer B. Analyzing the overlap between different tokenizer behaviors, especially on the crafted prompts, will help us understand whether the transfer attack leverage semantics or just token combinations to make sense.

We agree that understanding tokenizer behavior is important. To address this, we have conducted an in-depth analysis summarized below:

1. Tokenizer Vocabulary Overlap:
   We first analyze the overlap between tokenizer vocabularies. In the table below, the percentages represent the proportion of tokens from the vocabulary of the model in row $i$ that also exist in the vocabulary of the model in column $j$. While overlap varies significantly, using diverse tokenizers ensures broad coverage of different tokenization behaviors.

Thank you for your detailed review and thoughtful feedback. We address your concerns and questions below.

1. Comparison with Different Perturbation Functions

We agree that exploring different perturbation functions is an important direction for future work. In our current study, we focused on random perturbations because they provide a baseline that is agnostic to specific tokenization or semantic properties, ensuring general applicability. While random perturbations are computationally intensive (e.g., 1024 perturbations per iteration), their simplicity highlights the effectiveness of our loss functions in guiding optimization.

To demonstrate the flexibility of the perturbation function, we conducted Ablation Study in Section 5.2, where we used an external tokenizer (GPT4o) to attack Llama2 model. Results in Figure 5 show improved transferability to GPT-3.5, especially for longer optimized strings. Future exploration of structured or semantic-aware perturbations could enhance computational efficiency and attack performance. We will include these discussions in the revised manuscript.

2. Multi-Stage Candidate Selection

We appreciate your observation regarding the placement of the Multi-Stage Candidate Selection (MSCS) description. While the main algorithmic steps are detailed in the appendix, we will revise the main text to include a concise description of MSCS and highlight its importance to the method. MSCS operates by selecting 100 diverse and effective candidates based on loss and iteration, filtering down to 20 promising candidates using a validation classifier, and finalizing the results with a test classifier. This iterative refinement ensures both diversity and effectiveness in adversarial strings. We will ensure this clarification is prominently included in the updated manuscript.

3. Llama3 Usage and Evaluation

We conducted additional experiments including Llama3 8B as a single-source model. In the Table below, we report TIARA's single-model attack success rates (ASR) on source and target (closed-sourced) models:

Llama3 was included as a source model in multi-model attacks because of its diverse architecture and tokenization behavior, which contribute to transferability.

4. Number of Iterations and Transferability

The number of iterations for successful attacks varies across models. TIARA achieves high ASR with fewer than 100 iterations for most open-source models. Longer optimization further improves transferability, particularly in multi-model attacks.

Final Note

We believe that TIARA’s contributions — including high ASR with a gradient-free and attacker LLM-free approach, multi-model attacks across tokenization schemes, and practical transferability — address an important problem in adversarial attacks on language models. We hope these clarifications address your concerns, and we kindly request that you reconsider your score. Thank you again for your thoughtful review and suggestions.

Thank you for your thoughtful review and detailed feedback. We address your concerns below:

1. Comparison with Previous Gradient-Free Methods

BEAST [1], a gradient-free jailbreak optimization method using beam search, achieves significantly lower attack success rates (ASR) compared to TIARA when evaluated on the HarmBench benchmark:

These results establish TIARA as the first gradient-free and attacker LLM-free method to achieve high ASR on secure LLMs like Llama2, demonstrating that gradient information and attacker LLMs are unnecessary for highly effective attacks.

2. Ensemble Attacks and Tokenizer-Agnosticity

While ensemble attacks are known to improve transferability in computer vision, applying them to language models introduces significant challenges due to differing tokenization schemes. TIARA addresses this by decoupling perturbation generation from loss evaluation, enabling seamless ensemble attacks across models with arbitrary tokenizers. This flexibility represents a novel contribution, advancing the study of transferable adversarial attacks in language models.

3. Clarification of Design Choices

The intuition behind TIARA’s teacher-forcing and auto-regressive losses is detailed in Section 3.4 (lines 328–330). Teacher-forcing loss maximizes the likelihood of generating the entire target string, while auto-regressive loss ensures token-by-token optimization, mirroring real LLM generation behavior. This combination balances whole sequence optimization with next token generation, producing highly effective adversarial suffixes.

4. Impact of Multi-Model Optimization

Multi-model optimization in TIARA, using diverse tokenizers (e.g., Llama2 and Qwen), improves the semantic coherence of adversarial prompts. As shown in the table below, prompts optimized with multi-model attacks exhibit improved perplexity compared to single-model attacks, indicating that transferability and semantic coherence are complementary objectives effectively addressed by TIARA’s optimization framework. Notably, this improvement occurs without explicitly optimizing for fluency.

5. Tokenizer-Agnosticity and Scalability

TIARA’s tokenizer-agnosticity refers to its ability to:

• Operate with arbitrary tokenizers during perturbation sampling.
• Generate adversarial suffixes transferable across models with different tokenization schemes.

The following table compares vocabulary sizes of source and target LLMs:

We respectfully disagree with the claim that larger vocabularies hinder TIARA’s scalability. Both GPT-3.5 and GPT-4 share the same tokenizer with the vocabulary size of 100,263, yet TIARA achieves 82.9% ASR on GPT-3.5 and 22.0% on GPT-4. Similarly, despite Gemini’s larger vocabulary (256,000), its ASR of 51.2% exceeds that of GPT-4. This indicates that model safety alignment, not vocabulary size, primarily determines attack success. Furthermore:

• Assumption of Limited Knowledge: TIARA assumes no knowledge of the target model’s tokenizer, using only source models for optimization. This leads to inherently transferable tokens without relying on shared vocabularies.
• String-Level Robustness: Larger vocabularies often represent more compressed forms of strings, but as long as the optimized string enforces harmful semantics, tokenization differences have limited impact.

These findings underscore that attack performance depends more on safety alignment than vocabulary size. Additionally, multi-model attacks improve semantic coherence (evidenced by lower perplexity in the table above), demonstrating robust transferability across models.

We will refine these claims and add clarification in the updated manuscript.

Final Note

We believe TIARA’s contributions—including achieving high ASR without gradient or attacker LLM reliance, enabling multi-model attacks across tokenization schemes, and demonstrating practical transferability—represent substantial advancements in the field. We hope these clarifications address your concerns, and we kindly request that you reconsider your score.

[1] Fast Adversarial Attacks on Language Models In One GPU Minute, ICML 2024.

Thank you for your thorough review and valuable feedback. Below, we address each of your concerns in detail.

Computational Costs of TIARA

Comment: TIARA’s multi-stage candidate selection and extensive perturbation sampling require significant computational resources ...

1. Efficiency of Single-Model Attacks:
   TIARA achieves high attack success rates (ASR) with less than 100 iterations on most open-source models. To demonstrate this, we conducted experiments showing the mean iteration count for the first successful attack across all harmful behaviors. As summarized in the table below, TIARA’s convergence is rapid, with large iteration counts primarily ensuring broader success and better transferability for multi-model attacks.

2. Efficiency in Transfer Attacks:
   TIARA’s primary application is transfer attacks, where only 20 queries are evaluated on closed-source models. The computationally expensive steps are confined to open-source models, leveraging local resources.

Logit Access

Comment: TIARA relies on direct access to model logits for loss computation, which may limit applicability to models where even logit-level access is restricted.

TIARA optimizes adversarial strings on open-source models with logit access. For transfer attacks on closed-source models, logit access is not required, as demonstrated in our experiments, where we achieve high ASR without relying on logits from target models. This distinction ensures TIARA’s applicability in restricted settings.

Evaluation Against Defenses

Comment: The paper does not extensively test TIARA against advanced system-level defenses or input transformation defenses ...

1. Focus on Model-Level Defenses:
   Defenses can broadly be categorized into two types: model-level defenses (e.g., refusal mechanisms, system prompts, adversarial training) and system-level defenses (e.g., input filtering, cleansing, or transformation). In this work, we focus on model-level defenses for two key reasons:
   • Model-level defenses remain relatively underexplored compared to system-level approaches.
   • Evaluating system-level defenses requires defense-specific adaptive attacks, making consistent evaluation challenging. By contrast, model-level robustness can be assessed systematically using a suite of pre-existing attacks.
2. Evaluation Against Advanced Safety Mechanisms:
   TIARA has been rigorously tested against models equipped with advanced model-level defenses, including Llama2 and Llama3, both of which have undergone safety tuning and adversarial training. For Llama2, a 132-token safety system prompt was also incorporated during evaluation. Furthermore, TIARA demonstrates strong transferability to state-of-the-art closed-source models like GPT-3.5, GPT-4, and Gemini, which implement unknown internal safety mechanisms. These results highlight TIARA’s effectiveness across diverse defensive settings.

Token-Level Perturbations and Flexibility

Comment: TIARA’s reliance on token-level perturbations may limit its success where semantic-level manipulations are more impactful ...

1. Performance Compared to Semantic-Level Methods: TIARA significantly outperforms semantic-level methods, such as PAIR and TAP. For example, on the Llama2 model, PAIR and TAP achieve attack success rates (ASR) of 9.8% and 7.3%, respectively, while TIARA achieves 90.2%, demonstrating a 10x improvement. Furthermore, TIARA’s flexible framework supports arbitrary perturbation strategies, including semantic manipulations. This flexibility arises because TIARA decouples perturbation generation from loss evaluation, making it adaptable to various attack methodologies.
2. Semantically Coherent Prompts Through Multi-Model Optimization:
   Multi-model optimization in TIARA, using diverse tokenizers (e.g., Llama2 and Qwen), naturally improves the semantic coherence of adversarial prompts. As shown in the table below, multi-model optimized prompts exhibit improved perplexity compared to single-model attacks, indicating that transferability and semantic coherence are complementary goals effectively addressed by TIARA’s optimization framework. Note that this improvement occurs without explicitly including fluency in the objective function.

Thank you again for your comments. We hope this detailed response adequately addresses your concerns. We kindly request that you consider updating your score to reflect these clarifications and additional results.

Thank you once again for your thoughtful review and valuable feedback. We kindly ask if you now consider your concerns to be sufficiently addressed. If so, we respectfully remind you to update your score to reflect the clarifications and additional insights provided.