Thank you for finding our framework impressive! Here, we briefly compare h4rm3l to [1] and [2]. Our common response above both clarifies that h4rm3l can represent any jailbreak attack on black-box LLMs without restriction, and shows how h4rm3l can be used to represent and expand upon [3].

Our work is similar to [1] (LLM-Fuzzer) in that we iteratively improve an initial set of attacks. However, h4rm3l differs because it represents attacks in a formal composable language that includes all string transformations, and hence all black-box attacks. String transformations include, but are not limited to templated transformations such as the ones illustrated in ([1], Fig 3), which add a constant prefix or suffix to the original string. Our synthesis approach is also different from [1], as [1] produces templates while our approach produces h4rm3l programs, which are compositions of primitives including templated primitives and primitives that can perform arbitrary computation. For instance we implemented Kang et al. (2023)'s payload splitting attack in h4rm3l as the PayloadSplittingDecorator, which performs character-level manipulations of the input prompt (See Appendix B. for other state-of-the-art black-box jailbreak attacks in h4rm3l)

We agree with [2]'s definition of jailbreak attacks. But further, we propose a formal representation for "strategic manipulation of input prompts": h4rm3l programs, which are compositions of parameterized string transformation primitives. [2] is similar in spirit to our efforts: to systematically scale the generation of jailbreak attacks, learn from human jailbreakers, and understand the characteristics of attacks. However, we approached the problem differently. We first built basic extensible composable abstractions that completely cover the space of black-box attacks, and then introduced primitives and examples from previously published work into our framework. From that pool of initial formalized attacks, we employed program synthesis methods to generate new attacks in the same formal representation.

Our approach also enables possibilities that approaches such as ([1], [2], and [3] do not. Attacks expressed in h4rm3l are algorithmic, human-readable and can be quantitatively (Section 4) and qualitatively (Section 5) analyzed at scale; which enables progress in understanding LLM safety guardrails. For instance, by analyzing thousands of synthesized attacks, we showed that the success rate of attacks could grow with their compositional complexity, and that the distribution of primitives in successful attacks is target-LLM-specific (Fig 6).

Thank you for referencing [1], [2], and [3], which have enriched our paper's discussion of related works.

We will attempt to make the prose more concrete and connected.

• Section 3.1 Introduces the h4rm3l language, which represents jailbreak attacks as compositions of parameterized string transformation primitives.

• Section 3.2 Formulates the synthesis of jailbreak attacks as a program search problem over the space of programs in the language described in Section 3.1 with the objective of maximizing the ASR of synthesized programs. Specifically, the attack proposals generated by generateProposals at line 226 of the manuscript are compositions of primitives in h4rm3l, introduced in Section 3.1. In this section, we also proposed bandit-based few-shot program synthesis algorithms towards this objective.

$**Re: "more details about the generic decorator TransformFxDecorator" and **$ $**"concerned that the author's claim of “representing all black box attacks” is an overclaim."**$

Please see our common response above, which addresses this point.

$**Re: how are the selected jailbreak methods combined in this paper?**$

• h4rm3l enables arbitrarily complex composition of attack primitives. Technically, jailbreak attacks (including single primitives) are composed using their $.then()$ function. (See section 3.1).

• To generate new jailbreak attacks, we prompt an LLM with a set of examples of jailbreak attacks expressed in h4rm3l, and instructions to generate new ones using the rules and primitives of the language, which are supplied in the same prompt. These new attacks are compositions of h4rm3l primitives, which are also often successful and novel attacks. (See Section 3.2, and Section 4).

After reading Section 3 & 4.1 and the official comments by the authors, this reviewer still has some questions and views that differ from those of the authors, which prevented me from improving my rating.

This reviewer does not hold the view that "h4rm3l primitives are string-to-string functions written in python that can perform arbitrary computation on strings, and therefore can represent any black-box attack." As an implementation of a combine attack, the authors should have at least spelled out in more detail the categorization of black-box jailbreak attacks and how they are implemented, which in turn explains why the "transform(prompt, assistant, random_state)" function can perform arbitrary computation and represent all black-box attacks.

We thank the reviewer for finding h4rm3l innovative and the writing of the paper clear!

$**Re: The motivation and unique advantage of transforming jailbreak tasks into a formal language are not entirely clear.**$

We point the reviewer to our common response above, which addresses this point.

$**Re: How does it surpass the functionalities of traditional red-teaming frameworks?**$

The biggest advantage of h4rm3l is in being able to systematically compose jailbreak primitives to generate novel, performant jailbreaks at scale. Other advantages of h4rm3l include the fact that it is complete (can represent any black-box attack), extensible (with new primitives and examples), and interpretable (h4rm3l code is human-readable). Our synthesizer efficiently explores the large compositional space of jailbreak attacks using bandit algorithms to generate new jailbreak attacks that are more performant than initial few-shot examples, and expressible in a human readable language. This cannot be achieved with other frameworks.

$**Re: Is this prompt demonstrably superior to those used in other jailbreak studies?**$

We noticed a lack of consistency in the way ASRs are measured in this rapidly evolving field. We preferred to follow Wei et al., (2023)'s ternary taxonomy of (GOOD, BAD, UNCLEAR) BOT behavior, which we found superior given a dataset of prompts assumed to be harmful. To create our prompt we performed multiple rounds of human-in-the-loop evaluations to ensure acceptable and sound results, based on the definition of "BAD BOT behavior": the LLM returning interpretable and usable harmful information in response to an illicit request. Our specific prompt text was adapted from the prompt introduced in the same work (see Appendix B), which was used for human annotators. Our classifier achieved $88.3\pm2.1\%$ precision and $100\pm0\%$ recall on data independently annotated by two expert annotators following the ternary taxonomy. We note that we used 50 randomly selected illicit prompts from the AdvBench dataset to perform our final benchmarking to ensure reliable ASR estimates.

$**Re: I suggest the authors provide a more straightforward and detailed efficiency comparison.**$

Thank you for your suggestion. We will update our manuscript with both asymptotic time complexity, and wallclock times we measured in practice. The synthesizer's complexity is $O(N_{iters} \cdot N_{proposals} \cdot N_{illicitprompts})$, which is linear with the number of iterations, the number of proposals (fixed at 20), and the number of illicit requests used to estimate ASRs during synthesis (fixed at 5). It took 25 and 34 hours to generate 1713 and 1939 attacks in 100 iterations respectively targeting GPT3.5 and GPT-4o. This results in an attack synthesis rate of around 1 per minute.

We thank the reviewer for recognizing the value of h4rm3l as the first formal, composable representation of jailbreak attacks, and its effectiveness in LLM safety assessment at scale!

$**Re: Estimation of attack success rates**$

We do think our findings are generalizable, given the steps we took to develop and validate our harmful LLM behavior classifier. To choose our prompt we performed multiple rounds of human-in-the-loop evaluations to ensure acceptable and sound results, based on the definition of "BAD BOT behavior": the LLM returning interpretable and usable harmful information in response to an illicit request. Our prompt is based on the ternary taxonomy ({GOOD, BAD, UNCLEAR} BOT behavior) introduced by (Wei et al., 2023), and our specific prompt text was adapted from the prompt introduced in the same work (see Appendix B), which was used for human annotators. Our classifier achieved $88.3\pm2.1\%$ precision and $100\pm0\%$ recall on data independently annotated by two expert annotators following the ternary taxonomy. However, we agree that annotating responses from more LLMs with a larger number of human annotators would increase readers' confidence in our results. We will perform additional validations and update results in the final version of the paper.

$**Re: The relationship between various jailbreak methods and their impact on the results is not clearly articulated.**$

Please see our common response above, where we provide an example of how we compose jailbreak attack primitives to give rise to new, performant jaibreaks.

$**Re: What are the time and computational costs?**$

h4rm3l synthesized jailbreak attacks efficiently. It took 25 and 34 hours to generate 1713 and 1939 attacks in 100 iterations respectively targeting GPT3.5 and GPT-4o. This results in an attack synthesis rate of around 1 per minute. The computational cost associated with synthesis is dominated by the LLM calls, the number of which scales linearly with the number of iterations, as well as the number of proposals per iteration (fixed at 20) and the number of illicit prompts used to estimate ASRs (fixed at 5 during synthesis and 50 during the final benchmarking).

$**Re: What do the numbers (e.g. "00536") in Figure 4 represent?**$

These are part of the unique identifier of synthesized attacks, which can be matched with rows of Table 1 and records in our released data. See benchmark.html in our supplementary materials package (click on individual programs to view their source code in h4rm3l). Concretely 00536 is the composed attack: CipherDecorator().then(RefusalSuppressionDecorator()).then(VillainDecorator())

$**Re: Are the horizontal and vertical coordinates of Figure 5 meaningful?**$

Yes, they are the 2-dimentional t-SNE projection of CodeBERT embeddings of the source code of h4rm3l attacks. The proximity of attacks in the 2d space can be interpreted as the similarity of their source code in CodeBERT feature space.

$**Re: In line 928, "therefore we use proxy names"**$

Yes, G is a proxy for Good Bot, B is a proxy for Bad Bot, and U is a proxy for Unclear. We will make this clearer in the paper.

$**Re: What do the abbreviations BS, RP, and RT in Tables 3 and 4 signify?**$

BS, RP and RT respectively refer to Baseline, Rephrasing and Retokenization. We will clarify this in the paper.

$**Re: In line 1222, "Table 2 and Table 3" should be corrected to "Table 3 and Table 4."?**$

Thanks! We will make this change.

Listing 1:Implementation of DRA's components as granular h4rm3l primitives

from DRA import (
    word_puzzle_obfuscation, word_char_split, payload_reconstruction, context_manipulation
)

class WordPuzzleObfuscationDecorator(PromptDecorator):
    def __init__(self) -> None:
        super().__init__()

    def decorate(self, prompt):
        return word_puzzle_obfuscation(prompt)
    
class WordCharSplitDecorator(PromptDecorator):
    def __init__(self, tr, br) -> None:
        self.tr, self.br = tr, br
        super().__init__()

    def decorate(self, prompt):
        return word_char_split(promt, self.tr, self.br)

class PayloadReconstructionDecorator(PromptDecorator):
    def __init__(self) -> None:
        super().__init__()

    def decorate(self, prompt):
        return payload_reconstruction(prompt)

class ContextManipulationDecorator(PromptDecorator):
    def __init__(self) -> None:
        super().__init__()

    def decorate(self, prompt):
        return context_manipulation(prompt)

class DisguiseDecorator(PromptDecorator):
    def __init__(self, tr, br) -> None:
        super().__init__()
        self.disguise = WordPuzzleObfuscationDecorator().then(WordCharSplitDecorator(tr, br))

    def decorate(self, prompt):
        return self.disguise.decorate(prompt)

class ReconstructDecorator(PromptDecorator):
    def __init__(self, tr, br) -> None:
        super().__init__()
        self.reconstruct = PayloadReconstructionDecorator().then(ContextManipulationDecorator())

    def decorate(self, prompt):
        return self.reconstruct.decorate(prompt)

class DRADecorator(PromptDecorator):
    def __init__(self, tr, br) -> None:
        super().__init__()
        self.dra = DisguiseDecorator().then(ReconstructDecorator())

    def decorate(self, prompt):
        return self.dra.decorate(prompt)

Listing 2: Implementation of DRA as an instance of TransformFxDecorator

from DRA import (
    word_puzzle_obfuscation, word_char_split, payload_reconstruction, context_manipulation
)
TransformFxDecorator(
    transform_fx=
    """def transform(prompt, assistant, random_state):
    tr = 0.5
    br = 0.5

    prompt=word_puzzle_obfuscation(prompt)
    prompt=word_char_split(prompt, tr, br)
    prompt=payload_reconstruction(prompt)
    prompt=context_manipulation(prompt)

    return prompt"""
)