We thank the reviewer for recognizing the clarity of our writing and the relevance of our approach to prompt injection and jailbreak attacks. We appreciate your solid suggestions, which motivate us to further refine our work, and we are happy to address the concerns raised!

Q1: Unrealistic threat model

Sorry for the confusion regarding the threat model. Different from GCG, for LLM agentic scenarios, we cannot simply optimize for an affirmative response to trigger a specific malicious action. Therefore, we propose leveraging the original reasoning of the LLM agent (for which we cannot directly modify it, but we can query it) based on the current adversarial string. We then craft a surrogate modified response, $z^*$, as the optimization objective—similar to the “Sure” objective in GCG. After training, we generate a response based on the optimized string to see if it indeed triggers the target malicious action. Thus, $z^*$ serves as an optimization objective similar to "Sure" in GCG, and we do not have direct access to modify the real LLM agent's response or replace $z$ with $z^*$.

We hope this clarification resolves any confusion. If you have further questions, we are happy to provide additional details.

Q2: Minimal differentiation from GCG

Thank you for raising this point about differentiating our work from GCG!

GCG optimizes a fixed, affirmative prefix (e.g., "Sure"), which is static and suitable only for harmful requests. In contrast, UDora addresses the challenge that LLM agents perform extensive intermediate reasoning before executing their final action. Directly optimizing for the final action in this scenario with GCG is impractical. Instead, UDora uses a dynamic three-step optimization process:

1. Gather the Initial Response: We first query the initial response from the LLM.

2. Constructing a Surrogate Optimization Target: Leveraging the original response, we insert adversarial noise at optimal positions determined through a weighted interval algorithm guided by our positional scoring function (Equation 1). This modified surrogate response is hypothetical, not the actual response from the LLM.

3. Optimizing the Adversarial String: We then optimize the adversarial input string by maximizing our positional score rather than directly maximizing the probability as done in GCG. Note that, unlike GCG, we only leverage the gradient on the multiple noise insertions, rather than on the entire $z^*$.

This adaptive and dynamic process enables UDora to exploit the LLM agent’s reasoning in a way that a static, fixed objective like GCG cannot. Furthermore, UDora also significantly outperforms GCG in empirical results. For instance, with the LLaMA-3.1 model, UDora achieves:

• 99% ASR versus GCG’s 77% on the InjecAgent dataset,
• 61.75% ASR versus GCG’s 15.00% on the WebShop dataset,
• 97.7% ASR versus GCG’s 60.8% on the AgentHarm dataset.

Q3: High computational complexity

We leverage KV caching for generating the $z$ in our implementation, and due to the word limit, we kindly encourage you to review the discussion about the computation cost provided in Q4 for Reviewer c28e.

Q4: Insufficient experimentation

Thank you for this insightful suggestion! We indeed included a practical example (Fig 11) of attacking the AutoGen Email Agent from Microsoft, which employs GPT-4o as the base LLM (GPT-4 can also be applied), relying only on the returned log probabilities for the attack. Regarding models like Claude, where neither log probabilities nor tokenizers are available, the attack scenario becomes inherently more challenging and remains a direction for future exploration.

Concerning the AgentDojo dataset, currently, it only supports API-based models, lacking direct compatibility with HuggingFace models, which doesn’t allow the attack optimization here. In addition, we also have communicated directly with the authors of AgentDojo, exploring possibilities for data extraction or integration with HuggingFace models. Unfortunately, both the authors from AgentDojo and we have agreed that certain technical difficulties (primarily related to prompt injections within pre-designed functions) have made this quite challenging. We appreciate your understanding of this limitation.

Regarding the application of UDora to tasks involving multiple tool calls, our experiments with the AgentHarm dataset indeed encompass multi-step scenarios. We observed that once UDora successfully triggers the initial malicious tool call, subsequent steps often follow naturally. Thus, UDora can still effectively apply to multi-step tool call scenarios, as long as the first step is triggered. Moreover, adapting UDora explicitly for optimization across multiple steps is also feasible with minor code adjustments. Exploring and refining this multi-step optimization approach will be a focus of our next step, and your suggestion is greatly appreciated!

We sincerely thank the reviewer for recognizing UDora’s innovative approach in utilizing reasoning steps and for highlighting the robustness of our empirical results across diverse benchmarks, particularly in the real-world AutoGen email agent attack scenario. We deeply appreciate your thorough and thoughtful review, which has significantly strengthened the clarity and impact of our contributions!

Q1: While positional scoring function is empirically effective, its design choices lack formal analysis

Thank you for highlighting this point; we will include a more formal analysis to clarify our design choices in the final revision. Briefly, our positional scoring function (Equation 1) is guided by two primary principles: (1) we prioritize positions where some tokens already match the target; (2) if multiple positions have the same number of matched tokens, we prefer the one with the highest average token probability rather than the product, as the product would undesirably bias selections based on token length. Equation 1 precisely reflects these principles. We will provide additional intuition and a formal rationale for this approach in our final version.

Q2: Limited discussion of computational costs

Thank you for raising this important point! Due to the word limit, we kindly encourage you to review the statistic provided in Q4 for Reviewer c28e.

Q3: Essential References Not Discussed

Thank you for suggesting these essential references for contextualizing UDora’s adversarial focus! We have added all these in our current version.

Q4: Code Release: Provide a minimal implementation or pseudocode for key components (e.g., positional scoring).

Thank you for this insightful suggestion. We will release our code upon acceptance and also include the pseudocode for the positional scoring function, in the appendix of our final version.

Q5: Failure Analysis: Include examples of unsuccessful attacks to identify UDora’s limitations

Thank you for the valuable suggestion! While our current analysis primarily focuses on successful attacks, we agree that examining unsuccessful cases is equally important for understanding UDora’s limitations. In line with our current detailed successful-case analyses (Fig 5–9), we will include representative failure examples from each dataset in our final version.

Q6: Theoretical Basis: Can you formalize the conditions under which UDora’s noise insertion guarantees successful attacks?

Thank you for this insightful question! Empirically, we've observed that attacks typically succeed when the overall positional score across all noise insertion locations exceeds a threshold (e.g., greater than 3). Intuitively, this corresponds to cases where a specific target is mentioned multiple times during the LLM agent's reasoning, significantly increasing the likelihood that the agent adopts the associated malicious action. We hypothesize that this phenomenon arises because such repetitive mentions rarely occur in the LLM agent’s training data without the agent subsequently performing the mentioned action.

Q7: Defense Evaluation

Thank you for the insightful question! Regarding adversarial training, it could mitigate UDora-style attacks, but it typically involves a trade-off, potentially reducing the overall utility or generalization capability of the LLM agent. For detection mechanisms, following your suggestion and reviewer c28e’s recommendation, we will expand Section 7 to include preliminary experiments on defense mechanisms against UDora-style attacks. Specifically, we plan to explore reasoning validation techniques, such as detecting hallucinated or erroneous steps introduced by UDora to achieve the target malicious action. For instance, we can prompt the LLM to self-assess the consistency between its reasoning steps and the original instruction or the observation, thereby evaluating its ability to recognize and mitigate UDora-style adversarial manipulations.

We have included some initial results in our response to Q2 under Reviewer c28e. Due to the constraints of the word limit, we kindly invite you to examine the details there.

Q9: Ethical Considerations and Safeguards

Thank you for raising this critical concern. We will carefully control the release of adversarial strings derived from real-world examples, such as those involving AutoGen demonstrated in our paper, to prevent potential misuse. Additionally, we will discuss potential mitigation strategies, including reasoning validation (as previously mentioned) and methods like obscuring or summarizing intermediate reasoning steps presented to users (like open o1) during interactions with LLM agents. Furthermore, we agree that collaboration with red-teaming communities and implementing adversarial detection mechanisms would significantly enhance security. We will include a new paragraph addressing these ethical considerations and proposed safeguards in our final revision.

Many thanks to the reviewer for the thoughtful and detailed feedback. The expertise and time invested in this work have been instrumental in enhancing its quality!

Q1: UDora assumes the attacker can access either the entire model or its token probability distribution during reasoning, which is a strong assumption for real-world deployment.

We appreciate your insightful question regarding our assumption that attackers can access either the entire model or at least the token probability distribution! As discussed in the paper, this requirement is indeed a limitation in certain real-world settings—particularly for closed-source models (e.g., Claude) that do not expose log probabilities or even a tokenizer. However, we note that some open models (including GPT-series models) do provide token-level probabilities, which enables our proposed attack in practice. For instance, in our experiments, the GPT-4o-based AutoGen email agent made these probabilities accessible, facilitating UDora’s successful attack strategies.

Meanwhile, we observe a broader trend of rapidly improving open-source LLMs (e.g., DeepSeek-R1) being integrated into real-world applications, even in partnership with major industry players (e.g., Perplexity AI, NVIDIA, Microsoft, etc.) to reduce costs. As this trend continues, full or partial access to model internals (including token probabilities) may become increasingly common in LLM agentic systems. In such cases, UDora’s approach remains directly applicable. Extending UDora to a purely query-based attack is a natural and important direction as our future work, and we are also excited to explore this in our next step!

Q2: UDora lacks discussions on defenses.

Thank you for raising this important point about defenses! We will add a new paragraph in our final version to discuss how to mitigate potential misuse of UDora. One straightforward and practical strategy is for the agent provider to share only a condensed or sanitized summary of the reasoning steps like openai o1 reasoning model—rather than the full chain-of-thought. By limiting visibility into the underlying reasoning process, attackers lose critical information for inserting malicious noise in the reasoning steps. In our final version, we will also discuss concurrent work on guardrails for LLM agents, such as [1, 2].

Besides, following the suggestions from Reviewers c28e and vE1i, we have also explored defenses related to reasoning validation, i.e., using a separate LLM to determine if the attacked reasoning aligns with the instructions or environment. Specifically, we found the detection rates to be 97.0% and 75.0% on the WebShop dataset using the LLaMA3.1 and Ministral models, respectively. However, we also observed a high false positive rate, as 69.2% and 89.2% of the cases with normal reasoning were also incorrectly flagged as incorrect reasoning, indicating that reasoning validation in practice is still somewhat challenging as it is as difficult as detecting hallucinations in LLM agents.

[1] Xiang, Z., Zheng, L., Li, Y., Hong, J., Li, Q., Xie, H., ... & Li, B. (2024). GuardAgent: Safeguard LLM agents by a guard agent via knowledge-enabled reasoning. arXiv preprint arXiv:2406.09187.

[2] Luo, W., Dai, S., Liu, X., Banerjee, S., Sun, H., Chen, M., & Xiao, C. (2025). AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection. arXiv preprint arXiv:2502.11448.

Q3: Would the ASB benchmark provide a more comprehensive evaluation framework for assessing the effectiveness of UDora?

Thank you very much for suggesting this reference! We agree that the ASB benchmark would indeed provide a comprehensive evaluation framework for UDora, and we plan to include experiments using it in our final version. Due to the limited rebuttal period, fully familiarizing ourselves with the framework’s code and integrating our attack will require some additional time.

In the current submission, we have evaluated UDora across three diverse datasets, including InjecAgent, whose data characteristics look a bit similar to those of the ASB benchmark. Thus, the performance results obtained on InjecAgent could serve as a preliminary indicator of the expected performance on the ASB benchmark.

If you have any further questions or suggestions, please feel free to let us know. Your feedback is greatly appreciated and will certainly help us improve our work a lot!

We are deeply grateful to the reviewer for their thorough and insightful feedback. Your contribution of time and expertise has significantly enriched the development of our research!

Q1: Black-box setting

Thank you for the insightful question! In a pure black-box environment—where only the final response is observable—the attack indeed becomes more challenging. Still, UDora could be extended in a few ways:

1. Surrogate Model Training: Train a surrogate LLM to mimic the target LLM agent’s behavior based on observed responses, then apply UDora on the surrogate LLM. The resulting adversarial prompt may transfer to the real target model.

2. Cross-Model Transferability: Leverage the fact that adversarial prompts sometimes transfer between models, as illustrated by our Perplexity AI example using open-source models (e.g., LLaMA 3.1 or even the currently widely adopted DeepSeek-R1).

Extending UDora to a purely query-based, black-box attack is a natural and important next step, and we are actively exploring methods in that direction. We welcome any additional ideas you might have on this front!

Q2: Reasoning validation to detect adversarial manipulations

Thank you for raising the idea of using reasoning validation as a countermeasure! Our experiments (Figs. 5–9) show that UDora’s manipulations distort the agent’s reasoning—effectively causing "hallucinated/ wrong" reasoning paths that steer the agent toward harmful actions. For instance, in the WebShop scenario, the agent is tricked into believing an incorrect item meets all specified requirements (e.g., price, attributes), resulting in a malicious action.

Therefore, we conducted the following experiments on reasoning validation as follows: we used a separate LLM to independently determine if the reasoning from the LLM agent aligns with the instructions or the observation from the environment. Specifically, we found the detection rates to be 97.0% and 75.0% on the WebShop dataset using the LLaMA3.1 and Ministral models, respectively. However, we also observed a high false positive rate, as 69.2% and 89.2% of the cases with normal reasoning were also incorrectly flagged as incorrect reasoning, indicating that reasoning validation in practice is still somewhat challenging as it is as difficult as detecting hallucinations in LLM agents.

Q3: Compare against self-reflective adversarial attacks

Self-reflective adversarial attacks focus primarily on bypassing an LLM’s safety filters to extract harmful responses. By contrast, UDora targets a malicious tool or functionality rather than just overcoming safety barriers.

1. Malicious Environment: In contexts like WebShop or InjecAgent, UDora aims to manipulate the LLM into taking a malicious action (e.g., buying something undesirable) without producing obviously harmful or unethical text. Because the output may look innocuous, standard safety filters that rely on detecting overtly harmful or unethical content often fail to catch this subtle manipulation.

2. Malicious Instruction: Here, self-reflective attacks stop at bypassing the filter, but UDora must also ensure the malicious function is genuinely activated. As shown in Fig. 8 (4), even if the request bypasses safety checks, it might not trigger the intended malicious action, requiring further optimization of UDora.

Hence, UDora is more specialized than self-reflective methods: it not only defeats safety filters but also reliably orchestrates a specific malicious action.

Q4: Computational requirements and faster attack execution

In UDora, the computational overhead at each iteration primarily stems from two parts:

1. Query time: Generating the model’s reasoning process given the current adversarial string.
2. Optimization time: Finding the optimal position, computing gradients, and updating the adversarial string.

When running UDora with LLaMA 3.1:

• On the AgentHarm dataset, the average times per iteration for these two parts are 3.48s (query) and 11.22s (optimization).
• On the InjecAgent dataset, they are 5.57s (query) and 5.97s (optimization).

With the Ministral model:

• On AgentHarm, the corresponding times per iteration are 2.49s (query) and 6.04s (optimization).
• On InjecAgent, they are 6.86s (query) and 8.18s (optimization).

These results show that, similar to GCG, the main bottleneck is in the optimization phase. However, UDora typically only requires around 20 iterations for a successful attack (see Fig. 3).

UDora can also be optimized for faster attack execution:

1. Less frequent updates: Instead of updating the reasoning process after every iteration, update it at a specified interval (e.g., every 10 steps).

2. Partial reasoning generation: Generating only the first 100 tokens of reasoning during query can still yield high attack success rates. For instance, on AgentHarm with LLaMA 3.1, generating just the first 100 tokens still achieves a 97% ASR.