On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents

We deeply appreciate reviewer 3cYX’s time for reviewing and your insightful comments. We are particularly encouraged that you find that the experiments and analyses are sound and rigorous. We address your concerns one by one:

[Design & Analysis] [W1] Advanced methods like Graph-based frameworks.

Inspired by GPTSwarm and MacNet, we design two advanced graph-based multi-agent frameworks using four agents:

1. Complete graph (a flat structure): each agent generates their own answers. After receiving all others’ answers in the next run, they re-generate the answers after thinking. The final answer is the majority one.

2. Star (a hierarchical structure): one leader proposes three approaches and distributes them to the three agents. After receiving the solutions, the leader gives its evaluation and generates the final answer.

We evaluate GPT-3.5 using the Math task. The performance is shown in the table below. We can conclude that our methods and analyses are applicable to diverse frameworks. Flat structure still has a lower performance since there is no leader coordinating the work.

[Reference 1] More related work in 2024-2025.

We have thoroughly cited and compared relevant papers in 2024-2025 [1-9] in our paper, according to all reviewers’ suggestions. In short, the NetSafe is the most relevant. While they provide amazing findings about the impact of different numbers of faulty agents, they do not investigate more subjective tasks like translation and text evaluation. We include code generation task, enabling us to study the impact of error types.

Additionally, our proposed AutoTransform and AutoInject methods can be viewed as automated evaluation tools for simulating failure scenarios and systematically measuring the robustness of multi-agent systems. This aligns with recent trends in automatic evaluation techniques (e.g., Zhang et al., 2024; Ju et al., 2024) where LLMs are used to both generate perturbations and evaluate outcomes.

[1] Zhang et al., Multi-agent Architecture Search via Agentic Supernet. 2025.

[2] Yu et al., A Survey on Trustworthy LLM Agents: Threats and Countermeasures. 2025.

[3] Mao et al., AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management. 2025.

[4] Zhou et al., CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models. 2025.

[5] Wang et al., G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems. 2025.

[6] Yu et al., LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models. 2025.

[7] Zhang et al., G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks. 2024.

[8] Yu et al., NetSafe: Exploring the Topological Safety of Multi-agent Network. 2024.

[9 Tran et al., Multi-Agent Collaboration Mechanisms: A Survey of LLMs. 2025.

[Zhang et al. (already in our paper)] PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety.

[Ju et al. (already in our paper)] Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities.

[Reference 2] Improving the introduction section.

We appreciate that you like our basic ideas. We have moved the core claims to the beginning of our introduction section. The text is shown below:

Just as human organizations rely on resilient structures to function despite internal errors, multi-agent systems built on large language models (LLMs) must also withstand faulty participants to remain effective. In this paper, we argue that the resilience of LLM-based multi-agent collaboration critically depends on the system’s structural design. As these agents increasingly take on complex, collaborative tasks, their decentralized nature makes them vulnerable to clumsy or even malicious agents—those that degrade performance or disrupt workflows. Drawing a parallel to organizational theory, we find that hierarchical structures—common in robust human institutions—offer superior resilience over flat or linear ones. To rigorously test this hypothesis, we propose two methods for simulating faulty agents, and demonstrate that structural design, combined with simple defensive strategies, can significantly mitigate performance degradation.

We deeply thank reviewer iwu4 for reviewing and appreciate the suggestions. We are encouraged that you find our paper well-written and sufficiently motivated! We address your concerns one by one:

[Claim & Evidence 1] [Broader Literature] [Reference] [W1] Missing papers.

Thanks for providing the missing references. We have thoroughly cited and compared the works of Zhang and Yu in our Related Work section. Additionally, we have included discussions about other relevant papers [1-6]. In short, the NetSafe is the most relevant. While they provide amazing findings about the impact of different numbers of faulty agents, they do not investigate more subjective tasks like translation and text evaluation. We include code generation task, enabling us to study the impact of error types.

Additionally, according to ICML 2025 Reviewer Instructions (icml.cc/Conferences/2025/ReviewerInstructions), “Authors cannot expect to discuss other papers that have only been made publicly available within four months of the submission deadline.” Thus the two papers submitted in October are concurrent work of our submission.

[1] Zhang et al., Multi-agent Architecture Search via Agentic Supernet.

[2] Yu et al., A Survey on Trustworthy LLM Agents: Threats and Countermeasures.

[3] Mao et al., AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management.

[4] Zhou et al., CORBA: Contagious Recursive Blocking Attacks on Multi-Agent Systems Based on Large Language Models.

[5] Wang et al., G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems.

[6] Yu et al., LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models.

[Claim & Evidence 2] The communication scheme of agents.

In our experiment, all of our selected models adopt a communication scheme of direct messaging, meaning that only the recipient agent will be able to see the message and no other message processing schemes such as summarizing or broadcasting is involved in the process. Therefore, the communication scheme is a controlled variable in our experiment, and does not influence our conclusion on the system structures.

[Method & Evaluation 1] Whether AutoTransform can be generalized to more general agent settings (tool uses).

AutoTransform takes the whole agent profiles as the subject of our transformation. While system prompts can be diverse (for multiple tool uses), an agent’s profile is usually consistent, since it includes the main goals, task descriptions and constraints of an agent. Therefore, our method can work with agents equipped with tools that can interact with the environment.

Moreover, some of our selected systems are already equipped with external tools. For instance, Self-collab equipped agents with a python interpreter for code testing, and MetaGPT’s agents have access to a message pool from which to retrieve history messages when performing tasks. This ensures our method is still valid in the more general settings. We also compute the performance on code tasks of different systems by whether or not they have tool modules. As is shown below, the performance of AutoTransform is consistent with these two types of agent systems.

[Method & Evaluation 2] Other tasks.

We acknowledge the need for evaluating more complex, real-world tasks hard to complete by a standalone model, such as making complete, executable software. For example, generate a snake game with pygame. Using a single GPT-3.5, the code is incomplete and the snake cannot be controlled. Using CAMEL, we achieve an executable game, with extensive features.

When using AutoTransform, the code is still executable but the snake's speed is too fast and the mapping between arrow key and snake's movement is mixed (e.g., pressing "left" will make the snake move upward). When using AutoInject, the game is totally unexecutable due to multiple errors injected. Our method is still effective in such real-world scenarios.

Additionally, though the tasks can be completed by single agents, from the results of single agents vs. multi-agents in Figure 4 on page 6 we can see that multi-agent systems can improve the performance over single agents.

[Q1] CoT models.

We explore the impact of CoT using a reasoning model, o1-mini, on the Math task due to budget limits. The results are presented in the tables below. For models using CoT, the performance is largely improved. However, we can still observe the performance decrease caused by faulty agents. Hierarchical structures still outperform the other two.

We deeply thank reviewer xvqz for your time and effort in reviewing our work, and your invaluable comments that further enrich our paper. We are particularly encouraged that you find our claims well-supported, our methods & evaluations reasonable, and by your recognition of the importance and novelty of our work. We address your concerns here:

[Claim & Evidence] AutoTransform is harder to control than AutoInject.

The reason why faulty agents produced by AutoTransform have less impact than directly injecting errors (AutoInject) is because GPT-3.5-Turbo is weaker in terms of the precise control of generated errors. We conduct an analysis using AutoTransform to instruct agents to introduce syntax errors in 20% and 40% of the code lines. The results are summarized below:

These results indicate significant variability, with agents struggling to consistently achieve the precise error rates of 20% or 40%. This underscores the necessity and robustness of our AutoInject method.

[Method & Evaluation] [W2 & W3] Inclusion numerical data and specific evidence to support our claims.

We have added numbers at (1) Left-column: Line 248, 251, 261, 274 and (2) Right-column: Line 251, 253, 257, 263, 292, 308, from the results in Table 5, 6, 7, and 8 in Appendix C on page 13.

[Design & Analysis] Fair comparison using the 6 different code bases.

Different structures have various role designs. For example, a flat system may not have a leader, where the other two systems have. There is not a set of profiles applicable to all multi-agent systems. Therefore, instead of manually writing profiles, we use the code bases from published studies which have been optimized by the authors for the specific structures. To offer fairer comparison, i.e., to mitigate the impact of using different prompts, we have included two systems per structure.

[W1] Math definition on agent collaborations.

A multi-agent system can be defined as a graph: $G = (V, E)$, where $V$ represents agents and $E \subseteq V \times V$ is a set of directed edges. Each $(u, v) \in E$ denotes agent $u$ reports to agent $v$.

• Linear systems are: directed path graphs, where $\forall v \in V, v \neq s, v \neq t$, we have: $\deg^+(v) = \deg^-(v) = 1$; for the endpoints, $\deg^-(s) = 0$, $\deg^+(s) = 1$, $\deg^+(t) = 0$, and $\deg^-(t) = 1$. Agents in this structure form a chain from $s$ to $t$.

• Flat systems are: directed complete graphs with bidirectional edges, where $\forall u, v \in V, u \neq v$, both $(u, v) \in E$ and $(v, u) \in E$. This represents a fully connected, non-hierarchical structure.

• Hierarchical systems are: rooted directed trees, where there exists a unique root agent $r \in V$ such that $\deg^-(r) = 0$, and $\forall v \in V \setminus {r}$, $\deg^-(v) = 1$. The structure is acyclic and forms a strict top-down hierarchy.

[Q1] Other communication topologies.

Inspired by MacNet and GPTSwarm, we design two advanced graph-based multi-agent frameworks using four agents:

1. Complete graph (a flat structure): each agent generates their own answers. After receiving all others’ answers in the next run, they re-generate the answers after thinking. The final answer is the majority one.

2. Star (a hierarchical structure): one leader proposes three approaches and distributes them to the three agents. After receiving the solutions, the leader gives its evaluation and generates the final answer.

We evaluate GPT-3.5 using the Math task. The performance is shown in the table below. We can conclude that our methods and analyses are applicable to diverse frameworks. Flat structure still has a lower performance since there is no leader coordinating the work.

[Q2] Impact on single agents.

We conduct experiments on applying the two error-introducing methods on a single agent based on GPT-3.5-Turbo across all four tasks. The performance is shown below:

Compared to the performance of other multi-agent systems (the table below), we conclude that all three types of systems have better resilience against both methods compared to a single agent. This is because the systems have other “good” agents for reviewing and testing, which can identify the errors made by the faulty agent.

We deeply appreciate reviewer LJmt’s time for reviewing and providing valuable suggestions. We are encouraged that you find our experiments comprehensive and well-executed, methods reasonable, and conclusions helpful for the community. We address your concerns here:

[W1] Math definition on agent collaborations.

A multi-agent system: $G = (V, E)$, where $V$: agents and $E \subseteq V \times V$: each $(u, v) \in E$ denotes agent $u$ reports to agent $v$.

• Linear systems are: directed path graphs, where $\forall v \in V, v \neq s, v \neq t$, we have: $\deg^+(v) = \deg^-(v) = 1$; for the endpoints, $\deg^-(s) = 0$, $\deg^+(s) = 1$, $\deg^+(t) = 0$, and $\deg^-(t) = 1$.

• Flat systems are: directed complete graphs, where $\forall u, v \in V, u \neq v$, both $(u, v) \in E$ and $(v, u) \in E$.

• Hierarchical systems are: rooted directed trees, where there exists a unique root agent $r \in V$ such that $\deg^-(r) = 0$, and $\forall v \in V \setminus {r}$, $\deg^-(v) = 1$.

[W2] Real-world complicated scenario.

To generate a snake game with pygame. Using a single GPT-3.5, the code is incomplete and the snake cannot be controlled. Using CAMEL, we achieve an executable game.

When using AutoTransform, the code is still executable but the mapping between arrow key and snake's movement is mixed. When using AutoInject, the game is totally unexecutable. Our method is still effective in such real-world scenarios.

[W3] Other models & CoT.

We conduct experiments using the LLaMA-3.1-70B-Instruct model on all the four tasks and a reasoning model, o1-mini, on the Math task. The results are presented in the tables below. The hierarchical structures perform the best still hold for non-GPT-based models. The performance is largely improved with CoT (o1), while we can still observe the impact of faulty agents. Hierarchical structures still outperform the other two.

[W4] Advanced structures.

Inspired by MacNet and GPTSwarm, we design two advanced graph-based multi-agent frameworks using four agents:

1. Complete graph (a flat structure): each agent generates their own answers. In the next run, they re-generate the answers after thinking on others' results.

2. Star (a hierarchical structure): one leader proposes three approaches and distributes them to the three agents and generates the final answer.

We evaluate GPT-3.5 using the Math task. Our methods and analyses are applicable to diverse frameworks. Flat structure still has a lower performance.

[Q1] Multiple faulty agents.

We explore the scenario of two faulty agents in AgentVerse on the Math task using GPT-3.5. AgentVerse has 4 agents, with Solver faulty by default. Here, we make an additional one, either Critic or Planner, faulty. Our methods are still valid. The Planner, who decides the high-level direction, plays a more important role. Its faults cause a greater performance decrease.

[Q2] Explanation on performance of different structures.

We can have an analogy with real-world human organization structure. A hierarchical structure enables centralized decision-making, where a top-level role gathers information and efficiently distributes decisions through clear chains of command. In contrast, a flat structure often lacks clear leadership, leading to decision paralysis and coordination issues. A linear structure has a defined chain of command, but communication is slower, and top leaders have limited oversight of lower levels.

[Q3] Explanation on how noises improve performance.

(1) Double Checking: Injecting obvious errors prompts agents to respond with corrections, which often fix both the injected and existing issues. (2) Divergent Thinking: Systems can stagnate due to repetitive reasoning from identical LLMs. Introducing major errors shifts the discussion, promoting fresh insights.

[Q4] Practical implications.

(1) designing hierarchical multi-agent systems help, which reflects a prevalent collaboration mode in real-world human society. (2) Agents can have questions or suggestions on others' results, boosting fault recovery.

[References] Missing papers.

We have cited and compared the works of Perez, Tan, and Tran in our paper. In short, we consider a new scenario where the vulnerability comes from weaker agents in a multi-agent system.