GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modeling

We appreciate the reviewer’s positive feedback on the novelty and effectiveness of GUARDIAN, its model-agnostic design, and the clarity of our writing and figures. Thank you for the valuable review. Below we address the detailed comments.

Question 1: Downstream tasks may not adequately evaluate multi-agent safety claims

We sincerely thank the reviewer for the thoughtful feedback. While we agree that direct evaluations of safety—such as adversarial robustness or conflict resolution—are valuable, our goal in this work is to investigate multi-agent safety within general-purpose, realistic tasks that better reflect practical deployment scenarios.

Instead of relying solely on benchmarks explicitly designed for safety testing, we adopt widely used tasks such as MMLU and MATH, which allow us to explore how safety-related failures can emerge and be mitigated in collaborative reasoning. This perspective aligns with recent work that studies safety interventions in general-purpose reasoning tasks using standard benchmarks [1, 2].

Furthermore, our evaluation does include aspects of adversarial robustness: our error injection setup simulates compromised or adversarial agents contributing misleading information. GUARDIAN’s strong performance under these conditions demonstrates its effectiveness in maintaining coordination and robustness in the presence of such disruptions. We appreciate the suggestion, and in future work we plan to incorporate more explicit safety-focused benchmarks.

Question 2: Structure reconstruction may be ineffective in fully connected graphs; only node-level anomalies addressed

We thank the reviewer for raising these insightful concerns regarding the utility of the structure reconstruction decoder and the scope of anomaly modeling.

While our framework supports fully connected communication, we explicitly evaluate sparser topologies to avoid trivial structure decoding and better assess robustness. Specifically, we experiment with settings where each agent in the previous round is connected to only 25%, 50%, or 75% of the agents in the next round. In these cases, the structure reconstruction decoder does not produce an all-ones matrix, and as shown in Table 2, our method continues to achieve strong performance. To ensure fair comparison, all models are evaluated under the same topology settings.

Our anomaly detection module operates globally by aggregating outputs from previous rounds, enabling it to capture both temporal inconsistencies and structural anomalies. This temporal aggregation allows the model to detect patterns that may not be apparent within a single round or local neighborhood.

While our removal strategy focuses on anomalous nodes, our framework also implicitly accounts for edge-level anomalies. Faulty or misleading communication links often manifest as degraded outputs in the connected nodes in the subsequent round. By identifying and removing such affected nodes, our approach effectively mitigates the impact of corrupted edges and prevents further error propagation.

We will revise the manuscript to clarify these design choices and better articulate how both node- and edge-level anomalies are handled in our framework.

Table 2: Performance comparison under different connection sparsity (MATH dataset, GPT-3.5-turbo). Bold values indicate highest accuracy.

Question 3: Multi-agent setup appears simplistic; generalization to unseen agents

We sincerely thank the reviewer for the thoughtful and constructive feedback.

We fully agree that supporting complex agent behaviors—such as tool usage, memory, and heterogeneity—is important for advancing real-world multi-agent systems. In this work, our aim is to develop a general-purpose defense framework for multi-agent safety under minimal assumptions. To better isolate and analyze core safety challenges, we adopt a simplified setting where agents are functionally homogeneous and focus on basic text-based QA. This design enables controlled experimentation and fair comparison. The framework is modular and extensible: more advanced agent capabilities can be incorporated without fundamental changes to the core mechanism. We consider these extensions highly promising and plan to explore them in future work.

Regarding generalization, we emphasize that our method is model-agnostic and unsupervised—it does not rely on specific model architectures, training data, or prior knowledge of agent behavior. This enables effective generalization to unseen agents and novel multi-agent configurations, which we believe is essential for real-world applicability.

Question 4: Training data source and attack implementation details not specified

We sincerely thank the reviewer for raising these important points. We apologize for the lack of clarity in the original draft.

• Our framework is evaluated on four publicly available datasets, and for each dataset, we train a model independently using the Incremental Training Paradigm within that dataset.
• The three attack mechanisms are implemented as follows:
  • Hallucination amplification: a failure mode where incorrect information is reinforced through iterative reasoning.
  • Agent-targeted attacks: simulate compromised agents by modifying their role prompts to produce systematically incorrect outputs.
  • Communication-targeted attacks: simulate link corruption by perturbing transmitted messages to mislead downstream agents.

We will revise the manuscript to include clearer definitions, implementation details, and justifications for these settings to improve clarity and reproducibility.

Question 5: Temporal dependencies in Transformer not explicitly modeled

Thank you for the helpful comment.

Yes, our method implicitly models temporal dependencies by feeding $(Z_1, Z_2, \ldots, Z_{t-1})$ into a Transformer, which captures cross-time interactions via self-attention. We will clarify this point in the revised version.

Question 6: Visibility of earlier agents' outputs to later agents

Thank you for pointing it out. In our setting, agents in each round can access the outputs of only those agents they are directly connected to from the immediately preceding round. They do not have visibility into all prior agents or the full global history. We will make this point clearer in the revision.

References:

[1] Yoffe et al. DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metrics. arXiv 2024.
[2] Amayuelas et al. MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate. Findings of EMNLP 2024.

Thank you for the valuable comments and constructive suggestions that have helped us identify areas for improvement. Below we provide detailed responses to address your concerns.

Question 1: Some unclear information

We sincerely thank the reviewer for the thoughtful comments and helpful suggestions.

• Regarding Figure 1, it is referenced on line 37 as an illustrative example of the three major categories of safety risks in LLM-based multi-agent systems, which motivate our overall design.
• In Equation (1), L denotes the loss function used to train the anomaly detection model; we will further clarify it and make it clearer.
• In Section 5, we include a concrete walkthrough example (line 299, Figure 4), where GUARDIAN is applied to address the co-existence of hallucination and agent-targeted errors. We will improve the section and surrounding text to make this clearer.
• We appreciate the feedback on Figures 10–12, and will enlarge the font size to improve readability.

We are grateful for these suggestions and will incorporate them to enhance the clarity and presentation of the paper.

Question 2: Agent graph is not shown

We thank the reviewer for the helpful suggestion.

We would like to claim that the current presentation would benefit from a more explicit visualization of the multi-agent layout. While Figure 2(a) illustrates a 3-agent setup, we agree that including a concrete example of a 4-agent system would make the target-agent configuration clearer, and we will add such a layout in the revised version.

Our implementation leverages widely used multi-agent frameworks, including LangGraph, Google ADK, and CrewAI. We appreciate the suggestions to incorporate CAMEL and AutoGen, and will expand the discussion to include them for broader context and clarity.

Question 3: Dataset is small

We sincerely thank the reviewer for pointing it out. The dataset size plays a crucial role in evaluating the generality of a method. In our case, we would like to humbly clarify that the task under study is particularly challenging: solving a single example typically involves multiple rounds of agent collaboration, numerous LLM queries, and several external API calls. Due to this complexity, prior works in similar multi-agent settings [1, 2, 3] have also adopted evaluation sets of approximately 100 samples, which has emerged as a practical and useful setup in balancing task difficulty and experimental feasibility.

After thorough research and consideration, we evaluated our approach across four datasets and three independent runs, resulting in over 1,200 full executions. We believe this represents a meaningful experimental for this domain and provides empirical evidence supporting the robustness and general applicability of our method.

Question 4: Unknown speed impact

We thank the reviewer for raising this important and insightful point.

To address this concern, we have conducted additional experiments to measure the average runtime per question (in seconds) under communication-targeted attacks using 4 agents and 3 rounds of interaction, as shown in Table 1.

Table 1: Time cost (s) comparison under communication-targeted attacks with GPT-3.5-turbo. Bold represents the lowest time cost.

Our method achieves the lowest time overhead on MMLU and FEVER, and is only marginally slower than LLM Debate on MATH. These results demonstrate a favorable efficiency-performance trade-off. The reduced overhead is primarily due to node deletion, which results in fewer API calls and reduced inter-agent waiting time. Although anomaly detection is applied in every round, it incurs minimal cost due to the small size of the agent communication graph.

Question 5: Examples are too simple

We sincerely thank the reviewer for the thoughtful suggestion. The examples in Figures 10–12 were primarily selected to clearly illustrate the key mechanisms of our approach. We agree that more complex and realistic scenarios are also important for evaluating robustness. In our experiments, we did encounter such cases where simple majority voting fails, and our method shows clear benefits. We will include these more challenging examples in the revised version to better highlight the strengths of our approach under realistic conditions.

References:

[1] Yoffe et al. DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metrics. arXiv 2024.
[2] Amayuelas et al. MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate. Findings of EMNLP 2024.
[3] Huang et al. On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents. ICML 2025.

We thank the reviewer for highlighting the clarity, novelty, and strong performance of our work on a timely and important problem. Thank you for the valuable review. Below we address the detailed comments.

Question 1: Lack of intuitive explanation why the encoder-decoder architecture works in this context

Thank you for the insightful comment. We follow your suggestion to provide more intuition for the encoder-decoder design, and we will clarify this in the revision. LLM-based multi-agent collaboration is an increasingly important paradigm with significant potential, but it also introduces unique safety challenges such as hallucination amplification and error propagation. These issues are inherently unlabeled and thus cannot be effectively addressed using supervised approaches. To handle this, we adopt an unsupervised encoder-decoder architecture that models typical agent behavior over time. This design enables the model to identify agents whose behavior deviates from normal interaction patterns, without requiring labeled data. We appreciate the reviewer’s suggestion and will include this explanation in the revised version to improve clarity.

Question 2: Training and inference with Guardian can be resource intensive

Thank you for pointing it out. In experiments, we find that GUARDIAN is relatively efficient. As shown in Figure 5, it incurs the fewest API calls among all methods evaluated. This is because anomalous agents are progressively removed during each round, thereby reducing redundant API queries and improving efficiency over time.

Question 3: Discussion on non-malicious agents mistakenly flagged as anomalies

Thank you for pointing it out. We conduct the relevant experiments, and our results show that this risk is well controlled: GUARDIAN achieves an average anomaly detection accuracy above 80%, with peak performance reaching 94.74%, indicating that most removed agents are indeed problematic. In addition, we adopt a conservative removal strategy, where only the single agent with the highest anomaly score is removed per round, which helps minimize the likelihood and impact of false positives.

To understand the consequences of false positives, we conducted a case-level analysis. The results show that even in rare cases where a correct agent was mistakenly removed, the system does not exhibit hallucination amplification or notable degradation in answer quality. We find that the remaining agents are often able to reach a correct decision, preserving the robustness of the final output. Moreover, under our iterative defense framework, such errors can often be detected and corrected in subsequent rounds.

We will include representative examples and a brief analysis of these cases in the revised version to provide further clarity. Overall, even when false positives occur, their impact on final outcomes remains negligible, demonstrating the robustness of our method. We appreciate the opportunity to clarify this point.

Question 4: Trade-off between anomaly-detection threshold and the rate of incorrectly removed but benign agents

Thanks for the thoughtful question. As discussed in our response to Question 3, while our method is designed to effectively control the risk of mistakenly removing correct agents, we recognize that tuning the anomaly-detection threshold inevitably involves a balance between detection performance and false positive risk.

In our experiments, the system demonstrates relative robustness to threshold variations when operating under a conservative removal policy. Furthermore, since typical configurations involve only a small number of agents (e.g., 3–7), removing more than one agent per round may disrupt system functionality. To mitigate such risks, we adopt a cautious threshold and limit removal to at most one agent per round, which helps preserve both stability and overall performance.

Question 5: Citation format is not consistent with the template

Thank you for your suggestion. We will revise accordingly.

We appreciate the reviewer’s recognition of our timely and clearly presented work, its thorough evaluation, and the method’s accuracy and efficiency. Thank you for the valuable review. Below we address the detailed comments. We hope you may find our response satisfactory and increase the score accordingly.

Question 1: Limited technical novelty

Thanks for pointing it out. We would like to humbly claim that LLM-based multi-agent collaboration introduces an unique class of safety challenges that are fundamentally different from those in traditional multi-agent systems. In particular, issues such as hallucination amplification and error propagation across rounds are global, temporally distributed, and label-free, posing unique difficulties for detection. To tackle these challenges, in this paper we propose a tailored temporal graph modeling framework that integrates an unsupervised encoder-decoder architecture and GIB-based abstraction, specifically designed to work without labeled data or any modification to base LLMs. While inspired by existing techniques, the way these components are adapted and combined for this unique problem setting is, to our knowledge, unique. We believe this represents a meaningful step toward building safer LLM-based collaborative systems.

Question 2: Few lines are overly stated

We thank the reviewer for the valuable feedback and apologize for any confusion caused by our wording. Our intention in Lines 57–59 was not to claim a pioneering contribution to Information Bottleneck theory itself, but rather to highlight that, to the best of our knowledge, we are the first to systematically apply a graph abstraction mechanism grounded in IB theory to address safety risks in LLM-based multi-agent collaboration. We will revise it to better reflect the scope and context of our contribution.

Regarding the second point, we appreciate the reviewer pointing out the ambiguity. Our statement in Lines 306–307 was meant to emphasize that within each agent-number setting, our method consistently outperforms baselines, demonstrating robustness under different configurations. We did not intend to imply that absolute performance remains unchanged as the number of agents increases. As shown in Table 2 in this paper, the gradual decrease in accuracy with more agents is expected due to increased complexity and error propagation, but our method still maintains a clear advantage over other approaches. We will improve clarity in the revision.

Question 3: Figure 2(b) is confusing

Thank you for the helpful comment. We apologize for the confusion caused by the presentation of Figure 2(b). The left and right plots illustrate two independent scenarios: the left shows agent-targeted error injection, while the right shows communication-targeted error injection. They are not intended for direct visual comparison, but rather to demonstrate different types of anomalies under our framework. We will clarify this distinction explicitly in the caption to avoid misunderstanding.

Question 4: No discussion on failure modes

Thank you for raising this suggestion. To further assess the potential effects of mistakenly removing a correct agent, we conduct the corresponding experiments. The results show that this risk is well controlled: GUARDIAN achieves an average anomaly detection accuracy above 80%, with peak performance reaching 94.74%, indicating that most removed agents are indeed problematic. In addition, we adopt a conservative removal strategy, where only the single agent with the highest anomaly score is removed per round, which helps minimize the likelihood and impact of misclassifications.

To understand the consequences of incorrect removals, we conducted a case-level analysis. The results show that even in rare cases where a correct agent is mistakenly removed, the system does not exhibit hallucination amplification or notable degradation in answer quality. We observe that the remaining agents often still reach a correct decision, preserving the robustness of the final output. Moreover, under our iterative defense framework, incorrect decisions in one round can often be corrected in subsequent rounds.

We will include representative examples and a brief analysis of such cases in the revised version to further clarify this point. Overall, even when a correct agent is mistakenly removed, its impact on final outcomes remains negligible, further demonstrating the robustness of our method. We appreciate the opportunity to clarify this point.

Question 5: Training dataset is not discussed

Thank you for the question. We use four datasets in our experiments, and for each dataset, we train a separate model to achieve the corresponding capability. The Incremental Training Paradigm is applied within each dataset, rather than across datasets. That is, our method is designed specifically for in-distribution anomaly detection, where the underlying data distribution remains consistent while new episodes are incrementally introduced. This design choice enables us to systematically evaluate temporal anomaly detection performance under realistic conditions that preserve domain consistency.