CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation

Thank you for the insightful and positive comments! In the following, we provide our point-by-point response and hope our response helps address your concerns. We also look forward to the subsequent discussion which may further help to solve the current issues.

Q1: Better putting the discussion of the reused modules from CoELA into the preliminary sections.

Thank you for the suggestion! In revision, following your suggestion, we have moved this discussion into the preliminary section, and only introduced in the method section the improvements of our method on top of these modules. For example, on top vanilla memory modules, we introduce plan-related information like meta plan and task progress into the memory module for retrieval to discuss new meta plan or adaptation.

Q2: The contribution is limited since the only improvement is achieved by introducing a pre-defined communication protocol by prompting into an existing CoELA framework.

• We respectfully disagree with the claim that the contributions are limited compared to the original CoELA framework. We introduce two key innovative strategies that are not in the original framework and significantly improve the cooperation efficiency of embodied agents in the experiments:

  • Meta-plan generation mechanism that provides long-term strategic guidance, enabling agents to form coherent, goal-aligned plans that enhance coordination efficiency in decentralized settings. By contrast, CoELA relies on ad-hoc or reactive adjustments.This meta-plan mechanism ensures consistent and structured decision-making, crucial for complex multi-agent tasks.

  • Progress-adaptive meta-plan and execution strategy that dynamically adjusts and refines the meta-plan based on real-time progress, ensuring adaptability and responsiveness in unpredictable environments. This adaptiveness addresses the limitations of static or pre-defined strategies, particularly in scenarios with unexpected challenges.

• These contributions advance the field of LLM-driven embodied agent cooperation by introducing a dynamic escalation mechanism. Unlike traditional methods like CoELA, this mechanism enables agents to actively detect and discuss potential errors or inconsistencies, particularly in decentralized settings with partial observations. Such environments often lack shared global information, leading to low cooperation efficiency, as indicated in Table 3. By introducing active intervention, our strategies prevent error propagation and significantly enhance system robustness and efficiency, as demonstrated in Tables 1 and 2.

• Our strategies are indeed implemented through LLM prompting—a standard technique in LLM-based embodied agent methods like CoELA, RoCo [Mandi et al., 2023], and ProAgent [zhang2023proagent], and we never claimed otherwise. We emphasize that our novelty lies in the design and mechanisms of these strategies (meta-plan generation and its progressive adaptation) and the notable impact on agent cooperation, rather than the use of prompting itself, which is simply the ‘tool’.

Q4. It may be better to report the steps used for communicating as the communication cost.

Thank you for this valuable suggestion. In our CaPo framework, communication is considered costly, with 500 characters equating to a cost of 1 frame. Therefore, we agree that reporting the number of steps (frames) used for communication offers a more accurate representation of the communication cost.

Accordingly, we provide a detailed analysis of the communication cost in the following table by collecting statistics on the TDW-MAT tasks.

• Taking the food tasks as an example, our CaPo requires 14 meta-plan updates per task, with each update generating an average of 1,757 characters. Since 500 characters equate to 1 frame, this results in a communication cost of 56 frames out of the total 3,000 frames for each task. Similarly, the communication cost for each task on Stuff is 51 frames.

• Additionally, all steps (frames) consist of both moving steps and communication steps. Therefore, the communication cost is inherently reflected in the final efficiency metric, i.e., the transport rate and average steps. As shown in the table 1 and 2, our method exhibits significant efficiency improvement, highlighting a good tradeoff between communication cost and efficiency improvement. We include these results and analysis in A.6 of supplementary materials.

We hope such analysis could address your concern about communication cost.

Q5. Does the steps reported in the main table include the steps used to communicate or not? This seems an unfair comparison.

We appreciate your thoughtful question. The steps reported in the main table include both the steps used for communication and those used for movement. Regarding our previous response to Q4, we apologize for the typo in stating "needing an average of 92 and 83 moving steps on each task." To clarify, 92 and 83 refer to the total number of steps, which include both communication and movement.

Thank you sincerely for your insightful comments, which greatly help us improve the quality of this manuscript. We provided detailed responses to the existing issues below.

Q1. Typos in previous response to Q3.

Thank you for pointing out the potential typo in our response to Q3. We would like to clarify that the correct term should indeed be RHP in this context. Specifically, the sentence should read:

“When CaPo cooperates with RPH agents, it achieves a higher transport rate compared to both a team of two RHP agents and a combination of RHP and CoELA agents.”

We sincerely apologize for this oversight and have ensured all such typos are corrected in the revised manuscript. Thank you again for your attention to detail.

Q2. Confusing EI numbers in the previous Q3 table.

Thank you for pointing out the discrepancy in EI values between RHP+CoELA and RHP+CaPo in the previous Q3 Table. We understand how this might seem confusing, and we clarify as follows:

• Source of Results: The TR values for RHP and RHP+CoELA have been referenced directly from the CoELA paper, whereas the results for RHP+CaPo have been derived from our own experiments. Differences in result sources can lead to slight variations in the baseline performance of RHP, influenced by factors such as implementation details, hardware setups, or random seeds.

• Clarification with an Example: For example, if the baseline TR for RHP in CoELA has been 0.51 and RHP+CoELA achieves a TR of 0.85, the EI for RHP+CoELA would be approximately 40%. In contrast, in our experiments, the baseline TR for RHP might have slightly differed, e.g., 0.52. While the TR for RHP+CaPo also reaches 0.85, the EI would be approximately 38.8%. This small variation in the baseline explains the difference in EI values, even when the final TR values are identical.

• Clarification in Manuscript: To avoid any further confusion, we have explicitly indicated the sources of baseline results, Lines 801-2 of Appendix, and explained how these variations can affect EI calculations, even for identical TR values.

We appreciate your careful attention to this detail and have ensured that these points are clearly addressed in the revised manuscript. Thank you for your valuable feedback.

Q3. Concern about the pre-defined communication protocol.

Thank you for raising the concern about the pre-defined communication protocol, as reflected in the slightly worse performance of CaPo+CoELA compared to CoELA+CoELA (84 vs. 85). We provide the following clarification:

• Robustness of CaPo in heterogeneous settings: The slight performance drop is predictable, as CaPo+CoELA involves heterogeneous agents, where CaPo independently generates and updates meta-plans. This requires CaPo to adapt its strategies to align with CoELA’s planning and communication mechanisms, which may not be fully compatible. Beside, RHP+CaPo also achieves slightly higher efficiency than RHP+CoELA (82 vs 81), demonstrating the feasibility of CaPo cooperating with diverse agents without pre-defined communication protocol.

• Higher Efficiency in Homogeneous Settings: In homogeneous settings, CaPo+CaPo demonstrates significantly higher efficiency compared to CoELA+CoELA (89 vs 85). This improvement is attributed not only to the pre-defined communication protocol but also to our proposed meta-plan mechanism and progress-adaptive meta-plan updating. We hope these findings will inspire further advancements in the community.

• Acknowledging Limitations: Slight worse performance compared with CoELA+CoELA highlights the inherent challenges of CaPo in heterogeneous multi-agent cooperation setting . We have added a discussion of this limitation in LINE 536-39 to reflect this insight.

We appreciate your thoughtful feedback and hope this explanation clarifies the observed results and addresses your concerns.

Q4: Is setting the discussion budget to 3 rounds a reasonable choice given this observation?

We set the discussion budget to 3 rounds to balance the consensus rate and overall efficiency. As shown in the following table, with a 3-round limit, the failure rate decreases to 21.1%, and overall efficiency improves to 84%. Extending the discussion budget beyond 3 rounds offers only marginal benefits in reducing failed cases (e.g., a reduction to 17.8% with 4 rounds) while providing no additional efficiency gains, as efficiency stabilizes at 84%. These results demonstrate that a 3-round discussion budget is a reasonable choice, effectively balancing communication costs and task performance. We include these findings in the supplementary material for added clarity.

Thank you for the insightful and valuable comments! In the following, we provide our point-by-point response and hope our response helps address your concerns. We also look forward to the subsequent discussion which may further help to solve the current issues.

Q1: Can the author further illustrate their contribution compared to CoELA? CaPo reused some modules from CoELA, and 2 proposed modules can be easily implemented by prompting LLM?

• We respectfully disagree that the contributions are limited compared to the original CoELA framework. We introduce two key innovative strategies, that are not in the original framework, and that significantly improve the cooperation efficiency of embodied agents in the experiments:

  • Meta-plan generation mechanism that provides long-term strategic guidance, enabling agents to form coherent, goal-aligned plans that enhance coordination efficiency in decentralized settings. By contrast, CoELA relies on ad-hoc or reactive adjustments.This meta-plan mechanism ensures consistent and structured decision-making, crucial for complex multi-agent tasks.
  • Progress-adaptive meta-plan and execution strategy that dynamically adjusts and refines the meta-plan based on real-time progress, ensuring adaptability and responsiveness in unpredictable environments. This adaptiveness addresses the limitations of static or pre-defined strategies, particularly in scenarios with unexpected challenges.

• These contributions are both novel and impactful, advancing the field of LLM-driven embodied agent cooperation by introducing a dynamic escalation mechanism. Unlike traditional methods such as CoELA, this mechanism enables agents to actively detect and discuss potential errors or inconsistencies, particularly in decentralized settings with partial observations. Such environments often lack shared global information, leading to low cooperation efficiency, as indicated in Table 3. By introducing active intervention, our strategies prevent error propagation and significantly enhance system robustness and efficiency, as demonstrated in Tables 1 and 2.

• Moreover, for the reused modules, i.e., perception, memory, communication, and execution modules, we have moved them into the preliminary section, and only introduced the improvements on these modules in our method section. For example, comparing vanilla memory modules, we introduce plan related information like meta plan and task progress into the memory module for retrieval to discuss new meta plan or adaptation. Please refer to these changes in our new submission which will be polished better later. This will help readers clearly understand our main innovative contribution. We also emphasize that, throughout the entire initial submission, we did not claim these reused modules from CoELA as our contribution.

• Finally, while our strategies are implemented through LLM prompting—a standard technique in LLM-based embodied agent methods such as CoELA, RoCo (Mandi et al., 2023), and ProAgent (Zhang et al., 2023b)—our novelty lies in the design and application of these strategies (meta-plan generation and its progressive adaptation) and their profound impact on agent cooperation, rather than the use of prompting itself. That is, in this work, prompting is only the implementation tool of our novel strategies instead of the key contribution.

I thank the authors for their response and for addressing my questions. I appreciate the additional experiments on RoCoBench, which provide further evidence of the proposed framework's effectiveness. However, several fundamental weaknesses in this work remain unresolved.

First, the novelty of the proposed framework remains questionable. As noted in my initial review and echoed by Reviewers 8MR1 and kpSn, this work appears to be a relatively straightforward extension of CoELA. The response simply restates the introduction of Meta-plan generation and Progress-adaptive meta-plans as key contributions, which primarily rely on trivial prompt engineering. Furthermore, the performance improvements over baselines appear to hinge on the communication cost introduced by the "central agent." This represents a trade-off rather than a clear advancement, and the authors have not provided a thorough evaluation of the communication cost. I will elaborate on this issue in the next section.

Secondly, the evaluation of communication cost remains unclear. As Reviewer 8MR1 noted, "the running time includes both the model's inference time and the simulator's simulation and render speed, which does not seem a good metric to evaluate the efficiency of the method." While the authors provided updated results in their second-round response to Reviewer 8MR1, the evaluation still lacks clarity and rigor in assessing efficiency. The authors assert that 500 characters correspond to 1 frame, a setting adopted from CoELA. However, I find this conversion problematic, as communication cost cannot be directly equated to step cost in this manner. The actual cost depends heavily on the underlying communication network and hardware. A more appropriate and widely accepted metric would be the communication bandwidth or overhead during execution, as demonstrated in prior works on multi-agent communication [1,2]. A fair comparison between CaPo, CoELA, and other baselines should be conducted using such metrics to provide a comprehensive evaluation.

Given these unresolved issues, I am inclined to maintain my original rating.

[1] Wang, Yuanfei, et al. "Tom2c: Target-oriented multi-agent communication and cooperation with theory of mind." ICLR 2022.

[2] Ding, Ziluo, et al. "Learning individually inferred communication for multi-agent cooperation." NeurIPS 2020.

Dear Reviewer zu5a,

We sincerely appreciate your valuable comments. We have provided a detailed response in the boxes above to address your concerns. Here is a summary:

• Clarifying contribution

  • Both reviewers 8MR1 and Z2bg acknowledged the contributions of this work in advancing embodied multi-agent cooperation, particularly in enhancing cooperation efficiency through the novel meta-plan mechanism and progress-adaptive meta-plan updating.

  • To better highlight the contributions of this work, we have revised the manuscript, such as rewriting the introduction section and moving the introduction of the baseline CoELA to the preliminary section for improved clarity.

• Clarifying communication cost

  • We have provided a detailed evaluation of the communication cost in Appendix A6, demonstrating that our method achieves a good balance between efficiency and communication cost.

  • Both reviewers 8MR1 and Z2bg acknowledged the thoroughness and validity of our communication cost evaluation.

  • We have also cited the papers you provided and will evaluate communication cost with bandwidth or overhead metrics in future work, given the significant workload and limited rebuttal time.

We sincerely wish that our response has addressed your concerns, and turned your assessment to the positive side. If you have any more questions, please feel free to let us know. We appreciate your time and constructive suggestions! Thank you!

Dear Reviewer zu5a,

The discussion period closes in 1 day. We are trying best to address all your concerns with new results, clarifications, and an updated manuscript. Please let us know if you have any remaining concerns. We sincerely wish that our response has addressed your concerns, and turned your assessment to the positive side.

Best,

Authors

Thank you for the insightful and valuable comments! In the following, we provide our point-by-point response and hope our response helps address your concerns. We also look forward to the subsequent discussion which may further help to solve the current issues.

Q1: The main content of the article appears to be an extension of CoELA, enhancing its single-step planning approach by incorporating multi-turn discussions after the execution phase.

• We respectfully disagree that the contributions are just an extension of CoELA compared to the original CoELA framework. We introduce two key innovative strategies, that are not in the original framework, and that significantly improve the cooperation efficiency of embodied agents in the experiments:

  • Meta-plan generation mechanism that provides long-term strategic guidance, enabling agents to form coherent, goal-aligned plans that enhance coordination efficiency in decentralized settings. By contrast, CoELA relies on ad-hoc or reactive adjustments.This meta-plan mechanism ensures consistent and structured decision-making, crucial for complex multi-agent tasks.

  • Progress-adaptive meta-plan and execution strategy that dynamically adjusts and refines the meta-plan based on real-time progress, ensuring adaptability and responsiveness in unpredictable environments. This adaptiveness addresses the limitations of static or pre-defined strategies, particularly in scenarios with unexpected challenges.

• These contributions are both novel and impactful, advancing the field of LLM-driven embodied agent cooperation by introducing a dynamic escalation mechanism. Unlike traditional methods such as CoELA, this mechanism enables agents to actively detect and discuss potential errors or inconsistencies, particularly in decentralized settings with partial observations. Such environments often lack shared global information, leading to low cooperation efficiency, as indicated in Table 3. By introducing active intervention, our strategies prevent error propagation and significantly enhance system robustness and efficiency, as demonstrated in Tables 1 and 2.

Q2: [2,3] have already demonstrated multiple rounds of discussions in various contexts, so the novelty and significance of this article's contribution seem somewhat lacking.

• Thank you for bringing up prior works [2, 3] on multi-round discussions, which have indeed explored this concept in different contexts. Specifically, [2] focuses on enhancing the mathematical and strategic reasoning capabilities of large language models (LLMs) through multi-round debates between distinct LLM instances. On the other hand, [3] introduces a solo performance prompting strategy, transforming a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration using multiple personas.

• Our work, however, fundamentally differs from these approaches in two key aspects:

  • Distinct Objectives: While [2] and [3] investigate multi-round discussions to improve factuality, reasoning ([2]), or cognitive synergy ([3]) of LLMs, our work is specifically designed to enhance cooperation efficiency among embodied agents in decentralized settings, where agents operate with partial observations. This shift in focus addresses a unique and practical challenge not considered in the aforementioned studies.

  • Innovative Strategies: We introduce two novel strategies tailored for embodied agent systems: the Meta-Plan Generation Mechanism and the Progress-Adaptive Meta-Plan and Execution Strategy. These strategies are explicitly designed to significantly improve coordination and efficiency in decentralized multi-agent environments, addressing complexities that are beyond the scope of [2] and [3].

• By focusing on decentralized embodied agent cooperation and offering innovative, application-specific strategies, our work addresses a unique problem space and provides a tailored solution distinct from prior methods.

Thank you for the insightful and positive comments! In the following, we provide our point-by-point response and hope our response helps address your concerns. We also look forward to the subsequent discussion which may further help to solve the current issues.

Q1.The experiments could benefit more from being evaluated on real humans collaborating with LLM agents on tasks such as Watch and Help. Extra evaluations could show the same efficiency metrics but also metrics indicating which agents humans preferred playing with. If CaPo is very verbose when making plans, humans may not prefer interacting with it despite the efficiency gains compared to a method like CoELA. It would be interesting to dissect this. Nice to have - what are results from playing this method with humans compared to other baselines? What do humans prefer?

To evaluate human-agent cooperation, we follow the methodology of CoELA and conduct experiments on the C-WAH tasks, where the agent Bob (meta-plan evaluator) is controlled by real human participants. We adopt average steps as an efficiency metric and also ask three participants to rate their teammate agent (i.e., CaPo or CoELA) on a 7-point Likert Scale based on three criteria: communication effectiveness, helpfulness, and trust. The following results and discussion have been included in Appendix A7.

Based on above table, we have following findings:

• Efficiency (Average steps). Humans collaborating with CaPo generally achieve higher efficiency, completing tasks with fewer steps, i.e., 44 steps. This is because both agents and humans can avoid redundant actions through the guidance of the proposed meta-plan.

• Human Preferences

  • Communication effectiveness represents the effectiveness of communication between agent and human participants, e.g., whether the agent understands the message and provides useful information. In general, CaPo demonstrates clearer and more reliable communication than CoELA (6.3 vs. 5.8). This improvement is attributed to CaPo’s multi-round discussion mechanism, which creates a long-term, coherent meta-plan, reducing the need for frequent communication. In contrast, in CoELA, agents and humans have to exchange information more frequently, due to the lack of a coherent plan. Finally, agents didn’t show verbose behavior when making plans (as shown in Figure 3), as we constrained the LLM to generate concise plans and messages in prompting.
  • Helpfulness indicates whether the agent helps human participants complete the task faster. By multi-round discussion and creating coherent meta-plans, CaPo could help both agent and human participants to reduce redundant actions in the embodied environments, e.g., aimless exploration.
  • Trust measures how human participants trust their agent teammate. Humans show a stronger preference for trusting CaPo (6.3 vs. 6.0) during the cooperation. This is because CaPo involves humans in the planning process, seeking their input during multi-round discussions to collaboratively create the meta-plan. In contrast, CoELA agents act more independently and sometimes fail to respond to human messages, which weakens trust.

Through human-agent cooperation experiments, we demonstrate that CaPo enhances communication clarity, actively supports human decision-making, and fosters trust by engaging humans in the planning process. These advantages position CaPo as a more favorable teammate for human collaborators, underscoring its potential for real-world cooperative applications.