MeMAD: Structured Memory of Debates for Enhanced Multi-Agent Reasoning

Thank you for your insightful review! We are happy to clarify the points that may have caused confusion.

R1 & Q1：Regarding the Need for a MAD-specific Memory Framework

We suspect there might be a misunderstanding regarding our use of the term "framework" in MeMAD. We are NOT proposing a new memory infrastructure system akin to LangChain or MemGPT. Instead, MeMAD represents a methodological framework focused on systematically extracting, structuring, and reusing debate experiences from MAD processes. Our implementation actually utilizes existing tools—we built upon AutoGen and could equivalently use LangChain. Our key innovation is specifically in what we store and how we reuse MAD experiences, rather than the memory system itself.

For instance, MemGPT addresses the limitation of constrained context windows by leveraging virtual memory paging between main memory and disk, similar to traditional operating systems. Such methods focus primarily on memory forms (e.g., chunks or graphs) and operations such as reading, writing, and forgetting. In contrast, MeMAD prioritizes the content of memory, specifically the structured acquisition and effective utilization of debate experiences, a facet relatively underexplored in existing memory-augmentation approaches.

R2: Regarding Comparison with Inference-Time Training.

We appreciate your suggestion regarding Inference-Time Training approaches. When designing MeMAD, we specifically targeted scenarios where modifying model parameters is impractical—such as when using API-based LLMs. Many high-performing models (e.g., GPT-4, Claude3.7) do not provide training access, thus making parameter-free approaches essential for practical deployment.

We concur that storing memories in model parameters is fundamentally appealing. However, from our perspective, Inference-Time Training may potentially face certain challenges: (1) ensuring parameter updates do not negatively affect the model's existing knowledge during inference, and (2) managing substantial computational costs associated even with minimal parameter updates, especially at scale. Further exploration into these methods is warranted.

Due to current resource constraints, we are unable to perform experiments involving parameter updates.

R3 & Q2: Regarding Experimental Setup Clarity

Our method comprises two phases:

• Experience Accumulation Phase (Section 4.1): We use a separate training dataset (detailed in Appendix B) where each sample undergoes the full debate process to generate structured experiences, stored in the Memory Bank. Importantly, no memory retrieval occurs during this phase - each sample's experience accumulation is independent.
• Retrieval and Inference Phase (Section 4.2): Each test sample accesses the complete Memory Bank accumulated from the experience accumulation phase. The Memory Bank remains fixed during testing—no new experiences from test samples are added. Thus, the order of test samples does not influence results, as each sample accesses identical memory resources.

This design ensures fair evaluation and avoids the complexities associated with continuous learning during testing, which would require accurately judging response correctness without ground truth—a substantial challenge in complex reasoning tasks. We believe this area merits future exploration.

R4 & Q3： Regarding Limited Complexity Reduction

We acknowledge MeMAD demonstrates only marginal complexity reduction compared to MAD. However, our primary objective is performance improvement rather than cost reduction. As MeMAD incorporates memory retrieval, it naturally incurs additional costs. Our goal was to minimize this cost increase while achieving significant accuracy gains.

Regarding cost calculations: We measured only inference-time costs, excluding memory generation costs. Memory generation represents a fixed, one-time expense amortized across all test samples; consequently, as the number of inference samples grows, this initial cost per sample becomes negligible.

We hope these clarifications address your concerns. If you have any further questions or concerns, we warmly welcome continued discussion.

Thank you for your detailed review. Below, we address your concerns:

R1："Lacks a thorough investigation into scenarios where MeMAD fails or underperforms."

Our evaluation covers diverse domains to prevent "fitting" to benchmarks. Specifically, we evaluated MeMAD across four distinct domains: mathematics (MATH500), science (GPQA, covering physics, chemistry, biology), law, and economics.

We have already evaluated robustness to domain shifts. In Section 5.4 (Table 5), we demonstrate cross-task transferability by applying experiences accumulated from MATH to MMLUPro-Math. Despite significant format differences (open-ended vs. multiple-choice), MeMAD maintains a 4.0% improvement over the MAD baseline (0.765 vs. 0.725), demonstrating robust transfer capabilities.

Regarding noisy memory and adversarial examples, we recognize these as valuable areas for future exploration and intend to address them in subsequent studies.

R2: "Core components (self/peer reflection, semantic retrieval) adapted from prior works, limiting conceptual advancement."

Prior works ([1,2,3,4]) primarily optimize performance within a single debate or collaborative session without reusing accumulated experiences. For instance, Self-Refine iteratively improves answers within the same question instance, and [1] encourages divergent thinking within individual debate episodes. Corex [3] proposes Discuss, Review, and Retrieve modes.

In contrast, MeMAD is the first to systematically accumulate experiences across multiple debates and reuse them for future tasks. Our key innovation is systematically extracting, structuring, and reusing historical experiences from the MAD process.

R3: "Omission of memory-augmented single-agent baselines (e.g., MemGPT)."

We compared MeMAD against memory-augmented single-agent baselines in Appendix D.1 (Figure 3). The memory-augmented single-agent baselines utilize the same memory retrieval mechanism as MeMAD.

Results (replicated below from Appendix D.1 for clarity) show that memory augmentation alone for single agents (GPT-4o-mini + Memory) did not surpass traditional MAD performance, confirming that both memory augmentation and multi-agent debate are essential and complementary.

Regarding MemGPT specifically, MemGPT addresses the problem of "limited fixed-length context windows" in long conversations or document analysis by treating constrained context as a memory source. Therefore, MemGPT may not be suitable for our complex reasoning tasks.

R4: "Memory retrieval solely based on embedding similarity, limiting robustness."

We conducted empirical analyses to evaluate the robustness and generality of our retrieval mechanism in Section 5.3 (Detailed Analysis and Ablations of MeMAD).

1. Encoding methods (Table 3): We compared four encoding methods across all datasets—token-overlap-based BM25 and three embedding-based methods (bge-m3, nomic-embed-text, mxbai-embed-large). MeMAD consistently improves performance with all encoding methods compared to other baselines, with embedding-based approaches outperforming BM25. This demonstrates robustness regarding encoding choices.
2. Retrieval strategies (Table 4): On the Economics dataset, we analyzed different selection strategies (Random, Similarity, Diversity) across different memory configurations ($\mathcal{M}^+$, $\mathcal{M}^-$, $\mathcal{M}^+ \cup \mathcal{M}^-$). All retrieval strategies outperform the MAD baseline. This comprehensive evaluation demonstrates that memory retrieval benefits are robust across various strategies.

The consistent improvements across all encoding methods and retrieval strategies validate the robustness of our approach, addressing concerns regarding sensitivity to specific retrieval implementations.

Q1: "Some part of the paper, especially in the part about memory retrieval and feedback prompts is a little bit unclear"

The memory retrieval process (Section 4.2) utilizes semantic similarity to identify relevant experiences, with detailed steps provided in the section. The feedback prompt templates are fully specified in Appendix E.1 (Tables 8-9). We are happy to clarify any specific aspects that remain unclear.

We hope these clarifications address your concerns.

Thank you for your review and positive feedback. Your positive comments are encouraging for us.