Thank you for your positive evaluation of our work. We appreciate your recognition of our novel agentic memory system A-Mem, the effectiveness of our Zettelkasten-inspired approach with dynamic note construction, link generation, and memory evolution, and our ability to overcome the limitations of existing memory systems through dynamic memory organization. We are grateful for your acknowledgment of A-Mem's significant improvements in long-term conversational tasks and multi-hop reasoning, its strong adaptability, and its cost-effectiveness with low resource consumption. We address the points you raised to further enhance the clarity of our work.

---

Q1. Can A-mem provide a better memory retrieval strategy, rather than the most basic RAG method?

Unlike simple RAG methods, our A-mem approach employs a distinctive memory retrieval mechanism. As illustrated in Figure 2, during the memory retrieval stage, we extract query embeddings using a text encoding model and search the memory database for relevant matches. When related memory is retrieved, similar memories that are linked within the same box are also automatically accessed. We will revise Section 3.4 for clarity.

---

Q2. Is establishing links between memories solely based on similarity entirely reasonable?

Design philosophy. Our approach uses a two-stage mechanism that balances computational efficiency with semantic understanding. First, embedding similarity identifies top-k candidate memories from the existing collection, providing efficient filtering that scales without exhaustive comparisons. Second, an LLM analyzes these candidates to determine meaningful connections based on contextual relationships and semantic content in an agentic way. This hybrid architecture leverages the computational efficiency of vector similarity for initial retrieval while employing LLM reasoning to detect subtle patterns, causal relationships, and conceptual connections that transcend simple cosine similarity.

Case study. Case studies from our memory system reveal how the two-stage mechanism captures nuanced connections that pure embedding approaches miss.

Case 1. Our system connects two memories from Caroline's adoption journey. The first shows Caroline reaching out: "Hey Mel, what's up? Long time no see! I just contacted my mentor for adoption advice. I'm ready to be a mom and share my love and family." Nine days later, the second captures her progress: "Woohoo Melanie! I passed the adoption agency interviews last Friday! I'm so excited and thankful." This connection captures a clear temporal progression from preparation to achievement, demonstrating a causal relationship and narrative continuity in her adoption process.

Case 2. Our system links Caroline's artistic advocacy evolution. It connects her sharing a painting while explaining, "Representing inclusivity and diversity in my art is important to me. I also use it to speak up for the LGBTQ+ community" with her announcement eleven days later: "I'm putting together an LGBTQ art show next month and I'm gonna show my paintings. Super stoked!" This captures her progression from individual artistic expression to community action, showing how personal practice evolves into public advocacy.

These connections emerge because our LLM analysis stage recognizes thematic continuity, temporal coherence, and causal relationships that embedding similarity alone cannot capture, successfully identifying meaningful relationships through contextual understanding rather than lexical overlap.

---

Q3. Why does it not achieve the best performance in scaling experiments compared to memorybank?

For the scaling analysis, we provide a scalability assessment of our memory system. When we increase the memory size, the retrieval time increases minimally rather than intensely. Although retrieval time is slightly higher than MemoryBank [1], this is because our A-mem stores more contextual information to describe each memory node, including keywords, tags, and context. However, the increased time is quite small per request, while our A-mem provides richer memory representations that yield at least a two-fold improvement in LoCoMo comparison experiments. Based on both performance and scaling analysis, we conclude that our A-mem represents a significant advancement in building powerful and long-term memory mechanisms for LLM agents.

---

Q4. Why does increasing the k parameter not lead to improvement?

In the initial version of the paper, we regard the k value, which controls the number of relevant memories retrieved for each interaction, as a hyperparameter due to its direct influence on the balance between memory coverage and retrieval precision, requiring empirical tuning to achieve optimal performance across different scenarios.

When increasing the value of k, the length of input tokens to the LLM increases correspondingly, which may degrade LLM performance. As discussed in prior research [2-3], large language models suffer from fundamental architectural limitations, including finite context windows, quadratic attention complexity, and the "lost in the middle" problem, where performance degrades significantly when accessing information in the middle of long contexts, even for explicitly long-context models.

In the final version of the paper, we will integrate these two analyses into one section for clarity.

---

Q5. Is it reasonable for equation (3)? If it comes from Zettlekasten, how about removing 1 or 2 of the 4 factors? Are the results worse?

For the vector representation of memory notes, we utilize the attributes of context, content, keywords, and tags, which capture multiple perspectives of each memory note. To validate the effectiveness of this design, we conduct an ablation study by systematically removing each component. Here, $c$ represents the original content, $K$ denotes the keywords, $G$ represents the tags, and $X$ indicates the context. As shown in the table, each component demonstrates its effectiveness in the overall representation, with content and context being particularly impactful.

Results are reported in F1 and BLEU-1 (%) scores with foundational model GPT-4o-mini.

---

Q6. What are the experimental results in the agent systems?

To further demonstrate the effectiveness of our A-mem approach in agent systems, we conduct experiments across two scenarios: QA search and web shopping agents.

Experiments Setup. For the QA search task, we use a mixture of data from HotpotQA [4] and Natural Questions [5]. We evaluate performance using exact match accuracy and multi-objective F1 scores, comparing against the LoCoMo baseline [6] under two conditions: with and without token truncation. When multiple ground truths are available, we select the maximum F1 score across all alternatives. For multi-objective tasks, the final F1 score represents the sum of F1 scores across all sub-questions. We also report token usage statistics for all methods.

For the web shopping scenario, we evaluate on the WebShop [7] benchmark, measuring performance using the reward scores provided by the environment simulator.

Results Analysis. As shown in the following table, compared to LoCoMo, our A-mem achieves comparable performance while using only 33%-50% of the token budget, demonstrating its ability to provide more informative memory representations. This efficiency stems from our memory system's capacity to dynamically establish connections between memories based on shared attributes and continuously update existing memory descriptions with new contextual information, enabling better capture and utilization of inter-informational relationships.

Results on the QA agent task.

Results on WebShop task with GPT-4o as foundation model.

Reference:

[1] Zhong, Wanjun, et al. "Memorybank: Enhancing large language models with long-term memory." AAAI 2024.

[2] Liu, Nelson F., et al. "Lost in the middle: How language models use long contexts." arXiv:2307.03172 (2023).

[3] Kaddour, Jean, et al. "Challenges and applications of large language models." arXiv:2307.10169 (2023).

[4] Yang, Zhilin, et al. "HotpotQA: A dataset for diverse, explainable multi-hop question answering." arXiv:1809.09600.

[5] Kwiatkowski, Tom, et al. "Natural questions: a benchmark for question answering research." TACL (2019).

[6] Maharana, Adyasha, et al. "Evaluating very long-term conversational memory of llm agents." arXiv:2402.17753 (2024).

[7] Yao, Shunyu, et al. "Webshop: Towards scalable real-world web interaction with grounded language agents." NIPS 2022.

Thank you for your thoughtful and comprehensive evaluation of our work. We are deeply grateful for your recognition of A-Mem's originality and significance. We greatly appreciate your positive assessment of our comprehensive evaluation with multiple baselines, diverse metrics, and computational efficiency analysis. We are also thankful for your recognition of the manuscript's clarity and accessibility.

---

Q1. Whether the interaction between the environment and the LLM agent is equivalent to a query and the LLM’s response?

In Figure 2, the interaction between the environment and LLM agents is not equivalent to a query and the LLM's response. A chatbot, such as GPT and Claude, is one type of LLM agent that has more datasets to evaluate. However, our A-mem can be used in any kind of agent scenarios, such as coding LLM agents, embodied L/VLM agents, etc.

The environment represents the external world or task domain that the agent operates within, providing observations, rewards, and feedback based on the agent's actions. An interaction encompasses the complete cycle of the agent perceiving the environment state, taking an action, and receiving feedback or rewards [1]. Our A-mem serves a dual purpose: it records the entire process of these interactions as contextualized memory notes, and simultaneously provides relevant historical context information during each subsequent interaction to inform better decision-making.

However, due to the limitations of available long-term memory datasets, we primarily evaluate our approach using chatbot agent scenarios in LoCoMo and DialSum. To further demonstrate the effectiveness of our A-mem approach in agent systems, we conduct experiments across two scenarios: QA search and web shopping agents. As shown in the following table, compared to LoCoMo, our A-mem achieves comparable performance while using only 33%-50% of the token budget, demonstrating its ability to provide more informative memory representations. The full experiment details can be found in the response to Reviewer 3BrE Q6.

---

Q2. Does the proposed method update the memory after every single interaction? Whether forcing updates via the LLM lead to degraded performance in some situations? What are the advantages of using the LLM for memory evolution?

Storage Mechanism. The memory storage strategy can be designed to store and update memory at different frequencies. If LLM agents prefer coarse-grained memories (to save tokens and reduce computational overhead), the saving frequency can be set to a lower rate, such as at the episode level or after multiple sessions, rather than at each individual session.

Update Control. The memory evolution process operates in two stages: first, retrieving the top-k most similar memories based on text embedding similarity, then updating them via LLMs. The retrieval process is governed by a similarity threshold that determines which memories qualify for updates. By adjusting this threshold, we can control the frequency and scope of memory evolution—a higher threshold results in fewer, more selective updates, while a lower threshold allows more frequent updates across a broader range of memories.

In our benchmark evaluations, most memories exhibit strong interconnections, so we did not impose a retrieval threshold. However, for real-world deployment where memory relationships may be more sparse or varied, the retrieval threshold serves as a crucial hyperparameter. By setting an appropriate threshold, A-mem can avoid updating irrelevant or weakly related memories, thereby preventing potentially harmful updates that could degrade memory quality.

Advantage. Leveraging LLMs for memory updates allows the existing memory repository to continuously adapt and form associations with new memories, thereby capturing scenario-specific information more accurately. Additionally, the inherent pre-training capabilities of LLMs enable robust memory updating across diverse, general-purpose contexts.

---

Q3. What is the time for each part of constructing memory? Could you provide an analysis of how the retrieval time scales with the hyperparameter k?

For link generation and memory evolution, these two operations, the time cost includes two parts including the LLM calls and the retrieval time. For the LLM calls using GPT-4o-mini, the time is an average of 2.4s, which has a strong relation to your api level. For the retrieval time, it can refer to Table 4 in our paper.

Besides, we investigate the retrieval time scales with various k values. The retrieval time remains constant with respect to k due to fundamental algorithmic properties. The retrieval operation consists of four sequential components with distinct complexity characteristics. Query encoding operates at $O(1)$ with respect to k since the query embedding is computed once regardless of k. Similarity computation requires $O(N)$ time with respect to k, as cosine similarity must be computed against the entire corpus of N documents. Global sorting operates at $O(N \log N)$ with respect to k since the similarity scores for all N documents require sorting. Finally, top-k selection requires $O(k)$ time with respect to k for extracting the k highest-scoring results.

The computational bottleneck lies in the first three steps, which are invariant to k. Similarity computation dominates runtime as it requires $O(N)$ dot products between query and document embeddings. Sorting complexity depends on corpus size N, not retrieval size k. Top-k selection contributes negligible overhead, typically less than 0.1% of total time. Therefore, the total retrieval time follows:

$\text{Total Time} \approx T_{\text{embedding}} + T_{\text{similarity}} + T_{\text{sorting}} + \varepsilon(k)$

where $\varepsilon(k)$ is asymptotically negligible compared to the N-dependent terms. The key insight is that retrieval systems evaluate the entire corpus regardless of k value, with only the final extraction step scaling with k. This makes k selection performance-neutral, allowing optimization based on task requirements rather than computational constraints, since throughput depends on corpus size N rather than result set size k.

---

Q4. Other experiments do not consistently compare against all four baselines in DialSim.

We provide additional experimental results from other baselines on the DialSim dataset. Across six evaluation metrics, we can conclude that our A-mem achieves the best performance.

---

Q5. In-depth analysis explaining the reasons behind the model’s superior performance.

Our A-Mem system demonstrates superior performance by identifying meaningful semantic connections between different memories. Case studies from our memory system reveal how the two-stage mechanism captures nuanced connections that pure embedding approaches miss.

Case 1. Our system connects two memories from Caroline's adoption journey. The first shows Caroline reaching out: "Hey Mel, what's up? Long time no see! I just contacted my mentor for adoption advice. I'm ready to be a mom and share my love and family." Nine days later, the second captures her progress: "Woohoo Melanie! I passed the adoption agency interviews last Friday! I'm so excited and thankful." This connection captures a clear temporal progression from preparation to achievement, demonstrating a causal relationship and narrative continuity in her adoption process.

Case 2. Our system links Caroline's artistic advocacy evolution. It connects her sharing a painting while explaining "Representing inclusivity and diversity in my art is important to me. I also use it to speak up for the LGBTQ+ community" with her announcement eleven days later: "I'm putting together an LGBTQ art show next month and I'm gonna show my paintings. Super stoked!" This captures her progression from individual artistic expression to community action, showing how personal practice evolves into public advocacy.

These connections emerge because our LLM analysis stage recognizes thematic continuity, temporal coherence, and causal relationships that embedding similarity alone cannot capture, successfully identifying meaningful relationships through contextual understanding rather than lexical overlap.

---

Q6. The use of the same encoder for structurally different memory and query inputs appears overly simplistic, which may overlook the heterogeneity of this content.

Our approach is simple yet effective. Using a unified text embedder provides universal applicability with strong generalization—modern embedders trained on massive heterogeneous datasets demonstrate superior performance across diverse domains without requiring manual feature engineering [2]. Their excellent transfer learning capabilities enable broad applicability across real-world scenarios, providing the flexibility essential for practical deployment [3].

While future work could explore specialized architectures like graph neural networks to capture additional structural information, our current design strikes an optimal balance between strong performance, broad applicability, and ease of deployment.

References:

[1] Sutton, R. S., & Barto, A. G. (2018). The agent-environment interface. In Reinforcement learning: An introduction (Chapter 3.1). MIT press.

[2] Wang, Liang, et al. "Text embeddings by weakly-supervised contrastive pre-training." arXiv preprint arXiv:2212.03533 (2022).

[3] Cer, Daniel, et al. "Universal sentence encoder." arXiv preprint arXiv:1803.11175 (2018).

Thank you for your review. We sincerely appreciate your recognition that A-Mem addresses a significant and timely problem in LLM memory construction, which you identified as critical for advancing long-term reasoning. We're grateful you found our dynamically evolving memory structure intuitive and compelling, noting how it mirrors human memory functions. Thank you also for acknowledging our comprehensive approach that provides concrete implementation and empirical validation across six foundation models, demonstrating significant improvements over existing baselines.

---

Q1. If we already have a sufficiently capable LLM judge with comprehensive prior knowledge, is it still necessary to maintain a separate memory network? Have you conducted any experiments to evaluate the necessity of maintaining such a network under the assumption that a capable LLM judge is already available?

Even if we have a sufficiently capable LLM judge with comprehensive prior knowledge, memory systems remain essential for LLM agents. We provide a detailed explanation from the following perspectives.

Necessity of memory system for LLM. Large language models suffer from fundamental architectural limitations including finite context windows, quadratic attention complexity, and the "lost in the middle" problem where performance degrades significantly when accessing information in the middle of long contexts, even for explicitly long-context models [1-2]. Given these existing architectural constraints, it is impractical to feed all past interaction history of agents into LLMs while maintaining optimal performance. Therefore, memory systems enable LLM agents to transcend context window limitations by persistently storing experiences, abstracting knowledge, and supporting self-evolving capabilities for long-term agent-environment interactions [3].

LLMs in our A-mem. In our agentic memory system, we utilize LLMs to organize memory in a Zettelkasten-style storage structure. Additionally, we design a retriever to identify and retrieve similar information when accessing memory. By leveraging both LLMs and retrievers, our A-Mem provides condensed, informative context that enables LLM agents to transcend context window limitations with reasonable token consumption.

Experiment validation. In the Table 1-2 of our initial paper version, we conducted experiments comparing our approach against the LoCoMo method, which relies on LLMs to process all context information without memory retrieval. Despite LoCoMo [4] using approximately 7x more tokens than our A-Mem, it still underperforms compared to our method.

Conclusion. Due to the inherent limitations of LLM architecture, models cannot process excessively long contexts while maintaining optimal performance. With our A-Mem design, LLM agents can achieve superior performance through efficient memory retrieval mechanisms.

References:

[1] Liu, Nelson F., et al. "Lost in the middle: How language models use long contexts." arXiv preprint arXiv:2307.03172 (2023).

[2] Kaddour, Jean, et al. "Challenges and applications of large language models." arXiv preprint arXiv:2307.10169 (2023).

[3] Zhang, Zeyu, et al. "A survey on the memory mechanism of large language model based agents." ACM Transactions on Information Systems (2024).

[4] Maharana, Adyasha, et al. "Evaluating very long-term conversational memory of llm agents." arXiv preprint arXiv:2402.17753 (2024).

Thank you for your positive evaluation of our work. We appreciate your recognition of our key contributions regarding A-MEM, the effectiveness of our Zettelkasten-inspired approach with its three core components, and the comprehensive evaluation demonstrating consistent improvements over baselines. We are grateful for your acknowledgment of the well-designed LoCoMo and DialSim datasets, the thoroughness of our ablation studies, and the novelty of our approach in enabling systems to learn and evolve through accumulated experience. We address the points you raised to further enhance the clarity of our work.

---

Q1. Whether the approach is expensive because of LLM will be used multiple times for each memory?

As reported in the cost-efficiency analysis in the initial version of the paper, we demonstrate that memory operations using GPT-4o-mini cost less than $0.0003. Here, we provide a detailed analysis of the cost for memory operations. It should be noted that all price calculations are based on OpenAI pricing information from five months ago, and prices may fluctuate over time.

The analysis of the A-MEM memory system reveals the following key metrics:

• System Prompt Overhead: 363 tokens per operation
• Average Input Processing: 1,042.00 tokens per memory operation
• Average Output Generation: 131.00 tokens per memory operation
• Total Tokens Per Operation: 1,173.00 tokens
• For per memory storage, using the gpt-4o-mini api costs an average of 5.39 seconds, and using locally-hosted llama3.2 1B model costs an average of 1.12 seconds with a single GPU.

Based on the GPT-4o-mini pricing model ($0.150/1M tokens for input; $0.600/1M tokens for output):

• Input Cost Calculation: 1,042.00 tokens × ($0.150/1M tokens) = $0.0001563 per operation
• Output Cost Calculation: 131.00 tokens × ($0.600/1M tokens) = $0.0000786 per operation
• Total Cost Per Memory Operation: $0.0001563 + $0.0000786 = $0.0002349
• Projected Cost at Scale: For 1 million memory operations:$0.0002349 × 1,000,000 = $234.90

Conclusion. The A-MEM system demonstrates reasonable resource efficiency at approximately $0.0002349 per memory operation ($234.90 per million operations). Despite requiring multiple LLM calls during memory processing, this cost structure remains economically viable and scales predictably.

---

Q2. Could you please revise Figure 2 to remove duplicate terms like "box n+1" and "evolve"?

In the initial version of our paper, we used the terms "box n+1" and "box n+2" to indicate that these two boxes are newly added in the link generation process. In the final version, we will revise the figure for better clarity.

---

Q3. In section 4.4, please include links to Table 3 in the text.

Thank you for the suggestion. We will revise to include hyperlinks to Table 3 in Section 4.4 text.

---

Q4. Add the average number of tokens to Table 3.

For the "w/o LG & ME" method, the average token count is 605. For the "w/o ME" method, the average token count is 2,371. We will include this information in the final version of our paper. Note that the average token count is computed based on the QA test phase, excluding the memory construction phase.

---

Q5. Can you clarify the role of LoCoMo, as it appears to be used as both a baseline and a dataset?

LoCoMo [1] provides a dataset and a basic memory evaluation method in their paper. In the initial version of our paper, we used the same abbreviation to represent our method. In the final version, we will revise our method's abbreviation to avoid confusion and improve clarity.

---

Q6. Why is the memory usage the exact same for every method in Table 4?

For fair comparison, we use the same text embedding model across all three memory systems, resulting in the embedding storage occupying the majority of memory usage. We will update Table 4 in the final version of the paper to reflect this clarification.

---

Q7. Does the average token length in Table 1 include all tokens used for metadata generation during both note writing and memory evolution?

Since memory construction and QA evaluation are separate processes in the datasets, we do not include the average token cost of memory construction in Table 1. For the memory construction token cost, please refer to the response to Q1.

Reference:

[1] Maharana, Adyasha, et al. "Evaluating very long-term conversational memory of llm agents." arXiv preprint arXiv:2402.17753 (2024).