Multi-Grained Knowledge for Retrieval-Augmented Question Answering on Hyper-long Contexts

Weakness3 - Different Granular Levels Thanks for your suggestion, we have conducted additional ablation experiments focusing on micro, feature, and macro granularities across datasets.

These results illustrate the necessity of multi-grained representation for handling hyper-long contexts.

Weakness4 - Differences from Prompt/Context Compressing Methods Thank you for your thoughtful suggestions and for highlighting relevant works on context compression. While the overarching goal of handling long texts aligns across both our method and the suggested compression-based approaches, the methodologies differ significantly. The mentioned methods primarily focus on compressing long contexts (typically around 1M tokens) to fit within the input constraints of existing models. In contrast, our approach is designed to directly address hyper-long contexts (e.g., InfiniteBench’s 2,068.6k tokens) by constructing and leveraging multi-grained entity graphs and iterative retrieval mechanisms. Regarding baseline selection, we prioritized algorithms with publicly reported metrics on hyper-long context benchmarks such as LongBench and InfiniteBench. As the suggested compression-based methods have not been evaluated on these datasets, they were not included in our comparisons. We deeply appreciate your recommendations and will explore integrating context compression techniques with our framework in future work to expand its applicability and performance across diverse scenarios.

Dear reviewer ad5h,

Thank you for taking the time to read our paper. We hope our responses adequately addresses your concerns.

---

Weakness1 - Efficiency of Our Method

Following your great advice, we evaluate the computational efficiency of our proposed method on InfiniteBench (a long-document dataset) and MultiFieldQA-zh (a relatively short-document dataset), and the results are shown as follows.

Our method achieves significant improvements in accuracy and F1 scores compared to GPT-3.5Turbo-16k and RAG (Ernie-3.5-8k), while maintaining competitive inference times across both datasets. The average iteration rounds demonstrate the adaptability of our iterative retrieval mechanism, dynamically refining retrieval results based on query complexity. This configuration effectively balances retrieval quality and computational efficiency, showcasing the practicality of our approach for both long and short text scenarios.

Weakness2 - Other Entity Extraction Methods Thank you for your insightful feedback. Before adopting EntiGraph, we tried other entity extraction approaches, such as BERT-based Named Entity Recognition (NER) models and various Graph Convolutional Network (GCN) methods. However, we found that these methods are limited to processing hyper-long contexts and complex relationships. We compare our EntiGraph with DeepKE, a representative method. The experimental results are shown as follows, which demonstrate the strong performance of our EntiGraph.

Although our experimental results demonstrate that EntiGraph ensures high precision and significantly improves recall, effectively addressing some limitations of traditional methods, we acknowledge that our work does not specifically focus on scenarios involving sparse or ambiguous entity relationships. We believe this represents an important direction for future research, and we plan to further investigate methods to better handle such cases and assess their impact on downstream tasks.

Dear reviewer qk3D, Thank you for valuable suggestions! Please see the following for our point-by-point reply.

---

Weakness1 - The Key Innovations Entity-Centric Multi-Grained Knowledge Representation: Traditional RAG methods, such as chunk-based retrieval, struggle to integrate fragmented semantic information and often lose global context. MKRAG addresses this by leveraging entity-centric multi-grained graph representations (micro-, feature-, and macro-level) to connect and synthesize both local and global contexts. This ensures that all relevant information is cohesively linked, overcoming input-length constraints and achieving precise, detailed answers for hyper-long context QA.

Iterative Retrieval with Entity-Driven Reasoning: Single-pass retrieval methods lack the depth to resolve complex multi-entity questions, often leading to incomplete or incoherent results. MKRAG’s LoopAgent dynamically refines queries across multiple iterations by focusing on entities and their relationships. This iterative mechanism, combined with reasoning, ensures that the retrieval process captures all critical connections and provides semantically coherent results, significantly outperforming single-round or static retrieval strategies.

Comprehensive Understanding for Contextual Coherence: Unlike shallow graph-based RAG approaches that focus on limited entity relationships, MKRAG uses its EntiGraph system to extract fine-grained entities, attributes, relationships, and events, linking them into a unified temporal and contextual framework. By pruning irrelevant entities and aggregating temporal information, MKRAG connects all dispersed details into a coherent structure. This holistic approach ensures that even hyper-long texts with scattered information are fully captured, providing detailed and contextually rich QA responses.

Weakness2 - The Scalability in Resource-Constrained Environments Entity Extraction with Resource-Efficient "Offline" Processing: The entity graph and multi-grained representations are extracted in a manner that can be viewed as "offline". The one-time process significantly reduces runtime computational costs by amortizing the fixed extraction effort across multiple queries and users. This ensures scalability and efficiency in handling large datasets.

Iterative Retrieval for Rare Complex Queries: The iterative retrieval mechanism is primarily designed to address rare and highly complex queries through query rewriting and refinement. For experiment datasets, the average number of retrieval iterations remains low (e.g., 1.34 for InfiniteBench and 1.09 for MultiFieldQA-zh), meaning the approach adds negligible overhead while maintaining the flexibility to adapt to outlier cases.

Weakness3 - Experimental Details Hardware specifications: Our experiments were conducted on a cluster with Nvidia A100 GPUs (40GB, 4 cards). Stopping condition for iterative retrieval: The average number of iterations in the iterative retrieval process is 1.34 on InfiniteBench and 1.09 on MultiFieldQA-zh, as shown in Table 6 of the paper. The stopping condition is determined by the LoopAgent, which evaluates whether the retrieved information is related to the query. If relevant, the system outputs the answer; otherwise, it rewrites the query for further refinement. Convergence Criteria: Since only two modules, Entity Extraction and Multi-Grained Representation, undergo fine-tuning, their convergence is determined by the performance on validation set.

Weakness4 - Model Details

1. EntiGraph: Yes, EntiGraph is a distilled lightweight LLM, specifically optimized for entity extraction and multi-grained representation tasks.
2. Multi-grained QA pairs: The purpose of generating multi-grained QA pairs is to comprehensively extract information from the text. At the micro-level, the goal is to capture fine-grained details, such as specific attributes or relationships for entities. The feature-level expands this by integrating more attributes, relationships, and events to provide contextual richness. The macro-level combines all this information globally, enabling reasoning across entities. For instance, the model can leverage relationships like A is related to B and A is related to C to infer connections between B and C, thus creating a holistic view of the information.
3. The strength of multi-grained strategy: This strategy ensures highly accurate answers by addressing the fragmentation of information often seen in long and complex texts. By dynamically combining local and global contexts, the method connects dispersed information, reducing loss and enhancing reasoning capabilities. Additionally, the multi-level granularity supports adaptability to various query types, from detail-specific to complex reasoning.
4. The redundancy of multi-grained strategy: While this strategy may introduce some redundancy, it is managed through mechanisms like entity pruning (Section 4.3) and adaptive granularity selection, which minimize unnecessary outputs. These measures ensure that only relevant QA pairs are used during retrieval, balancing comprehensiveness and efficiency.

Weakness5 - The Optimization Process of The Model Joint Optimization of Modules:The model is designed to allow modular operation without requiring joint optimization. This design ensures flexibility and adaptability to various production scenarios while reducing overall complexity. The Entity Extraction Module and Multi-Grained Representation Module are optimized independently through fine-tuning and aligned data processing. The Iterative Retrieval Agent dynamically refines queries and integrates results without requiring training, leveraging the representations from the other modules. Training Data Construction in entity extraction: a. Coarse-Labeled Data Preparation: We used GPT-4 to extract entities from texts in domains such as novels and financial reports, focusing on two categories: key entities and generic entities. b. Manual Refinement: The coarse-labeled data was manually screened and annotated, resulting in a high-quality dataset of approximately 3k samples. c. Synthetic Sample Generation: GPT-4 was further used to generate additional samples by synthesizing attributes, relationships, and events for annotated entities. Multi-Grained Representation Module: Data from the entity extraction process was used to construct QA pairs across micro, feature, and macro granularity levels, ensuring alignment with the downstream question-answering task.

Question3 - Average Number of Rewrites & Computation Cost of LoopAgent We supplemented experiments and the results have been added to the revised manuscript. The details are shown below:

On the test datasets, the maximum iteration rounds were set to 2, with average iteration rounds of 1.34 (InfiniteBench) and 1.09 (MultiFieldQA-zh). The inference time for our method ranged from 1.89s to 2.03s.

Question4 - Short Tokens We only decompose the query if it contains multiple entities. If the query contains only one entity, the number of subquery is 1.

Dear reviewer jrNj,

Thank you so much for these very valuable and constructive suggestions! Please kindly find the point-to-point responses below.

---

Weakness1 - Effectiveness of Information Extraction Module & Weakness3 - Information Missing of EntiGraph

Thanks for your feedback!

The performance of the EntiGraph module is thoroughly evaluated in the revised manuscript, as shown below:

Compared to BERT-based methods, EntiGraph achieves a significantly higher F1 score of 89.3% (vs. 27.4%) and extracts six times more entities. This ensures comprehensive coverage and reliability of the information used for downstream tasks. Information loss is addressed by leveraging a specialized entity extraction model to maximize the capture of necessary details.

Weakness2 - Effect of threshold $\tau$ We conduct experiments to analyze the effect of threshold $\tau$, and the results are as follows.

Weakness4 - Chuck Size The chunk size used in MKRAG is 500 tokens. We recall top-k chunks, not only one. Hence, the chunk size cannot be set to 4k tokens. Besides, the longer the chunk size, the more scattered the information the model needs to pay attention to. After experiments, we found that the chunk size of 500 tokens is the most appropriate.

Weakness5 - Online Test We have reported the result of online test in Section 5.4. However, due to data sensitivity, it is recommended not to evaluate other baseline models on the online platform. Hence, we conduct experiments on public datasets and compare the experimental results with them to illustrate the effectiveness of our approach.

Question1 - The Weights of Entities $w_k$ represents the importance weight of each entity. We define four weight types as inputs for the baseline LLM (Ernie-3.5-8k): general or ambiguous entities (e.g., "villager" or "merchant") - 0.3; related to the context, describing a specific entity (e.g., "Zhu Bajie's wife") - 0.5; rare or unique entities, often with unique names or backgrounds (e.g., "White Bone Demon") - 0.7; core entities, including only entities that are indispensable to the context (e.g., "Sun Wukong") - 1. Thanks for your advice, we will add this part in the manuscript.

Question2 - Module Performance in Subtask We evaluated the performance of each module, and the results are shown below. All details have been added to the revised manuscript.

Entity Extraction Module

Effectiveness of Entity Pruning

The experiment demonstrates that a pruning threshold (τ) of 0.5 achieves the best balance between Precision and Recall, detailed in Appendix.

Performance of Multi-Grained Representation

Dear reviewer vkcq, Thank you so much for these very insightful and constructive comments, please see the following for our point-by-point responses.

---

Effectiveness of Information Extraction Module

Following your great advice, we conduct more experiments to analyze the reliability of information extraction module, and the results are shown as follows.

Our experiments demonstrate that LLM-based methods significantly outperform BERT-based methods across all metrics. Additionally, the survey "Large Language Models for Generative Information Extraction: A Survey" (https://arxiv.org/pdf/2312.17617) highlights extensive work leveraging LLMs for IE tasks, including entity extraction, relation extraction, and event extraction. These studies and results collectively demonstrate the superiority and reliability of LLMs in information extraction tasks.

Comparison with other ERNIE variants

The variants of ERNIE do not contain ERNIE-Turbo-128K, andd the maximum length of ERNIE-Turbo is 8k. ERNIE has a simple version of ERNIE-Speed-128K. ERNIE-Speed-128K has longer input tokens and faster inference speed, but its performance is not better than the baseline model (i.e., Yi-6B-200K, Kimi-Chat, Chatglm3-6B-128K). We have compared our method with these baseline models, and the experimental results demonstrate the effectiveness of our model.