Thank you very much for your valuable comments and constructive suggestions on our paper. Your insights have helped us identify key areas for improvement, and we have carefully addressed each point as follows.

---

Q1: Inconsistency in experiments across datasets

• Table 1 (MLVU dataset): The core purpose is to verify the universal improvement of AdaVideoRAG on the base MLLM models. Therefore, we included multiple groups of mainstream MLLMs (such as GPT-4o, Qwen2.5-VL, LLaVA-Video, VideoLLaMA3). Through the comparison of "with vs. without AdaVideoRAG", we demonstrate that the framework can stably enhance the performance of different models. Finally, VideoLLaMA3, which shows the most significant improvement in the results, is selected as the benchmark model for subsequent experiments to ensure the representativeness of the core conclusions.
• Table 2 (VideoMME dataset): It focuses on the horizontal comparison with existing SOTA methods in the Video-RAG category. Using VideoLLaMA3, which performed the best in Table 1, as the benchmark, we compare the differences between AdaVideoRAG and methods like VideoRAG [32], highlighting the superiority of the dynamic retrieval strategy. To further address your concerns, we have added a comparison with SOTA Video-RAG methods in Table 1 (MLVU). Since the MLVU_Test benchmark, released in 2025, contains more challenging video cases, our AdaVideoRAG method can achieve greater performance gains.

---

Q2: Reliance on LLM as a judge and lack of human-annotated gold answers in HiVU

• The core goal of HiVU is to evaluate a model's ability to handle video logics of varying complexities. However, existing benchmarks (such as MLVU and VideoMME) involve relatively simple logics in their questions, making it difficult to cover such high-difficulty tasks. For the automatic evaluation of these complex tasks, there is currently no unified standard in the academic community. The scheme of using LLM as a judge has been widely adopted because it can parse long logic chains (e.g., [1r] and [2r]).
• To reduce the bias of the LLM judge, we systematically compared the evaluation results of different judge models in Table 3. Therefore, in this paper, we use the DeepSeek-32B model with chain-of-thought (CoT) logical reasoning as the evaluation arbiter for the HiVU benchmark evaluation to ensure the accuracy and reliability of the evaluation results.
• To further address your concerns, we supplemented a verification experiment: using VideoLLaMa3 as the MLLM benchmark model, we tested the reliability of DeepSeek-32B as a judge on the VideoMME dataset, which has existing manually annotated ground truths. The results show that the evaluation results of DeepSeek-32B are highly consistent with the manual ground truths, proving its accuracy as an evaluation arbiter. This verification provides a direct basis for using this model for evaluation in HiVU.

• In future work, we also plan to consider manually annotating answers for a portion of the HiVU dataset.

---

Q3: Impact of removing query classification (treating all queries as hardest)

• Impact of different classifier models on accuracy (MLLM classifiers and LLM classifiers of varying sizes): We conducted classification accuracy tests on HiVU, as HiVU has ground-truth labels for Levels. Considering the impact of the classifier on subsequent tasks, we also performed tests on the VideoMME benchmark with ground-truth labels. The results demonstrate the effectiveness of the Intent Classification model we selected:

• Furthermore, we removed the classifier and conducted classification tests on the impact of retrieval branches with different difficulty levels on accuracy, using VideoLLaMa3 as the base model on the VideoMME benchmark. The results obtained when all retrieval queries were processed through Level-1, Level-2, and Level-3 paths respectively are as follows. These results demonstrate that a single branch cannot effectively handle queries involving multiple difficulty levels. Our proposed AdaVideoRAG can adaptively route to the corresponding difficulty path according to the difficulty of the input query, which not only ensures improved performance but also achieves higher efficiency:

• To verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3).

The summary table of efficiency for AdaVideoRAG is as follows:

---

Q4: Formatting issues in Table 2

We apologize for the formatting errors in Table 2. The incorrect highlighting in the "Short" column and inconsistent notation in the "Gain" column are due to a typesetting oversight. In the revised manuscript, we will:

• Correct the highlighting to properly emphasize the best results in all columns.
• Standardize the "Gain" column to use consistent notation, i.e., "+X%" for all entries.

[1r] Guo Z, Xia L, Yu Y, et al. Lightrag: Simple and fast retrieval-augmented generation[J]. arXiv, 2024.

[2r] Ren X, Xu L, Xia L, et al. Videorag: Retrieval-augmented generation with extreme long-context videos[J]. arXiv, 2025.

---

We sincerely thank you for your detailed review and constructive feedback. Addressing these points has strengthened our manuscript: we have clarified experimental inconsistencies, added human annotations to HiVU, supplemented experiments on query classification ablation, and fixed formatting issues. We believe these revisions address your concerns and improve the rigor and clarity of our work.

Thank you for your detailed and constructive feedback on our manuscript. We greatly appreciate your recognition of the core contributions of AdaVideoRAG.

---

Q1: Efficiency Analysis

To verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3). The specific results are as follows:

• Database construction

  1. Level-1 performs direct inference without the need for database construction.
  2. The average time consumed for Level-2 construction is 351s.
  3. The average time consumed for Level-3 construction is 412s.

• In the single H20 GPU card, single-process processing program, there are significant differences in the average response time among the three scenarios: the average response time for Level-1 (no retrieval) is 8s; for Level-2 (Simple reasoning) it is 26s; for Level-3 (hard reasoning) it is 27s, while the processing time using AdaVideoRAG is 20s.

• The comparison of inference efficiency between AdaVideoRAG and VideoRAG [32] on the MLVU dataset with 100 videos using a single H20 card is shown in the following table. In comparison, our method is more efficient while achieving higher metrics (see Table 2 and Table 4).

• For more explanations about Parallelization, please refer to Reviewer szi1 (Q4).

---

Q2: Contribution of Different Retrieval Strategies to Efficiency

To clarify how retrieval strategies (none, naive, graph) impact efficiency across query types, we will add a dedicated analysis:

• L1 Queries (No Retrieval): By directly leveraging MLLMs’ inherent knowledge, this strategy eliminates the need for database interactions.
• L2 Queries (Naive Retrieval): This strategy limits retrieval to text (caption/ASR/OCR) and visual feature databases, avoiding the high cost of graph construction (e.g., entity extraction, relationship modeling).
• L3 Queries (Graph Retrieval): This strategy is dedicated to handling the most complex problems that require the longest graph construction time and reasoning latency.
• To accelerate the time efficiency of both Naive Retrieval and Graph Retrieval in graph construction and retrieval, we have engineered a multi-GPU, multi-threading framework, further boosting overall efficiency.
• During inference, to improve average efficiency, we introduced an Intent Classification Model that can integrate with the RAG architecture as a plug-and-play API without fine-tuning.

---

Q3: Practical Usage of Multi-Modal Data (Captions, ASR, OCR, Visual Features)

We will elaborate on how multi-modal data is integrated in practice:

• Text-Base (Captions, ASR, OCR):
  • Captions: Generated by MiniCPM-V [46], these provide high-level semantic descriptions of video clips (e.g., “a woman wearing red clothes walking in the rain”) and are used as primary anchors for Level-2 query matching.
  • ASR: Converted from audio via FastWhisper [33], ASR text captures dialogue and narration (e.g., “The temperature will drop tomorrow”), which is critical for queries involving speech content (e.g., “What did the speaker say about the weather?”).
  • OCR: Extracted via EasyOCR [22], OCR text captures on-screen text (e.g., subtitles, signs) and is prioritized for queries like “What does the sign at the 5th minute say?”.
• Visual-Base: Extracted via ImageBind [16], visual features complement text by capturing details hard to describe linguistically (e.g., facial expressions, object motion). For example, in Level-2 queries like “Why did the woman cry before the rain?”, visual features (e.g., tearful eyes) are fused with ASR (e.g., argumentative dialogue) to strengthen causal reasoning.
• Integration Logic: For Level-2 retrieval, we compute cross-modal similarity between query embeddings and text/visual embeddings, then merge top-K results from each modality (weighted by their relevance scores) to form the evidence pool. This ensures that no single modality dominates, especially for cross-modal queries. For the more challenging Level-3 path, the video obtained through retrieval and the results from graph retrieval are combined to construct a multi-level retrieval evidence pool for hard reasoning under Level-3.
• In addition, we have supplemented the ablation experiments on Caption, ASR, and OCR with VideoLLaMA3 as the base model on the Video-MME benchmark, as shown in the following table. The results indicate that each modality contributes to the final outcome, and the model achieves the optimal result when all modalities are incorporated.

---

Q4: Analysis of Threshold for Text-Visual Embedding Matching

We conducted ablation experiments under different thresholds with VideoLLaMA3 as the base model on MLVU_Test, and the quantitative results are shown in the following table. Finally, we selected the hyperparameter of 0.5, where it effectively filters out redundant information while preserving valid information.

---

Q5: Impact of Different LLM Backbones

Thank you for pointing out the issue regarding the impact of different LLM backbone models. We have supplemented relevant experiments to improve the analysis:

• Expanding the testing of baseline models on HiVU. We have added more mainstream MLLMs as baseline models (including Qwen2.5-VL and InternVL2.5, etc.) to test the adaptability of AdaVideoRAG on the HiVU benchmark. The overall results are shown in the following table, and detailed results will be added to the revised version. It can be seen that AdaVideoRAG has strong adaptability, which can stably improve the results of different baselines.

• We further compared AdaVideoRAG with mainstream long video understanding methods. We selected COT-based 2CoF [1r] and Agent-based VideoMind [2r] to compare with our RAG-based method on the VideoMME benchmark:
  • COT-based CoF uses InternVL2.5-4B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results are shown in the following table: | Method | Overall Score on VideoMME | | :----: | :----: | | InternVL2.5-4B | 54.9 | | CoF | 59.7 | | AdaVideoRAG | 63.3 |
  • Agent-based VideoMind uses Qwen2-VL-7B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results consistently demonstrate the effectiveness of our method: | Method | Overall Score on VideoMME | | :----: | :----: | | Qwen2-VL-7B | 41.9 | | VideoMind-7B | 58.2 | | Qwen2-VL-7B with AdaVideoRAG | 59.1 |

---

Q6: Ablation of the Intent Classification Model

• Impact of different classifier models on accuracy (MLLM classifiers and LLM classifiers of varying sizes): We conducted classification accuracy tests on HiVU, as HiVU has ground-truth labels for Levels. Considering the impact of the classifier on subsequent tasks, we also performed tests on the VideoMME benchmark with ground-truth labels. The results demonstrate the effectiveness of the Intent Classification model we selected:

• Furthermore, we removed the classifier and conducted classification tests on the impact of retrieval branches with different difficulty levels on accuracy, using VideoLLaMa3 as the base model on the VideoMME benchmark. The results obtained when all retrieval queries were processed through Level-1, Level-2, and Level-3 paths respectively are as follows. These results demonstrate that a single branch cannot effectively handle queries involving multiple difficulty levels. Our proposed AdaVideoRAG can adaptively route to the corresponding difficulty path according to the difficulty of the input query, which not only ensures improved performance but also achieves higher efficiency:

---

[1r] Ghazanfari, Sara, et al. "Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning." arXiv, 2025.

[2r] Liu, Ye, et al. "VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning." arXiv preprint arXiv:2503.13444 (2025).

Thank you very much for your valuable comments and constructive suggestions, which have provided important guidance for improving our paper. We have carefully studied each point and would like to respond in detail as follows.

---

Q1: Limited comparison with prior Video-RAG baselines

Thank you for your valuable advice. AdaVideoRAG shares a similar plug-and-play design concept with VideoRAG [32]. Since this method is relatively new and achieves good results, this paper focuses on it due to space constraints. We fully agree with your suggestion that comparing Jeong et al. [1] and VideoRAG [36] will improve the completeness of this paper, and the following content will be added to the revised version:

• We have supplemented the results of VideoRAG [36] on the VideoMME benchmark as shown in the following table (corresponding to Table 2). AdaVideoRAG achieves the optimal results, and both AdaVideoRAG and VideoRAG [32] obtain significantly higher overall scores than VideoRAG [36]. However, in simple short videos, the performance of VideoRAG [36] declines significantly because of the introduction of excessive redundant information and the filtering out of useful video clip information.

• We have supplemented the results of AdaVideoRAG and VideoRAG [36] on the HiVU benchmark as shown in the following table. The results indicate that our AdaVideoRAG also achieves significantly better results compared to VideoRAG [36].

• Since the work of Jeong et al. [1] has not yet made its source code publicly available (though there is a GitHub repository, it lacks the core running code) nor provided reproducible experimental configurations, we are currently unable to conduct a fair experimental comparison. However, we commit to updating the comparative analysis results once its complete code is released in the future.

---

Q2: Limited baseline diversity on the HiVU benchmark

Thanks for the suggestion. We agree that expanding the set of evaluated models on HiVU is crucial to verifying the generalizability of AdaVideoRAG. Specifically, we have supplemented the use of 1) Qwen2.5-VL-7B [1r], 2) InternVL2.5-8B [2r], and 3) LLaVa-Video-7B [3r] as baselines. The overall results are shown in the following table, and detailed results will be added to the revised version. It can be seen that AdaVideoRAG has strong adaptability, which can stably improve the results of different baselines.

---

Q3: Clarification on Table 4 structure and inconsistencies

• Level-Specific Columns (Level-2/Level-3): These columns validate two key aspects: 1) the differentiation of HiVU’s difficulty levels and 2) the effectiveness of hard retrieval for complex tasks. To isolate the impact of retrieval strategies, we use HiVU’s ground-truth level labels (not model predictions) to evaluate model results across different difficulty levels.

• Why no Level-1 Column: Level-1 is omitted because all methods perform identically (no retrieval needed).

• Overall Column: This column uses model-predicted level labels (not ground truth) to reflect real-world performance, directly comparing AdaVideoRAG against baselines in end-to-end scenarios.

• Left vs. Right Sections: The left section focuses on “without vs. with AdaVideoRAG” to assess the necessity and effectiveness of the AdaVideoRAG model, with the compared baseline being the basic VideoLLaMA3; the right section compares “different RAG schemes” to highlight AdaVideoRAG’s superiority over the recently powerful VideoRAG [32].

We will add a footnote stating: “Level-2/3 use ground-truth labels to isolate retrieval effects; Level-1 is excluded as results are uniform.”

---

Q4: Efficiency Analysis

To verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3). The specific results are as follows:

• Database construction

  1. Level-1 performs direct inference without the need for database construction.
  2. The average time consumed for Level-2 construction is 351s.
  3. The average time consumed for Level-3 construction is 412s. This difference stems from the fact that Level-3 requires graph construction, resulting in higher preprocessing costs. However, the dynamic strategy of AdaVideoRAG can reduce the frequency of such high-cost operations by prioritizing triggering simple retrieval in applicable scenarios.

• In the single H20 GPU card, single-process processing program, there are significant differences in the average response time among the three scenarios: the average response time for Level-1 (no retrieval) is 8s; for Level-2 (Simple reasoning) it is 26s; for Level-3 (hard reasoning) it is 27s, while the processing time using AdaVideoRAG is 20s.

• Parallelization To further improve the efficiency of actual deployment, we have implemented parallel processing in the code. We also used 100 videos from the MLVU dataset as test references:

  • For database construction with dual processes on a single H20 GPU (the maximum batch size supported by 96G memory is 2), the speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the construction time becoming 176s and 210s, respectively.
  • When further expanding to multi-card (8 H20 GPUs) database construction, the average speed of each database construction for Level-2 and Level-3 is accelerated by approximately 8 times (linear acceleration), with the construction time becoming 22s and 26s, respectively.
  • For multi-process retrieval on a single H20 GPU: the processing speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the final total processing time becoming 15s and 16s, respectively, while the processing time using AdaVideoRAG is 20s.

The summary table of efficiency for AdaVideoRAG is as follows:

• The comparison of inference efficiency between AdaVideoRAG and VideoRAG [32] on the MLVU dataset with 100 videos using a single H20 card is shown in the following table. In comparison, our method is more efficient while achieving higher metrics (see Table 2 and Table 4).

---

[1r] Bai, Shuai, et al. "Qwen2. 5-vl technical report." arXiv, 2025.

[2r] Chen, Zhe, et al. "Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling." arXiv. 2024.

[3r] Zhang, Yuanhan, et al. "Video instruction tuning with synthetic data." arXiv, 2024.

Thank you for your insightful comments and constructive feedback on our manuscript. We appreciate the careful evaluation of our work and agree with the points raised, which help strengthen the rigor and clarity of our research. Below, we address each concern in detail:

---

Q1: Choice of Intent Classifier (Qwen2.5-7B) and Its "Lightweight" Nature

• Rationale for an independent classifier: The MLLM in this framework is mainly responsible for multimodal information integration and generation, which involves a large amount of computation (such as processing video frames and cross-modal alignment). If MLLM is used for intent classification, the entire video context needs to be input into the model before inference, resulting in redundant processing of video data even for simple queries that do not require retrieval (Level 1). In contrast, the intent classifier only processes text queries. Models of the same scale reduce the feature pre-extraction process for input video frames and the number of input tokens, thereby reducing end-to-end latency. Therefore, we chose Qwen2.5-7B to achieve the functional separation of "video understanding" and "intent classification", establish a unified benchmark for experiments, and ensure efficient retrieval. In addition, Qwen2.5-7B not only undertakes the classification task but also is responsible for query decoupling and fact filtering of video clips. This "one-model-multiple-uses" design can improve the accuracy of information acquisition, which is why we did not directly use MLLM.

• "Lightweight" justification: Although Qwen2.5-7B has 7 billion parameters, its computational overhead in the entire process is extremely small. As described in Sec. 2.1, the time it takes accounts for ≤5% of the total process. The reasons include: (1) It only processes text queries (no need for video frames/audio); (2) Plug-and-play is realized through local lightweight API deployment and calling; (3) It avoids the expensive cross-modal encoding step in MLLM. Compared with fixed retrieval strategies, the marginal cost of the classifier is offset by the efficiency improvement brought by simplifying the retrieval of simple queries.

• Ablation on classifier choice: To address this issue, we will supplement ablation experiments: (a) Qwen2.5-7B (our choice), (b) MLLM itself (VideoLLaMA3-7B) as the classifier, and (c) smaller models (such as Qwen2.5-1.5B). Preliminary results show that using MLLM (VideoLLaMA3-7B) as the classifier will increase latency by about 3 times (due to video context loading), but the classification accuracy will slightly decrease. Although the smaller model (Qwen2.5-1.5B) reduces latency by 24%, the accuracy drops significantly, affecting the selection of retrieval strategies. This supports our choice of Qwen2.5-7B to balance efficiency and accuracy.

---

Q2: Novelty and Significance of the Proposed Method

The reviewer notes that the motivation may seem incremental, and we aim to clarify the novelty of AdaVideoRAG:

• Beyond "combining two RAG methods with a classifier": Existing VideoRAG methods (e.g., [32, 36]) use fixed retrieval paradigms (naive retrieval or graph retrieval for all queries). AdaVideoRAG introduces a hierarchical adaptive mechanism that dynamically scales retrieval complexity with query difficulty, which is absent in prior work. This includes:
• A fine-grained intent classification system (Level-1/2/3) tailored to video-specific reasoning demands (spatio-temporal logic, cross-modal causality).
• The Omni-Knowledge Indexing module, which unifies text (ASR/OCR/captions), visual features, and knowledge graphs into a hierarchical knowledge base—enabling seamless transitions from no retrieval (Level-1) to graph retrieval (Level-3).
• Empirical evidence (Table 2) shows that this adaptivity leads to a overall +4.8（from +7.9 to +12.7） gain over fixed VideoRAG [32] with Qwen2.5-VL-7B on Video-MME. Meanwhile, Table 4 also shows that the proposed AdaVideoRAG significantly outperforms VideoRAG [32] on HiVU, which demonstrates the generalization and effectiveness of the proposed method.
• Broader impact: This work establishes a new paradigm for adaptive retrieval augmentation in video analysis, where resource allocation is optimized for real-world scenarios (e.g., simple frame-level queries vs. complex multi-hop reasoning). This is distinct from static RAG or MLLM-only approaches, as demonstrated by its universal integration with existing MLLMs (Tab. 1 shows gains across Qwen2.5-VL, VideoLLaMA3, etc.).

---

Q3: Efficiency Analysis

To verify the efficiency differences among different retrieval paths, we randomly selected 100 videos from the MLVU dataset and conducted systematic time cost tests for three scenarios: no retrieval (Level-1), simple retrieval (Level-2), and hard retrieval (Level-3). The specific results are as follows:

• Database construction

  1. Level-1 performs direct inference without the need for database construction.
  2. The average time consumed for Level-2 construction is 351s.
  3. The average time consumed for Level-3 construction is 412s.

• In the single H20 GPU card, single-process processing program, there are significant differences in the average response time among the three scenarios: the average response time for Level-1 (no retrieval) is 8s; for Level-2 (Simple reasoning) it is 26s; for Level-3 (hard reasoning) it is 27s, while the processing time using AdaVideoRAG is 20s.

• Parallelization To further improve the efficiency of actual deployment, we have implemented parallel processing in the code. We also used 100 videos from the MLVU dataset as test references:

  • For database construction with dual processes on a single H20 GPU (the maximum batch size supported by 96G memory is 2), the speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the construction time becoming 176s and 210s, respectively.
  • When further expanding to multi-card (8 H20 GPUs) database construction, the average speed of each database construction for Level-2 and Level-3 is accelerated by approximately 8 times (linear acceleration), with the construction time becoming 22s and 26s, respectively.
  • For multi-process retrieval on a single H20 GPU: the processing speed of Level-2 and Level-3 is accelerated by approximately 2 times, with the final total processing time becoming 15s and 16s, respectively, while the processing time using AdaVideoRAG is 20s.

The summary table of efficiency for AdaVideoRAG is as follows:

• The comparison of inference efficiency between AdaVideoRAG and VideoRAG [32] on the MLVU dataset with 100 videos using a single H20 card is shown in the following table. In comparison, our method is more efficient while achieving higher metrics (see Table 2 and Table 4).

---

Q4: Comparison with Other Long Video Understanding Methods (CoT/Agent-Based)

We appreciate the suggestion to connect with other long video understanding Methods methods. We will add a discussion in Related Work section highlighting key differences and complements:

• CoT-based methods (e.g., [1r,2r,3r]) focus on decomposing reasoning steps within the MLLM, enhancing internal logic but not addressing knowledge limitations (e.g., fixed pre-trained knowledge). AdaVideoRAG complements this by augmenting with external knowledge (retrieval) when needed.
• Agent-based methods (e.g., [4r,5r]) use LLM as a planner to orchestrate tools but often lack adaptive retrieval granularity. AdaVideoRAG’s intent classifier can be integrated into agent frameworks to optimize tool usage (e.g., skip retrieval tools for simple queries).
• To further visually compare with the above two types of methods, we selected COT-based CoF [1r] and Agent-based VideoMind [4r] to compare with our method on the VideoMME benchmark:
  • COT-based CoF uses InternVL2.5-4B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results are as follows:

• Agent-based VideoMind uses Qwen2-VL-7B as the baseline model, and we therefore adopted the same baseline model for AdaVideoRAG testing (using this multimodal large language model as the foundation for the classifier, disentangler, and filter). The results consistently demonstrate the effectiveness of our method:

The above content will be added to the revised version.

---

Q5: Minor Issues and Formatting

Thanks for pointing out this.

• Figure 2 typo: The labels "Sec. 3.1/3.2/3.3" will be corrected to "Sec. 2.1/2.2/2.3".
• Appendix placement: The optional technical appendix will be moved after the checklist to better comply with formatting guidelines.

Q1-r: About the memory efficiency

Thank you for your question. Regarding model scale and memory efficiency, we wish to supplement the following points based on the characteristics of the RAG framework. As a Retrieval-Augmented Generation (RAG) framework, AdaVideoRAG’s core optimization focuses on retrieval efficiency (e.g., response speed) rather than pure model parameter scale. For practical deployment, we offer two flexible solutions to balance performance and resource consumption:

• External API Mode (Recommended)
  The framework natively supports calling external APIs (e.g., Qwen2.5-7B, VideoLLaMA3-7B), eliminating the need to load full model weights locally. Memory usage can be reduced to megabytes (MB), aligning with lightweight deployment practices common in RAG systems, especially suitable for resource-constrained scenarios.
• Memory Optimization Logic for Local Deployment
  a. We have also implemented multi-threaded vLLM local deployment, while also supporting private parallel API service calls. b. Notably, the classifier runs only during the initial task phase (determining query difficulty levels) and releases resources afterward. The video understanding model (VideoLLaMA3-7B) loads dynamically based on classification results in later stages. Since these components do not occupy memory concurrently, actual usage does not reach the theoretical "7B+7B=14B" peak.
  Additionally, we provide specific memory efficiency metrics for local deployment for your reference:

Q2-r: About the performan of Qwen2-VL-7B

Thank you for your attention to the data discrepancy. We hereby explain the specific reasons and supplement the validation results:

• In the previous comparison with VideoRAG [32], to maintain consistency in experimental settings, we adopted the 32-frame video sampling strategy used by VideoRAG [32] (differing from the default 768 frames in the Qwen2-VL official technical report). This resulted in Qwen2-VL-7B achieving a performance score of 41.9 on VideoMME. The baselines and comparative results in the rebuttal, based on this 32-frame sampling rate, are fair and valid.
• To verify the impact of setup differences, we supplemented experiments using the official 768-frame sampling method (with prompts consistent with the official approach). The results are as follows. Under the same settings, VideoMind significantly underperforms compared to the RAG-based results.

• Additionally, we found that after following the Qwen2-VL-7B official settings, the expected results were not achieved. We also noted that the official VideoMME benchmark does not report Qwen2-VL-7B results. Furthermore, referencing the experimental results(Table 3) of Qwen2-VL-7B in the paper [1r], the obtained results were similar—both around 58 points. Therefore, to ensure fairness across models, we primarily validated the effectiveness of the proposed AdaVideoRAG method through locally reproduced experiments.

Q3-r: About the performan of InternVL2.5-VL-4B

• Thank you for your attention to the data discrepancy. We hereby clarify the specific reasons and supplement with verification results:
  The previously cited results for InternVL2.5-VL-4B (54.9) and CoT-based (59.7) methods in the rebuttal were directly sourced from Table 1 of reference paper [1r], aiming to maintain consistency with InternVL2.5-VL’s experimental setup (including prompt templates, evaluation metrics, etc.). We will mark the data source in the revised version. Thank you for pointing out this.
• For further verification, we independently retested InternVL2.5-VL-4B using the official configuration (with prompts consistent with the official setup). The results show its accuracy on VideoMME is 56.5 (lower than the officially reported 62.3/63.6).
• Additionally, we found that following the official settings did not yield the expected outcomes, and we also observed that the official VideoMME benchmark did not report results for this method. Considering the need for a fair comparison, we primarily relied on locally conducted reproduction experiments to validate the effectiveness of the proposed AdaVideoRAG method. | Method | Overall Score on VideoMME | | :----: | :----: | | InternVL2.5-4B |56.5 | | CoF | 59.7 | | AdaVideoRAG | 63.3 |