ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding

We are grateful for your comments and will address your concerns point by point in the following:

Q1: Given the difference in model capacity and parameter count is unaccounted for, could the authors provide a fairer comparison with a similar capacity model, or further justification for the timing results presented in Table 3?

A1: Thank you for the question. All inference time comparisons in Table 3 are conducted under the same model architecture and parameter scale. Taking LLaVA-Video-7B as an example, we control only whether the frame selection strategy is applied, in order to evaluate its impact on inference efficiency and accuracy. The results show that introducing the frame selection strategy leads to improvements in both efficiency and accuracy compared to the original model, demonstrating that the frame selection component in the ReAgent-V framework contributes positively to overall performance and ensures the objectivity and validity of the evaluation results.

---

Q2: Could the authors clarify whether the CLIP embedding extraction is performed online during inference or as an offline pre-processing step, as this impacts the interpretation of the reported timings?

A2: Thank you for the question, and we apologize for not making this point clear in the original manuscript. All inference-time measurements reported in Table 3 include the CLIP feature extraction stage, which is performed online during inference rather than as an offline pre-processing step.

---

Q3: Could the authors provide an analysis of the failure modes that most frequently trigger the reflection mechanism? Specifically, which of the reasoning aspects outlined in the evaluation criteria tend to fail most often, necessitating refinement?

A3: We sincerely thank you for your thoughtful question. To better understand which reasoning dimensions most frequently trigger the reflection mechanism, we analyzed the reflection rates on the VideoMME benchmark using ReAgent-V with Qwen-2.5-VL 7B as the base model. Our analysis shows that when categorizing videos by length - Short, Medium, and Long - the corresponding reflection rates are 36%, 45%, and 67%, respectively. This suggests a positive correlation between reflection frequency and task difficulty, as current models tend to perform worse on longer videos. In addition, we found that tasks related to Spatial Perception and Temporal Perception exhibit particularly high reflection rates, at 57% and 62% respectively. This indicates that current models are relatively weak in handling spatial and temporal reasoning in videos, often requiring multiple rounds of reflection to generate more accurate responses.

---

Q4: Could the authors provide an analysis of the confidence distributions for each of the reflection types (conservative, neutral, and aggressive)? This would help clarify whether the prompting strategy is manipulating the agent's output confidence.

A4: Thank you again for your insightful question. We provide additional quantitative evidence based on experiments using ReAgent-V with Qwen-2.5-VL 7B as the base model on the VideoMME benchmark. We focused on the task with the highest reflection rate - Temporal Perception - and analyzed the average output confidence under the three reflection strategies. The conservative strategy produced an average confidence of 0.63, the neutral strategy 0.71, and the aggressive strategy the lowest at 0.45, with larger variance.

However, as shown in Figure 4, all three strategies are able to correct a portion of the answers after reflection. Moreover, combining all three strategies yields a higher overall accuracy than using any single strategy alone. This suggests that even though the conservative and aggressive strategies tend to generate lower confidence under standard model decoding, they may still introduce correct information in certain cases, ultimately leading to improved answer accuracy.

---

Q5: Typo on Line 200: Should be a space between ReAgent-V and as Line 272 appears to have an incorrect reference to Figure 4. Likely should be Figure 5. The question "What is the telling..." in Figure 5 does not make sense and is inconsistent with the text on Line 287

A5: Really thank you for your careful reading - we will correct the missing space on Line 200 (“ReAgent-V as”), fix the incorrect figure reference on Line 272 (should be Figure 5), and revise the unclear phrasing in Figure 5 to align with the question on Line 287 in the final version. We will also carefully review the manuscript to ensure overall consistency and clarity in the final version.

Thank you for your detailed and critical feedback - we have carefully reviewed each point and respectfully address them below:

Q1: (Quality and significance) Although the evaluation is extensive, on many different benchmarks, I feel the results are somewhat of limited impact, … To experimentally evaluate this, more comparisons with methods employing such approaches should be provided (for example, taken from the related works, [1][2] etc ) instead of testing multiple VLMs.

References:

[1] Videorag: Retrieval augmented generation over video corpus.

[2] Reflexion: Language agents with verbal reinforcement learning.

A1: Thank you for the insightful question. Our evaluation focuses on whether large vision-language models (LVLMs) benefit from the ReAgent-V framework under different parameter settings. As shown in the results, ReAgent-V consistently outperforms its base models and surpasses most strong baselines, such as InternVL-2.5-8B and BIMBA-LLaVA. We further conducted additional experiments comparing ReAgent-V with the retrieval-based VideoRAG approach and a Reflexion-inspired variant, as shown in the tables below. Although Reflexion [2] was originally designed for language tasks, we adapted its core mechanisms. All models were evaluated under identical settings for fairness. Due to space constraints, detailed implementation specifics will be included in the final version release.

The results show that ReAgent-V consistently outperforms [1], [2], and other baselines, demonstrating its effectiveness. All experiments were repeated three times, and our method achieves statistically significant improvements over models without ReAgent-V across all benchmarks (p < 0.01, two-tailed test), further validating its robustness.

Beyond video understanding, we evaluate ReAgent-V on two additional tasks: Video LLM Reasoning and VLA Alignment. In Video LLM Reasoning, ReAgent-V outperforms the GRPO-trained model from [3] using only 45% of the training data. In VLA Alignment, our reflection-based reward replaces the hand-crafted cost function in GRAPE [4], yielding improved performance.

References:

[3] Video-r1: Reinforcing video reasoning in mllms, 2025.

[4] GRAPE: Generalizing robot policy via preference alignment, 2025.

---

Q2: (Clarity) Discussion/tests about the sensitivity of the entropy-based frame selection to the various hyperparameters is missing.

A2: Thank you for the question. We provide a robustness analysis of the two key hyperparameters in Eq(4) - the scaling factor $k$ and threshold $\tau$ - using Qwen2-VL-7B on the short split of VideoMME.

First, we fix $\tau = 0.7$ and vary $k \in \{1, 2, 3, 4\}$:

Table R1: Accuracy under different values of $k$

Next, we fix $k = 1$ and vary $\tau \in \{0.5, 0.6, 0.7, 0.8\}$:

Table R2: Accuracy under different values of $\tau$

The results show that our method is stable across a wide range of parameter values. Overly strict thresholds (e.g., $\tau = 0.8$) may select too few frames and hurt performance. We adopt $k = 1$ and $\tau = 0.7$ as default for a good balance of robustness and effectiveness, and will include this in the revised paper.

---

Q3: There are some other problems:

i) the section 2.2 on tool selection has little … How?

ii) Similarly, 2.3 could be improved. I understand the specific details are in the appendix, but I feel the main text should still include more details.

iii)Small typo in line 200: missing space 'ReAgent-V as...'

iv) In figure 5, I feel the question at the top …burger ..."

A3: Thank you for your careful reading and valuable comments. We will fix the typo on line 200 (“ReAgent-Vas...”) and revise unclear phrasing in Figure 5 (e.g., “What is the telling…”). We also acknowledge that Sections 2.2 and 2.3 lack implementation details. For tool selection, the base model determines which tools to invoke based on the question, the video content, and its own reasoning needs. In the revision, we will clarify how base models select tools based on the question and video content, and provide a more detailed explanation of tool selection and reflection strategies in the main text rather than in the appendix

---

Q4: (Originality) Besides the entropy-based frame selection, the proposed algorithm provides limited insights. … I get limited insights for possible future works, only the 'quantitative' results, and the choices made seem pretty arbitrary. I'd like to see more discussions in this sense.

A4: Thank you again for the insightful comments. The three reflection strategies are designed to mitigate overconfidence in single-strategy responses, which often cause hallucinations and errors [1, 2]. Each strategy provides a distinct perspective, and their combination enables the model to integrate complementary signals, leading to more accurate answers. As shown in Figure 4, this combined approach outperforms individual strategies on benchmarks such as VideoMME, LongBench, and EgoSchema.

To further support this, we analyzed the VideoMME benchmark using ReAgent-V with Qwen2.5-VL-7B, focusing on the Temporal Perception task where reflection is frequently triggered. The average output confidences for the conservative, neutral, and aggressive strategies were 0.63, 0.71, and 0.45, respectively, with the aggressive strategy showing the highest variance. This suggests base models are less likely to produce conservative or aggressive responses due to lower confidence, but by explicitly prompting these strategies and merging their outputs, we can capture cases that single-strategy responses may overlook. ReAgent-V’s reward module not only generates reward reports to guide data selection for GRPO/DPO training but also directly improves base model reasoning through answer revision - a capability absent in prior agent frameworks. In the Video LLM Reasoning task, using ReAgent-V’s reward reports to filter GRPO training data yields more efficient performance gains than standard methods.

References:

[1] Analyzing and mitigating object hallucination in large vision-language models

[2] VL-Uncertainty: Detecting hallucination in large vision-language model via uncertainty estimation

---

Q5: How does frame selection reduce the inference time so much? (Table 3). Is the time required by this procedure offset by the reduction in computation?

A5: Thank you for the question. While ECRS-based frame selection introduces some overhead, it significantly reduces the number of frames passed to the vision-language model - the main inference bottleneck - resulting in efficiency gains that far outweigh the selection cost.

---

Q6: Why is the amount of data used in Table 2 different for the various models? To validate the implicit claim (that ReAgent-V is better with less data) … or that ReAgent-V stays constant even after adding more data.

A6: Thank you for this interesting and careful question. In the Video LLM Reasoning application, ReAgent-V is used to filter GRPO training data via a reflection-based mechanism. As pointed out in [1], more challenging examples often lead to greater improvements in GRPO training.

Using the dataset from [2], we leveraged ReAgent-V’s reflection signals to identify difficult samples - those that triggered reflection - and retained only these difficult samples, comprising 45% of the original data. Training the base model from [2] on this subset under the same GRPO setup resulted in better performance than using the full dataset. This shows that ReAgent-V not only improves reasoning but also serves as an effective data filtering mechanism for reinforcement learning pipelines.

To further validate this, we conducted a control experiment by randomly selecting an equal-sized subset (52k samples) from the original training set. As shown below, the model trained on randomly selected data underperformed compared to both the full-data baseline and our reflection-based filtering:

These results highlight the importance of selecting informative training samples rather than relying on quantity, and confirm the utility of ReAgent-V as a high-quality data selector for downstream learning tasks.

References:

[1] Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement, 2025.

[2] Video-r1: Reinforcing video reasoning in mllms, 2025.

Thank you for your thoughtful and constructive comments, we address each of your concerns in detail below:

Q1: The selection threshold used in Eq(4) involves multiple parameters. It is unclear if the method is robust to those parameters or requires careful parameter tuning.

A1: We sincerely thank you for this thoughtful question! This is indeed an important aspect that we should have clarified in the main text. Due to time limitations, we only included partial results earlier. Here, we conduct a more comprehensive robustness study on the parameters used in Eq (4), specifically the scaling factor $k$ and the threshold $\tau$, using Qwen2-VL-7B as the base model on the short split of the VideoMME. We vary the scaling parameter $k \in \{1, 2, 3, 4\}$ while keeping $\tau = 0.7$, and observe the following results:

Table R1: Robustness Analysis of Parameters in Eq(4) on VideoMME (Short Split, Qwen2-VL-7B)

We also fix $k = 1$ and vary the threshold $\tau \in \{0.5, 0.6, 0.7, 0.8\}$:

Overall, the results show that our method is robust to the choice of parameters. Performance remains relatively stable across a wide range of $k$ and $\tau$ values. We also note that setting $\tau$ above 0.8 may result in failure to select enough frames (e.g., 32), as the threshold becomes overly strict. Based on these observations, we adopt default values of $k = 1$ and $\tau = 0.7$, which provide strong and stable performance without requiring fine-grained tuning.

---

Q2: In Table 1, the best performance is achieved with Qwen 2.5 72B, which is a larger MLLM compared to those used in prior work. Results with models of similar scale show mixed performance, making it difficult to disentangle the improvement brought by the MLLM itself from the gains introduced by the proposed method. (Please provide analysis for model robustness on hyper-parameters and discussion on the effect of a stronger MLLM backbone.)

A2: Thank you once again for your insightful question. To clarify the source of performance improvements, we conducted controlled experiments in Table 1 across different models (LLaVA-Video and Qwen2.5-VL) at both 7B and 72B scales. In both settings, incorporating ReAgent-V consistently leads to significant improvements over the corresponding base models. This indicates that the gains are not merely due to the use of a stronger backbone, but rather stem from our proposed reflection mechanism and reward-driven reasoning framework. These components enable the ReAgent-V framework to exhibit strong robustness across different model capacities.

We appreciate your insightful feedback, and we respond to your points individually as follows:

Q1: The paper lacks quantitative analysis on the accuracy of ECRS. There’s no data showing how well frames selected by ECRS align with actual keyframes related to the query. Could the authors provide more quantitative results on the selection quality of ECRS?

A1: Thank you for your insightful question! To quantitatively assess how well ECRS-selected frames align with ground-truth keyframes, we randomly sampled 100 videos from the VideoMME dataset, LongBench dataset and manually annotated the keyframes for each. We then compared ECRS against two common baselines - CLIP-based similarity and Entropy-based selection - on both VideoMME and LongBench benchmarks using five evaluation metrics: Recall, Precision, IoU, F1 Score, and Redundancy (defined as 1 minus the average pairwise CLIP similarity, where lower is better).

As shown in Table R1, ECRS significantly outperforms both baselines across all metrics. On VideoMME, ECRS achieves markedly higher Recall (0.891), Precision (0.872), IoU (0.816), and F1 Score (0.623), while reducing Redundancy (0.182) compared to CLIP (0.763/0.649/0.607/0.486/0.242) and Entropy (0.604/0.570/0.455/0.421/0.279). On LongBench, which involves long-form videos with sparse but critical evidence, ECRS maintains top performance with Recall (0.852), Precision (0.841), and IoU (0.891), again outperforming CLIP and Entropy by large margins and demonstrating its robustness in both short and long video settings.

---

Table R1: Quantitative Evaluation of Frame Selection Accuracy

---

Q2: Could the authors clarify how the conservative, neutral, and aggressive reflection strategies differ in practice by cases? How robust are these strategies to prompt during reflection?

A2: We sincerely thank you again for this interesting question. The prompts for all three strategies are provided in Appendix B.2. Specifically, when the base model's answer is deemed unsatisfactory, the conservative strategy only modifies the final answer while leaving the rest of the response unchanged. The neutral strategy makes moderate adjustments, including correcting the answer and any relevant nouns. The aggressive strategy goes further by not only correcting the answer and related nouns but also modifying the logical relationships within the sentence. For an analysis of the robustness of each strategy, please refer to Figure 4 in the main text. We report both the probability of making edits and the post-editing accuracy for each strategy. The results show that while each individual strategy can improve the response to some extent, combining all three strategies yields the highest overall accuracy.