Modularized Self-Reflected Video Reasoner for Multimodal LLM with Application to Video Question Answering

We sincerely thank the reviewer for taking time to review our paper and providing insightful feedback and suggestions. We address the weaknesses and questions as follows:

Methods And Evaluation Criteria

Q1 Lack of Novelty

We highlight the contributions of our paper as follows:

MSR-ViR enhances interpretability in Multimodal LLMs for VideoQA by integrating a modular network for interpretable reasoning. An alternate self-reflection training strategy is proposed wherein Multimodal LLM provides feedback to perform DPO training on Question Parser that generates reasoning paths. To the best of our knowledge, this is the first work that not only integrates a modular network into a Multimodal LLM but also jointly optimize them with self-reflection training for reasoning path refinement and QA accuracy improvement. Prior modular-based works(VideoAgent, AoTD) directly use the results of the modular system without optimizing or refining them, and some of them are built on Uni-modal LLM(MoReVQA, ProViQ). As for the limited performance improvement issue, we address as follows:

• Grounding-based methods are not capable of significantly improving the accuracy of Multimodal LLMs in answering questions, as they basically only change the input without changing the model's internal structure. More relevant visual information helps improve QA accuracy, but the improvement is limited. This aligns with existing grounding-based approaches, where improvements remain modest (e.g., SeViLa +1.2%, GCG +2.1%). The improvement of our method is comparable to that of the existing works.
• It can be observed that the performance improvement on EgoSchema is more significant compared to that on NExT-QA. We claim that MSR-ViR shows greater performance gains on longer video datasets, where grounding and reasoning are more essential. For further validation, we train Qwen2-VL and LLaVA-Video with baseline method and MSR-ViR on a subset of LLaVA-Video-178K and test them on NExT-QA and a longer VideoQA dataset VideoMME. The results demonstrate the superiority of MSR-ViR, especially on longer videos.

Q2 Self-Reflection Learning for Multiple-choice QAs

The loss here refers to SFT loss. For multi-choice QAs, this loss simplifies to the correctness of the selected option. For policy pairs that yield the same option, we do not use them for DPO training. About 50% of policy pairs could be used in the first epoch, adequate for DPO training of Question Parser.

Q3 Handling Non-Spatial-Temporal Questions

The interpretability in our work is considered from the perspective of spatial-temporal grounding. Non-spatial-temporal questions constitute a special scenario for our framework. In these cases, MSR-ViR asserts that answering the questions necessitates the entirety of video information. In other words, the grounding result would be the full video. Future work could expand MSR-ViR with non-spatial-temporal modules for further interpretability.

Theoretical Claims

We are sorry about the typo. Please refer to "Theoretical Claims" part in our rebuttal to reviewer NjDQ for more details.

Experimental Designs Or Analyses

Q4 Frame Sampling

The current frame sampling rates are optimized. Due to character limitation, we present several ablation on frame sampling for NExT-QA in this link: https://anonymous.4open.science/api/repo/frame-ablation-C4EB/file/frame-ablation.png?v=54b8a357. For MSR-ViR, the number represents (temporal + spatial + global).

Q5 Base Models

We conduct experiments for Qwen2-VL and LLaVA-Video on NExT-QA and VideoMME. Please refer to Q1.

Essential References Not Discussed

STAIR generates policies through small models trained with supervision on a single dataset, thus the types of questions it could solve are severely limited. ProViQ and ENTER utilize LLM to generate programs. However, they are based on Uni-modal LLM, and may fail to generate executable programs due to lack of training. VideoAgent and MotionEpic utilize Chain-of-Thought which involves multi-round conversation with LLM, while AoTD distills knowledge from CoT into Video LLM to improve instruction tuning. The idea of jointly optimizing Multimodal LLM and modular system via self-reflection training in our work represents a novel approach not explored in these works. We will add detailed discussion to the final version of our paper.

We sincerely thank the reviewer for taking time to review our paper and providing insightful feedback and suggestions. We address the questions as follows:

Supplementary Material

We will open-source our code after the review and provide detailed documentation to facilitate the reproduction and application of our framework.

Essential References Not Discussed

The CVPR paper addresses the challenge of open-ended VideoQA in long egocentric videos. The authors propose a novel approach that integrates query grounding and answer generation into a unified model to achieve grounded VideoQA. However, the proposed unified multimodal model is still black-box, lacking interpretability. Different from this paper, our work provides interpretability in VideoQA tasks for Multimodal LLMs. We will add detailed discussion to the final version of our paper.

We sincerely thank the reviewer for taking time to review our paper and providing insightful feedback and suggestions. We address the weaknesses and questions as follows:

Claims And Evidence

1.The Question Parser's prompt is meticulously crafted. It details each module's function and includes carefully-chosen basic examples, covering common and corner cases, to guide policy generation. As seen in Figures 4,5,6, the generated policy is a concise JSON string, not some heavy structure. Although the prompt is long due to multiple policy examples, this is essential for the Question Parser to produce reasonable policies.

2.Please refer to "Experimental Designs Or Analyses" part.

Theoretical Claims

We are sorry about the typo in Preposition 3.1. Preposition 3.1 should be:

Given parameters of Multimodal LLM $P_1$, $P_2$, $P_3$ and $P_4$, video with $N$ input frames and resolution $H \times W$, text input with length $l$, the complexity of Multimodal Answerer is $O\left(P_1NH^2W^2 + P_2N^2 + P_3l^2 + P_4Nl \right)$.

As we have only selected Qwen-VL as the base model to analyze the computational complexity, $P_1$, $P_2$, $P_3$ and $P_4$ represent constants relevant to parameters of Qwen-VL. Specifically, according to the proof of Preposition G.5 in Appendix G, given the parameters of the vision transformer $d_{VT}, L_{VT}, d_{ff_{VT}}$, the parameters of QwenLM are $d_Q, L_Q, d_{ff_Q}$, the number of queries in the cross attention layer $n_q$ and the kernel size and stride in the convolution layer $s$, we have: $P_1 = \frac{2L_{VT}d_{VT}}{s^4}$, $P_2 = 2L_Qd_Qn_q^2$, $P_3 = 2L_Qd_Q$, $P_4 = 4L_Qd_Qn_q$.

Experimental Designs Or Analyses

We test the inference speed of our method and Multimodal LLM-based baselines on an NVIDIA A100 GPU. Results are presented in the above table. We omit GCG from the table as its repository lacks inference code. LSTP, designed for high-efficiency inference with optical flow, is marked. For BLIP-2, LSTP, InstructBLIP, SeViLa and Qwen-VL, we sample 4 frames from the video. For LLaVA-Next, we sample 8 frames from the video. The settings of MSR-ViR keep consistent with those in our paper.

Our framework's additional parameters mainly stem from the Question Parser; MoST-Grounding contributes less than 0.1B. For comprehensive comparison, we test total parameters, inference speed, and accuracy using Question Parsers of different sizes (Qwen2-7B and Qwen2-1.5B). With Qwen2-7B, the inference speed of MSR-ViR is about twice that of the direct baseline, consistent with our complexity estimates. This computational overhead is reasonable, as reasoning-based methods typically require more time to answer questions due to the step-by-step reasoning process, like GPT-o1 over GPT-4o, and DeepSeek-R1 over DeepSeek-V3. With Qwen2-1.5B, although accuracy slightly drops, it still outperforms the direct baseline with fewer additional parameters and less computational overhead.

Although MSR-ViR introduces additional parameters and computational overhead, we address its advantages as follows:

• MSR-ViR provides interpretable reasoning path which is refined by self-reflection training, while previous methods fail to do so. As MoST-Grounding utilizes faster small modules, the computational overhead mainly comes from Question Parser that provides interpretability of our framework, and the computational overhead is reasonable.
• As for the issue of insignificant improvement in Table 1, please refer to Q1 in our rebuttal to reviewer u3Mw. We claim that the impact of grounding for Multimodal LLM on VideoQA datasets is limited, consistent with previous grounding-based method. Besides, we further conduct experiments on long-form VideoQA dataset VideoMME, where grounding and reasoning are more important, to demonstrate the superiority of our framework.
• Table 3 demonstrates that MSR-ViR achieves significantly higher grounding accuracy and grounded-QA accuracy than previous methods. The IoU is even higher than that of LLoVi and MoReVQA which utilize significantly larger LLMs GPT-4 and Palm-2. This demonstrates that MSR-ViR is capable of more accurately grounding and answering questions.

We sincerely thank the reviewer for taking time to review our paper and providing insightful feedback and suggestions. We address the weaknesses and questions as follows:

Claims And Evidence

Providing interpretable reasoning paths for black-box Multimodal LLMs in the scenario of VideoQA is one of the major claims in our paper. Although the reasoning path is generated by a text-only LLM, the reasoning process of MSR-ViR is multimodal, as presented in Figure 8, 9, 10. MoST-Grounding localizes the temporal segments and spatial regions related to the question from the video step by step according to the reasoning path. This reasoning process requires not only the textual information in the question but also the visual information in the video to perform step-by-step grounding, and thus it is multimodal.

Due to the lack of label annotations, we are unable to directly evaluate the reasoning path. However, this does not mean the reasoning path is not critical in our framework. Keeping small modules in MoST-Grounding fixed, the reasoning path determines the final spatial-temporal grounding result input to the Multimodal LLM. Therefore, it is crucial for whether the Multimodal LLM can correctly answer the questions and whether the framework can provide accurate grounding results. For this reason, the evaluation of metrics such as QA accuracy, Acc@GQA, IoU is an indirect evaluation of the reasoning path. For example in Table 4, through these metrics, we verify that self-reflection training can improve the quality of the reasoning paths generated by the Question Parser.

To further verify the importance of the reasoning path, we provide another ablation study by using MoST-Grounding to directly provide grounding results with UniVTG and YOLO-World based on the original question without the reasoning path. The results on NExT-GQA are shown in the following table. It can be seen that without the guidance of the reasoning path, both the accuracy in answering questions and the accuracy of the grounding results have significantly decreased. This further illustrates the necessity of the reasoning path.

Experimental Designs Or Analyses

From the experiment results, it can be seen that the MSR-ViR is capable of providing reasonable reasoning paths, and thereby improves the accuracy of Multimodal LLM in VideoQA. As can be seen in Table 4, self-reflection training plays a key role in improving QA accuracy, which is in line with expectations and is also one of the main contributions of this paper. From the additional ablation study in the above table, it can also be seen that reasoning paths and self-reflection training have significantly improved the performance of grounded-QA, alleviating error accumulation of temporal localizer and spatial localizer. Directly using a video LLM with grounding capability provides a solution to grounding-based VideoQA, but the black-box end-to-end video LLM contradicts the core issue to be addressed in our paper - the issue of interpretability, and thus is not within the scope of discussion in our paper. For the issue of insignificant performance improvement, please refer to Q1 in our rebuttal to reviewer u3Mw. We claim that the impact of grounding for Multimodal LLM on VideoQA datasets is limited, consistent with previous grounding-based method. Besides, we further conduct experiments on long-form VideoQA dataset VideoMME, where grounding and reasoning are more important, to demonstrate the superiority of our framework.

Relation To Broader Scientific Literature

We thank the reviewer for the suggestions of missing references, and discuss these works as follows:

[A] introduces Visual Table, a novel visual representation that provides detailed object descriptions and knowledge in structured text, significantly boosting performance in visual reasoning tasks, while [B] presents MM-Reasoner, which leverages vision APIs and LLMs to extract and utilize query-specific knowledge. Other works explore the use of outputs for reinforcement learning of the model, such as [C] that improves image captioning quality by normalizing rewards with test-time inference outputs, leading to significant performance gains. These works share some similarities with certain ideas in our work. However, they are all focused on image tasks, rather than VideoQA task which involves more complex scenarios and requires more grounding and reasoning. We will add detailed discussion to the final version of our paper.

Other Comments Or Suggestions

We will make revisions to the layout of Figure 2 for clarity and comprehensibility.