LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization

Q1: Comparison of fairness with released weights

We would like to clarify that our Stage 1 and Stage 2 training involved no additional manual annotations. All training signals are derived from the LLaVA-video-178k Caption dataset, which is publicly available and already used as part of the InternVL-2.5-8B SFT training data [1].

As suggested by reviewer dFSN, we report additional baseline results, including Video-LLaMA3 and InternVideo2.5, both of which have also been fine-tuned on more diversified video datasets that are either the same as or highly overlapping with LLaVA-video-178k. Therefore, our method does not benefit from any original dataset.

Our performance gains stem not from access to additional data, but rather from our method's capability to more efficiently organize and utilize existing data, through multi-stage optimization and explicit long-video reasoning.

Performance Summary

Q2: Isolating the contribution of DPO

As shown in Figure 5 of our main paper, we conducted experiments that fine-tuned InternVL-2.5-8B using only our Stage 1 caption data. Interestingly, we observed significant performance degradation, especially due to overfitting. This is because the Stage 1 Caption data overlaps with InternVL’s own SFT data, and repeated fine-tuning on this data leads to overfitting.

Ablation Results (InternVL family):

We applied a second-stage SFT using preferred responses generated from our pipeline, which alleviates overfitting and demonstrates the effectiveness of our multi-stage design. These results suggest that while data quality matters, our training strategy and pipeline design are key contributors to performance improvement.

Q3: Risk of self-reinforcement bias in Stage 2

We understand the concern regarding self-reinforcement bias. However, under the context of limited long-video QA data, our focus is on developing a scalable, low-cost strategy to expand long-context capabilities in VLMs without relying on expensive human annotations.

1. The preference order is valid in most cases. In Stage 2, we generate the preferred answer $y_0$ using the full video context, and construct a dispreferred answer $y_1$ using either an irrelevant scene or only a partial relevant one. Although the generated answer $y_0$ is not perfect, it tends to hallucinate less than $y_1$, which lacks the necessary context. Therefore, we argue that the preference order $y_0 \succ y_1$ holds true in the majority of cases.

2. Empirical evidence against reinforcement bias. To examine whether self-reinforcement causes overfitting or degradation, we repeated Stage 2 DPO training twice iteratively using InternVideo2.5. We observed consistent performance improvement across multiple benchmarks, indicating robust generalization rather than overfitting to earlier outputs.

This result suggests that LongVPO has generalizable benefits, even under repeated preference sampling and DPO training.

3. On the use of external judges or human evaluation. We agree that incorporating independent judges (e.g., Gemini 2.5 Pro or human annotators) could improve reliability. However, such strategies would significantly increase the cost and conflict with the paper’s core goal of efficient and scalable training for long video reasoning.

Our approach aligns with the philosophy adopted by recent image-domain methods, such as MPO [1], which automatically constructed a large-scale preference dataset of over 1M pairs—mostly without the use of external judges. Although this scale is significantly larger than ours, MPO has demonstrated the effectiveness and scalability of self-training strategies on image reasoning tasks. This supports the feasibility of similar approaches (i.e., self-training) in the long-video domain, especially under limited annotation resources.

Thus, while more rigorous preference curation could further improve results, we believe our current results are sufficient to demonstrate the promise of this self-training paradigm in the long-video setting. We plan to apply further preference filtering using LLMs and captions before open-sourcing our data, and leave more fine-grained supervision or human/VLM evaluation as future work.

References

[1] InternVL2.5 Technical Report: Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling

[2] Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization

Thank the authors for the response. My concerns have been sovled and I will consider to improve my rating score.

Q1. On the motivation of extending short-context VLMs

We would like to clarify that our goal is not simply to "extend" legacy models, but to address a more fundamental and persistent challenge: how to efficiently align the powerful, pre-existing semantic capabilities of foundation models for robust long-video reasoning.

Our central insight is not model-specific but applies broadly: "Transferring and aligning strong short-context capabilities into long-context reasoning can be more efficient than direct long-context tuning."

While it is true that many foundation VLMs (e.g., Qwen2.5-VL, Video-LLaMA3, Seed-VL) now claim to support long-context inputs, several observations challenge the effectiveness of such support:

• The training data landscape exhibits a pronounced long-tail distribution in context length. The vast majority of available multimodal supervision—such as image task pairs or short video clips—consists of short-context data. In stark contrast, annotated long videos suitable for instruction tuning or reasoning tasks are extremely scarce, as they demand substantial computational resources and costly labeling efforts. As a result, existing models risk being insufficiently trained for genuine long-context understanding.

• On long-video benchmarks such as LongVideoBench, Video-MME, or LVBench, many long-context VLMs experience performance degradation as input length increases, highlighting the gap between architectural support and actual long-range reasoning ability.

LongVPO is a highly efficient and general alignment method for both short- and long-context models.

• First, InternVL2.5-LongVPO itself already achieves performance that surpasses or matches most dedicated long-video models.
• More crucially, our approach remains highly effective even when applied to specialized video models such as Video-LLaMA3 and InternVideo 2.5, which have already been fine-tuned on valuable long-video data.
  • This is particularly noteworthy because the LLaVA-Video-178k dataset, from which we generate our preference data, was already included in the initial supervised fine-tuning (SFT) of these models.
  • The fact that LongVPO still delivers significant additional gains demonstrates that its strength comes from the alignment method itself, not merely from exposure to new data, highlighting its power as a truly efficient and general alignment technique.

Q2. On limited gains from increasing frame count.

There is no significant drop in performance with more frames in our baseline models. The previously reported LVBench score of 72.0 for InternVL2.5 (64 frames) was a known error; we have corrected it to 43.2, in line with the results reported in Video-LLaMA3's technical report.

Increasing the number of input frames from ​​64 → 512​​ yields consistent improvements (+0.6% avg). When combined with our ​​Stage 1​​ and ​​Stage 2​​ training, performance improves substantially, absolute gain over the baseline: ​​+5.2%​​ (64frames), +4.6% (512frames).

*The average of two Video-MME scores is used to calculate the global average.

Q3. On DPO deviation from the reference model

We adopt a widely-used solution by Joint optimization with SFT loss (i.e., $L_{\text{total}} = L_{\text{LongVPO}} + L_{\text{SFT}}$ ) as common practice in many recent VLMs (e.g., [1]), ensuring that the model remains close to the reference distribution while still learning preference-based alignment to preserve generation quality.

Furthermore, we argue that some deviation from the reference model is not only acceptable, but desirable in our setting as an open research question: Our goal is to equip the model with new capabilities (long-context reasoning) beyond the reference. Recent RLHF frameworks (e.g., DAPO [2], GLM-4.1V [3]) have removed KL-regularization between the policy model and the reference model to allow more flexible policy exploration.

References

[1] Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization

[2] DAPO: An Open-Source LLM Reinforcement Learning System at Scale

[3] GLM-4.1V-Thinking

Thank you for your constructive feedback. We understand your concerns regarding the motivation behind long-context VLMs. While the train-test gap may contribute to performance degradation, the intrinsic limitations of sparse sampling methods present a more fundamental challenge, especially for long-video understanding. We would like to discuss your points below:

1. On "Longer is Not Always Better"

Current benchmarks reveal critical limitations of fixed-length sparse sampling (e.g., 64 frames):

• Human-level understanding requires dense context: According to the current benchmark for advanced video reasoning and understanding ([1], Fig 9), human performance shows a significant improvement with increasing the number of frames from 1 to 256, reaching 90% accuracy, while GPT-4o/LLaVA-Video plateau at around 8 frames with performance below 50%, indicating a ~40% performance gap.

• Significant performance drop occurs as video duration increases under identical frame budgets:

  • LLaVA-Video-72B on VideoMME: Short[< 2min] (81.4%) vs. Long[30min ~ 60min] (61.5%); on LongVideoBench [8-15s] (72.4%) vs. [900-3600s] (59.3%).
  • Contrary to the performance claims in Our Main Figure 4, Sparse sampling ​​fundamentally cannot achieve reliable needle-in-a-haystack retrieval​​ [2].
  • This huge gap suggests sparse sampling struggles to robustly represent temporal sparse information in long videos.

• Why Sparse Sampling Fails for Precise Queries: LVBench (adopted by Gemini in their technical report [3] to demonstrate long-video capabilities)

  • Question: "How does the goalkeeper prevent Liverpool's shot at 81:38 in this 1:40:20 match?"

  • Sparse sampling struggles to reliably locate such precise moments without luck. In contrast, long-context models are specifically designed for systematically processing relevant segments, ensuring more accurate results.

2. So, Do We Truly Need Native Long-Context Support?

While key-frame extraction (e.g., [4]) can mitigate sparse sampling issues, it introduces ​​pipeline complexity​​ and ​​information loss​​ (disrupting temporal continuity). In contrast, ​​native long-context modeling​​ enables:

• Complex Interaction: Real-world applications (e.g., multi-turn agent operation interaction, streaming chatting) require retaining and reasoning over linearly growing video-history contexts.
• Reasoning enhancement: Recent work [5] demonstrates that long-context capabilities are essential for advanced reasoning, and we believe this also applies to the VLM domain.

3. Training Cost Efficiency

Our approach achieves competitive results with modest resource investment compared to large scale long context SFT:

• The total training time is ～2 hours on 4×8 H100 GPUs for Internvideo2.5-7B and <10 hours on 4×8 H100 GPUs for Internvl2.5-7B.
• Stage-1 training cost reduced to 70% through LongVPO optimization, as concerned by Reviewer 4ZfK.

Native long-context support not only aims to bridges the performance gap but also lays the foundation for complex video tasks. We believe these advances, coupled with our contributions to pipeline construction and efficient DPO training, justify the paper's significance.

References
[1] Towards Video Thinking Test: A Holistic Benchmark for Advanced Video Reasoning and Understanding

[2] Long Context Transfer from Language to Vision

[3] Advancing the frontier of video understanding with Gemini 2.5

[4] VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos

[5] Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning

Q1: Stage 1 Approximation Error Analysis

To directly analyze the “approximation gap,” we conducted a head-to-head comparison between Stage 1 LongVPO and a standard DPO baseline (reference model on the full synthetic video).

The table below summarizes the results, including the original 128-frame/32k-token setting from Figure 5 of the main paper, and an extended 256-frame/64k-token experiment that further demonstrates scalability.

Performance Gains of LongVPO vs. Direct DPO (Stage 1)

Stage 1 approximation consistently improves performance, especially at longer contexts.

Q3: Computational Complexity

From the table in Q1:

• Stage 1: Our LongVPO Stage 1 achieves better performance than standard DPO while requiring ~30% less training time, demonstrating its efficiency advantage.
• Stage 2: The training cost is identical to standard DPO, ensuring that the overall improvement does not come at additional computational expense. Both cost 4.4h.

Q2: Incomplete Comparison (Table 1)

• For InternVL 64Frames, we corrected the missing/broken entries in our initial submission and fixed a known LVBench score error (72.0 → 43.2) for InternVL2.5 (64 frames), consistent with the reproduced results in Video-LLaMA3’s report. The updated results confirm that the InternVL-2.5 (512 frames) model is the strongest baseline.

*AVG is computed using the mean of the two Video-MME scores.

• Regarding the missing data for models other than InternVL, all figures in the table are cited directly from their respective official reports. These entries are missing because the original authors did not publish results for these specific benchmarks.

Q4: Why Localizing Robustness Matters in Long-Form Video Understanding？

Robustly localizing key information and gathering information to answer correctly is not just a proxy task; it is a fundamental prerequisite for any meaningful long-video understanding as a Common solution paradigm [1][2]. The reasoning is twofold:

• Sparsity of Information: User queries about long videos often depend on a few critical moments. The model must first reliably find this "needle" before it can reason about it.
• Prerequisite for Reasoning: Complex reasoning requires integrating evidence, often from multiple, non-contiguous scenes. If the model cannot robustly locate each piece of evidence, its reasoning process fails at the first step.

Our positional bias experiment in Figure 1 directly simulates this challenge. This is analogous to the widely used "needle-in-a-haystack" tests[3] in long-text evaluation, which are designed specifically to pressure-test a model's ability to retrieve sparse, critical information from a long, noisy context. Without this robustness, a model cannot truly comprehend long-form content.

Q5: Artifacts of Tuning with Synthetic Data

We did observe minor stylistic artifacts in a small subset of open-ended generation tasks (e.g., summarization). These typically involve the model adopting a structural narration style, using phrases like

"The video begins with..." or "The scene transitions to...".

These artifacts are a byproduct of the captioning style used to generate training data and do not harm factual accuracy on question-answering tasks (Please see the qualitative comparison in the main paper and appendix for more examples). We view this as a data diversity challenge that can be effectively addressed in future work with simple techniques like style-aware prompting, rather than a fundamental flaw in the model's reasoning capabilities.

References

[1] Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding

[2] VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos

[3] Needle In A Haystack – Pressure Testing LLMs

Thank you for your valuable comments and insights, which allow us to better highlight the strengths of LongVPO. We will carefully take your suggestions to enhance the clarity and persuasiveness of the revised version.

Q1: Concern Regarding the Role of Qwen-2.5 32B in LongVPO Stage2

We would like to clarify that LongVPO only uses Qwen-2.5 32B in a limited role: 1) It generates questions from video captions, and 2) provides binary relevance hints for each video segment. Most importantly, Qwen-2.5 32B is not used to generate the model’s preferred responses in Stage 2.

To further address the reviewer’s concern, we conducted additional ablation studies:

1. LongVPO without Qwen-2.5 32B exhibits only slight performance degradation. We replaced Qwen-2.5 32B with InternLM2.5-7B-chat (the backbone LLM of our baseline InternVL2.5-8B). This version, LongVPO-InternLM-Stage2, shows only marginal degradation:

2. Directly using Qwen2.5-32B Generated Preferred Responses is much worse. As shown in Table 2 of the main paper: Directly using Qwen-2.5 32B responses as preferred answers for DPO performs worse than using our model’s self-generated responses: with LVBench: -1.7% and LongVideoBench: -1.9% This highlights that the external model does not dominate the learning signal and its deep involvement is not beneficial to performance.

3.The reliance on Qwen-2.5 32B is primarily for its instruction-following and formatting capabilities, e.g., outputting desired JSON-formatted question-answer pairs. Fundamentally, our method focuses on enabling the VLM to self-correct its own response preferences based on valid guidance, rather than directly depending on a powerful external model—this is also the key distinction from SFT or other methods that rely heavily on strong external supervision.

Q2: On the Necessity of Stage 1 Approximation vs. Direct DPO

• Figure 5 Recap: Our main paper’s Figure 5 shows that standard DPO (reference model on full synthetic video) underperforms LongVPO on Video-MME and LVBench, due to weaker long-context alignment.

• Extended Ablation: We further increased the max video length from the original 128 frames / 32k visual tokens training input to 256 frames / 64k tokens. As training context length grows, LongVPO’s Stage 1 approximation consistently yields extra gains, confirming that our design is not only more efficient but also more effective than a naïve full-video DPO baseline.

Performance Gains of LongVPO vs. Direct DPO in Stage 1 Data

We hope these additional results sufficiently address both concerns, as there is only minimal impact of external LLM reliance and the effectiveness of our Stage 1 approximation beyond efficiency.

Dear Reviewer FVvM,

Thank you for your insightful feedback. We are writing to follow up on our recent rebuttal of our manuscript. We sincerely appreciate the time and effort you dedicated to your thoughtful review.

With the discussion period ending in 3 days, we are keen to ensure we have resolved your concerns. Please let us know if our revisions are satisfactory or if anything else requires clarification.