Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration

W1: Motivation for Using Referring Video Segmentation (RVS)

---

W1.1 Why Referring Video Segmentation Task?

The Referring Video Segmentation (RVS) task can be naturally decomposed into three key components:

1. Understanding complex user instructions and identifying the referred object across the video, which requires the model to perform global temporal reasoning over the entire video.
2. Once the object is identified, the model needs to accurately localize and segment it in high-resolution frames, requiring strong fine-grained spatial understanding.
3. Propagating segmentation masks from sparse keyframes to the entire video via object tracking, which is more about low-level visual perception and can be handled by off-the-shelf tools like SAM2—thus not the focus of this paper.

The first two components align well with the challenge we highlight in the Introduction: the trade-off between long-horizon temporal reasoning and fine-grained spatial understanding. In contrast, many standard long-video understanding tasks primarily require global reasoning and often do not demand such precise spatial understanding. In those cases, System 1 alone may be sufficient to handle the task. Therefore, RVS provides a more rigorous testbed for our proposed two-system architecture.

W1.3 How Are Masks Generated for Non-Key Frames?

For frames not directly selected by System 1, we adopt SAM2 as a tracking and propagation module. SAM2 is a strong visual foundation model capable of propagating object masks across time based on the sparse predictions on keyframes.

This approach follows the standard paradigm in RVS literature: high-cost grounding and segmentation are performed on sparse frames, and light-weight temporal propagation is used to fill in the rest. Our contribution lies in learning to select the most informative keyframes, which significantly boosts both efficiency and segmentation consistency, especially under challenging conditions like motion blur, occlusion, or scene transitions.

W1.2 Why Is Key Frame Selection Still Necessary?

Given the above two premises, the necessity of selecting key frames can be justified from two main perspectives:

1. Grounding models lack long-range temporal reasoning: Modern vision-language (grounding-capable) models models (e.g., Sa2VA,Qwen2.5VL,Qwen2.5Omni) excel at precise localization in static images but struggle when provided with long, complex video sequences and ambiguous or complex instructions. If we directly input the full video and a temporally complex prompt, these models struggle to identify the correct object. In contrast, by leveraging the global reasoning ability of System 1, we can first select informative key frames and reformulate the instruction into simpler, localized prompts. As a result, the grounding model only needs to perform fine-grained segmentation on single high-resolution frames guided by short and unambiguous instructions. These two steps—key frame selection and instruction simplification—are strongly coupled, and together enable the model to more effectively fulfill the first two components described in W1.1.

2. More intelligent than rule-based sparsity: Prior works like Sa2VA also rely on sparse frame segmentation followed by mask propagation. However, their frame sampling is rule-based (e.g., uniformly sampled or first five frames), lacking flexibility and potentially leading to informational loss. In contrast, our System 1 can adaptively select frames with large semantic transitions, such as those involving object occlusion, reappearance, or camera scene cuts. These challenging segments are often the hard cases for standard tracking models. By letting System 1 select such frames for precise grounding by System 2, we can significantly reduce the tracking burden and overall segmentation error.

---

W2: Computational Complexity

We would like to clarify that the trade-off between long-horizon temporal reasoning and fine-grained spatial understanding primarily affects the training phase, rather than inference.

At inference time, increasing frame resolution or sampling rate is indeed a straightforward way to enhance performance. However, if the model has not been trained with long sequences and high-resolution inputs, it will lack the necessary capacity for temporal and spatial reasoning—thus limiting the benefit of high-resolution inference. A concrete example can be found in Appendix Table 3, where merely increasing input quality does not close the performance gap without structural support.

In our framework, we decouple training as follows:

• System 1 operates on inputs with resolution 100×28×28, with a maximum of 24 frames, and is trained end-to-end.
• System 2 does not require training or backpropagation, and handles only 5 high-resolution frames (1200×28×28). Furthermore, because System 2 processes frames independently during inference, we leverage the vLLM engine to parallelize its execution efficiently.

If we were to train the whole pipeline end-to-end using high-resolution frames (e.g., 1200×28×28), the compute budget would limit the number of frames to around 5, making it impossible to learn long-range video reasoning. This constraint directly limits global temporal understanding.

As a concrete example, Sa2VA, an end-to-end trained model, only supports 5 frames for both training and inference—confirming that a monolithic approach struggles with long-horizon temporal modeling.

Thus, our proposed two-system architecture offers a computationally efficient and scalable way to address this trade-off, enabling both global video reasoning and fine-grained visual grounding.

W3: Reward Design

Thank you for raising this point. We have conducted additional experiments to more comprehensively evaluate the effectiveness of our hierarchical reward design.

As discussed in the paper, the grounding-based reward $R_G$ is strongly coupled with System 2, meaning it depends heavily on the specific outputs of a particular System 2 model. This tight coupling can lead to reduced generalization when switching to a different System 2. Although adding $R_A$ to （ $R_G$ and $R_K$ ） may slightly reduce performance under the same System 2, it significantly improves generalization to other System 2 implementations.

To validate this, we perform ablation studies where we replace System 2 with Sa2VA at inference time and evaluate how different reward combinations affect final performance:

The results show that:

• When using Sa2VA as System 2, adding $R_A$ leads to notable performance improvements, especially on the Reasoning sub-task.
• This is because $R_A$ is a weakly coupled reward, focusing on the alignment between visual features and language prompts at the single-frame level. It provides better transferability across different System 2 implementations.
• In contrast, $R_G$ tends to overfit to the specific structure and behavior of the System 2 used during training, limiting generalization.

In summary, even if $R_A$ shows limited benefit in fixed setups, it plays an important role in enabling general-purpose and modular policy learning, which is essential for a scalable architecture.

W1: Correction of Δ Values

Thank you for pointing this out. We acknowledge the error in the originally reported delta values and have corrected them accordingly. The revised table is shown below:

We will correct this in the next version of the paper. Thanks again for your careful reading.

W2: Data Scale

We believe the relationship between data scale and RL-driven improvements can be analyzed from two perspectives:

Impact of Data Volume on a Single Task

We conducted an experiment on RefAVS, comparing models trained with 1600 and 10400 samples. The results are shown below (see Appendix Table 3 for more details):

We observe that increasing data size improves task-specific metrics, particularly in the seen subset. However, it may also introduce slightly higher hallucination rates, as reflected in AVHBench accuracy.

Multi-Datasets Joint Training

We further investigate the effect of multi-task data mixing, especially combining ReVOS and RefAVS datasets.

As shown in Appendix Table 4, training on both datasets leads to consistent performance gains even on ReVOS—which was not directly targeted during joint training:

• ReVOS overall score: 44.7 → 47.6 (+2.9)

This demonstrates that RL-based joint training across tasks and distributions can enhance general video understanding capabilities, without requiring dataset-specific tuning.

Q1: Out-of-Domain Generalization

Thank you for raising this important point. To evaluate out-of-domain generalization, we tested both Qwen2.5 Omni-7B and Omni-R1 on the DriveBench[1] dataset, which targets the autonomous driving domain—a distribution notably different from our training data.

The behavior prediction task in DriveBench requires the model to predict future steering angles and vehicle speeds based on multi-view images (from six camera directions). It is worth noting that DriveLM, the strongest baseline, is specifically fine-tuned on large-scale driving data.

We evaluate the behavior accuracy metric (as it does not depend on GPT APIs for scoring):

While DriveLM benefits from domain-specific fine-tuning, Omni-R1 is trained in a general-purpose fashion without any driving-specific data. Despite this, it still outperforms its baselines(Qwen2.5 Omni-7B), demonstrating clear improvements in out-of-domain generalization brought by our reinforcement learning framework.

Q2: Key Frame Quality Reward

Thank you for the insightful question. We would like to clarify that the underlying principles of the Key Frame Quality Reward are generally applicable across video tasks, although the specific hyperparameter settings may vary depending on the dataset and task characteristics.

Specifically:

1. $R_{\text{diversity}}$ encourages the model to select non-overlapping segments as keyframes, promoting diversity. This reward is task-agnostic and generally useful across different video scenarios.

2. $R_{\text{num}}$ softly constrains the number of selected keyframes. We acknowledge that different videos may require different numbers of keyframes, so we do not enforce a fixed value. Instead, we define a reasonable range bounded by $K_{\min}$ and $K_{\max}$, and penalize the model only if it selects too few or too many frames. The optimal number of keyframes is learned implicitly through the downstream rewards $R_A$ and $R_G$. When training on multi-domain data, we simply need to ensure that the range defined by $K_{\min}$ and $K_{\max}$ is sufficiently broad.

3. $R_{\text{saliency}}$ is only applied when ground-truth object-level annotations (e.g., masks or boxes) are available, such as in tasks like RefAVS or ReVOS. For high-level video understanding tasks that lack such annotations, we set its weight $\lambda_3$ to zero.

In summary, only $R_{\text{num}}$ requires mild adjustment when applied to joint training over diverse datasets. The other two reward components remain broadly applicable and robust across various settings.

Reference

[1] Xie, S., Kong, L., Dong, Y., Sima, C., Zhang, W., Chen, Q. A., ... & Pan, L. (2025). Are vlms ready for autonomous driving? an empirical study from the reliability, data, and metric perspectives. arXiv preprint arXiv:2501.04003.

Dear Reviewer zCqj,

As the discussion phase is approaching its conclusion, we would like to kindly check if you have any further questions, concerns, or suggestions regarding our rebuttal. Your additional input would be invaluable in helping us further clarify and improve our work.

Thank you again for your time and thoughtful engagement throughout the review process.

Best regards

W1: Architecture-Level Ablations

We appreciate the reviewer’s suggestion. To evaluate the impact of architectural choices within our pipeline, we conduct ablation studies by replacing both the grounding model and the segmentation model, examining how these changes affect training and inference.

Grounding Model Replacement We replaced the original Qwen2.5-Omni-7B with VisionReasoner[1], a unified model for visual perception tasks enhanced via GRPO across grounding, counting, and segmentation. To ensure fair comparison, we aligned all training hyperparameters and tested two settings:

• Replace only during training
• Replace during both training and inference

Results on ReVOS are shown below:

As seen, simply replacing the grounding model during training leads to noticeable gains. A full switch to VisionReasoner during both training and inference yields even greater improvements across most sub-tasks.

Segmentation Model Replacement We further replaced the segmentation model used during inference. Specifically, we switched from SAM2 to TAM [2], a more efficient segmentation tracker, while keeping VisionReasoner as the grounding model (training only).

Although the overall score drops slightly (-1.1%), TAM shows better performance on multi-object tracking, indicating its advantage in handling complex scenarios.

---

Conclusion: These results validate the robustness and generalizability of our architecture. Stronger grounding or segmentation models can further enhance performance, and our pipeline remains effective under architectural variations.

Q1: Integrating Sa2VA as System 2

Yes, your understanding is correct. The purpose of this experiment is to demonstrate that System 1 and System 2 are modular and can be decoupled—replacing System 2 with a stronger module like Sa2VA leads to better performance.

Although Sa2VA was not involved in our reinforcement learning training, it naturally fits the role of System 2. In its original setting, Sa2VA takes as input either the first five frames of a video or five frames uniformly sampled from the whole video, along with the original global instruction.

In our setup, we leverage System 1 to select five key frames and generate localized, simplified instructions for each segment. These keyframes and instructions are then fed into Sa2VA for fine-grained segmentation.

This decoupled design allows System 1 to fully exploit its strength in global video reasoning, while Sa2VA focuses on precise, local-level understanding. As shown in Table 2, this integration yields improved performance without any modification to Sa2VA’s parameters or architecture, further validating the effectiveness of our modular two-system design.

Reference

[1] Liu, Y., Qu, T., Zhong, Z., Peng, B., Liu, S., Yu, B., & Jia, J. (2025). VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning. arXiv preprint arXiv:2505.12081.

[2] Xiong, Y., Zhou, C., Xiang, X., Wu, L., Zhu, C., Liu, Z., ... & Chandra, V. (2024). Efficient track anything. arXiv preprint arXiv:2411.18933.

W1: Omnimodal Approach

One of the central challenges in building an OmniLLM lies in handling long-term temporal reasoning in combination with high-resolution fine-grained understanding. This challenge is especially prominent when integrating modalities like video and audio, which have vastly different token densities: for example, 1 second of audio may require only ~25 tokens, while a single 448×448 video frame typically takes 256 visual tokens. Thus, the core conflict—and bottleneck—resides in efficiently performing global reasoning over long-range video sequences while preserving detailed understanding at key moments.

Our dual-system architecture is specifically designed to address this challenge. As mentioned in L149, our System 1 can process both video and audio inputs jointly. While our method section focuses primarily on video understanding, we also validate the generality of the framework on RefAVS, which requires both audio and visual cues for temporal localization and grounding. In this task, System 1 learns to reason over the audio and visual sequence and generate global-to-local instructions.

Importantly, our results on RefAVS show that joint audio-video reinforcement learning improves performance significantly, demonstrating that our system does not rely solely on vision. Additionally, we show that the same omnimodal reinforcement learning paradigm improves performance on Omnibench—an image-audio task—further verifying the generality and scalability of our proposed Omni-R1 framework.

Therefore, while the most substantial improvements are observed on video tasks (which are inherently more complex), our method is truly omnimodal, and effective across vision, audio, and multi-modal reasoning scenarios.

W2: Training Details

Thank you for the suggestion. We will include reward curves in the next version of the paper. Due to image restrictions during the rebuttal phase, we provide a brief summary here:

During training, the reward component $R_K$ converges quickly and stably, indicating that the model rapidly adapts to the temporal distribution of selected frames. In contrast, $R_A$ exhibits fluctuations around 0.3, which we attribute to inconsistencies in local frame-level grounding. As for $R_G$, the reward shows an oscillating trend throughout training. Notably, high-quality rollout trajectories (where $R_G > 0.5$) ultimately stabilize around 0.8, suggesting effective convergence under high-performing behavior.

Regarding the number of rollouts, we clarify in L248 that the group size is 8, which corresponds to 8 rollouts per update in the GRPO training process.

W3: Case Studies

Thank you for your suggestion. We will include additional case studies in the next version of the paper to illustrate the outputs of both System-1 and System-2, as well as the intermediate results from the rollout process.

While we are currently unable to add new figures during the rebuttal phase, we kindly refer you to Appendix Figures 3 and 4, which already show examples of the reasoning behaviors of both systems. These examples provide an initial demonstration of how System-1 and System-2 work together to perform multimodal reasoning.

W4: Evaluations on Image and Video Understanding Tasks

Thank you for the suggestion. In the main paper (Table 3), we evaluate our model on video understanding benchmarks such as VideoMME and MVBench, as well as the OmniBench benchmark for image understanding.

To further assess image-level understanding capabilities, we additionally evaluate on MME[1] and DriveBench[2]. These results support the generality of our framework across both video and image modalities.

MME Results:

DriveBench Results (Behavior Accuracy):

We would like to note that DriveLM is specifically trained on driving-domain data, while Omni-R1 is trained in a general-purpose manner without any driving-specific fine-tuning. Despite this, Omni-R1 still achieves competitive performance across domains. Importantly, compared to the base Qwen2.5 Omni-7B, Omni-R1 consistently improves performance on both tasks (e.g., MME and DriveBench), demonstrating the effectiveness of our reinforcement learning–based optimization strategy.

Q1: $R_{diversity}$ and $R_{num}$

Thank you for the thoughtful question. We would like to further clarify the design and intent behind the two keyframe quality rewards:

• $R_{diversity}$ is designed to encourage the selection of non-overlapping and semantically diverse video segments as keyframes. It penalizes redundant or closely clustered selections. This formulation is generalizable across different types and durations of videos, as it operates purely on the segment selection patterns without requiring prior content-specific assumptions.

• $R_{num}$ We acknowledges that different videos may require different numbers of keyframes. Therefore, we do not enforce a fixed number of selected keyframes. Instead, we define a soft constraint via two thresholds: $K_{\text{min}}$ and $K_{\text{max}}$. A penalty is applied only when the number of selected keyframes falls outside this range. The model then learns to select the optimal number of keyframes based on downstream rewards $R_A$ and $R_G$.

This design enables the system to adaptively balance diversity and sufficiency of information while remaining general-purpose and content-agnostic.

Reference

[1] MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

[2] Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives

Dear Reviewer vim3,

As the discussion phase is approaching its conclusion, we would like to kindly check if you have any further questions, concerns, or suggestions regarding our rebuttal. Your additional feedback would be invaluable in helping us further clarify and improve our work.

Thank you again for your time and thoughtful engagement throughout the review process.

Best regards