MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO

We begin by sincerely appreciating the reviewer for the respectful and patient feedback. We completely understand the concerns raised and will address them individually for further clarification.

W1: Question about the CLIP reward.

Ans: We agree with the reviewer that CLIP-based similarity metrics have known limitations in capturing complex spatial or logical relationships. However, our main contributions are the proposal of a unified reasoning-generation model and a training RL framework that enhances its reasoning generation ability. While we initially chose CLIP rewards to discourage benchmark hacking, we also explored a more capable reward model that better aligns with fine-grained reasoning. This alternative yielded a 0.8-point performance improvement under the same settings, as shown in Table 13 (in the response to reviewer Joq5’s W3).

W2: Question about the training stage 2.

Ans: On the one hand, the CoT data in stage 2 is composed of short, coarse image captions as input prompts, with the corresponding “reasoning” content being detailed and descriptive long-form captions, which are generated by Qwen2.5VL-72B. On the other hand, we utilize this data to train only the generation decoder, enabling it to adapt specifically to Chain-of-Thought contextual inputs. The VLM component remains frozen throughout this phase, thereby preserving its general reasoning capabilities and ensuring that the model is not constrained by the limitations of the additional model. If this stage is omitted, the model's performance on the WISE benchmark will drop from 60% to 49% as shown in Table 5 in the manuscript.

Q1: The claim that the model can ”accurately play chess” is a particularly strong statement.

Ans: We regret any potential misunderstanding. Our goal was to underscore the model’s ability to perform fine-grained editing tasks, such as manipulating chess pieces according to specific instructions. We create a chessboard test dataset to assess this functionality and also evaluate the model on public editing benchmarks (GEdit and ImgEdit) as shown in Table 15 below. This fine-grained control is essential for enabling future research into adaptive chess playing. We will include these experimental results and the corresponding benchmark descriptions in the revised version.

Table 15: Performance of editing task.

Q2: Question about training stage 3.

Ans: We have provided extensive ablation studies in Tables 7–9 of the manuscript, evaluating different aspects of the RGPO reinforcement learning strategy, including KL regularization schemes, group numbers, and reward strategies. Specifically, Table 7 shows that standard single-KL term ($\beta^{I}$=0) reduces performance by 1.6 points on the WISE benchmark, underscoring the effectiveness of our dual KL approach. Furthermore, Figure 4 illustrates stable reward behavior throughout training, suggesting that KL regularization effectively mitigates reward collapse. We acknowledge the overthinking phenomenon observed during our RL process and note that it aligns with findings in prior works [1,2]. We will incorporate a discussion of this effect and relevant citations in the revision.

[1] Mitigating Overthinking in Large Reasoning Models via Manifold Steering

[2] Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models

Please let us know if you have any unsolved or other concerns.

We sincerely appreciate the reviewer for the precious time and valuable comments and will address the concerns raised.

W1: The difference from SimplerAR​.

Ans: Our contributions lie in two key areas: (1) constructing a unified VLM capable of both understanding and generation, and (2) leveraging reinforcement learning to unleash reasoning generation. To the best of our knowledge, SimpleAR and our work are conducted concurrently. But, unlike SimpleAR, which is an image generation expert method and employs a single KL constraint, our method supports both understanding and generation, as well as integrates a dual-KL formulation. We have included comprehensive ablation studies as shown in Tables 7-9 of the manuscript to validate its effectiveness within a unified architecture.

W2: Lack of user studies.

Ans: We conduct a user study as shown in Table 12 below, with 50 participants following the previous method. Participants are required to select the most preferred results from a set of methods, including only-support generation, FLUX.dev, and the unified understanding and generation counterparts, Janus-Pro and Bagel.

Table 12: User Study.

W3: Questions about consistency reward.

Ans: Potential misunderstandings may arise here. In the third stage, our consistency reward is calculated on the generated RGB image rather than the low-level latent representation, as detailed in Section 3.4 of the manuscript. Additionally, we conduct ablation studies to evaluate the effects of different reward models under consistent settings as shown in Tables 7-9 of the manuscript. Switching to a VLM-based reward model directly offers no substantial advantage compared to the CLIP model. We will further investigate the role of the reward model in the reinforcement learning process in our future work.

Table 13: Ablation of different reward models.

W4: The Number of available examples is limited.

Ans: We have presented some text-controlled image generation results in Figure 7 in the appendix part of the manuscript, but due to rebuttal policy limitations, we cannot provide more at this rebuttal period. We will release our model publicly and update the revised version with more diversity visualizations.

Q1: The output images have simpler backgrounds.

Ans: If the reviewer is referring to the example in Figure 1, the observed content stems from the model-generated CoT, which includes certain stylistic descriptions, which are illustrated in Figures 8–10 in the appendix. Our model exhibits strong prompt-following capabilities, which likely accounts for this phenomenon. By tuning the temperature and sampling configurations of the VLM, one can control the variability of the generated outputs. What's more, additional results of text-controlled generation are presented in Figure 7.

Q2: Insufficient attention is paid to the real limitations.

Ans: The primary limitation at present lies in the reward function. The CLIP-score-based consistency reward does not adequately capture low-level image details or numerical information. We plan to further explore this direction in future work.

We sincerely appreciate the reviewer for the time and comments and will address the concerns raised.

W1: Limited multimodal CoT.

Ans: Our work aims to move beyond the limitations of conventional fine-tuning and prompt rewriting strategies, which often fail to fully harness the reasoning potential of large models due to incorrect text guidance or inefficient prompt following execution. We introduce a reinforcement learning-based approach that unleashes the reasoning generation capabilities under our developed unified VLM. While current CoT reasoning remains text-based, we have shown it can substantially improve performance, laying a foundation for future multimodal CoT extensions research.

Table 10: Effectiveness of the CoT of our model.

Q1: Closed models' performance.

Ans: We perform iterative training using open-source data and evaluate GPT-4o on the WISE benchmark. As shown in Table 11, our model significantly narrows the performance gap with closed-source counterparts compared to open models.

Table 11: Performance Comparison.

Dear Reviewer s7fx,

Thanks for your response. We are glad that our rebuttal is able to address your concerns. We sincerely appreciate your review time and the valuable comments. We will attach great importance to revising the paper according to your suggestions.

Best regards,

Paper 13149 Authors

We sincerely appreciate the reviewer for the constructive comments and will address the concerns raised.

W1&Q2: Abstract visual reasoning benchmark.

Ans: We appreciate the reviewer’s valuable suggestion. While our work mainly centers on reinforcement learning strategies for image reasoning generation, our dual KL approach ensures that model performance does not deviate significantly from the original policy. Accordingly, as shown in Table 14, our RG-VLM performs similarly to Qwen2.5-VL on the visual reasoning benchmark mentioned by the reviewer.

Table 14: Training GPU Hours comparison for unified models.

W2&Q1: Discussion on RG-VLM limitations is extremely brief.

Ans: An additional limitation at present lies in the reward function. The CLIP-score-based consistency reward does not adequately capture low-level image details or numerical information. We plan to further explore this in future work.

Q3: Conceptual differences between RG-VLM's and DeepSeek's Chain-of-Thought reasoning.

Ans: RG-VLM is capable of both multimodal input and output, whereas DeepSeek is limited to textual processing. Moreover, the Chain-of-Thought (CoT) in RG-VLM serves as a reasoning mechanism to enhance visual generation, while DeepSeek’s CoT is designed solely for textual reasoning tasks.

Dear Reviewer BGbw,

Thanks for your response. We are glad that our rebuttal is able to address your concerns. We sincerely appreciate your time and effort in reviewing our paper. Your valuable comments are crucial in improving the quality of our work.

Additionally, we apologize for the typo in the caption of Table 14. This table simply presents the performance results for your listed tasks. The revised version is "Table 14: Performance of visual reasoning tasks" accordingly. We followed the methodologies outlined in [1][2][3] to evaluate these three benchmarks, and our results indicate that our model performs comparably to the Qwen2.5-VL (the expert model for visual understanding), benefiting from our dual-KL strategy in the RL stage.

Best regards,

Paper 13149 Authors

[1] https://github.com/apple/ml-rpm-bench

[2] Wüst, Antonia, et al. "Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad?." arXiv preprint arXiv:2410.19546.

[3] Bi, Jing, et al. "Verify: A benchmark of visual explanation and reasoning for investigating multimodal reasoning fidelity." arXiv preprint arXiv:2503.11557.

Dear Reviewer BGbw,

Thanks for your response. It appears there may have been some misunderstanding. The core contribution of our work is the proposed reinforcement learning strategy, which is specifically designed for reasoning generation tasks, rather than visual understanding. As demonstrated in the ablation study in Table 5 of the manuscript, this RL strategy improves the generation performance of our unified VLM by 6% points.

Moreover, Table 14 presents zero-shot test results on the visual reasoning tasks of our model trained with only image generation data. To further clarify our proposed RL strategy, we included only 1k relevant visual understanding samples during the RL stage, enabling us to further assess the model's performance on these tasks as shown in Table 16. Our initial goal is to use RL to transfer the reasoning capabilities of VLM to generation. However, this training method can also be fed back to VLM to improve understanding ability.

If you still have any concerns, we would be pleased to discuss them further with you!

Table 16: Detailed performance comparison. * indicates the model with more 1k visual understanding training data in the RL stage. Und. and Gen. denote understanding and generation, respectively.

Best regards,

Paper 13149 Authors