Dear Reviewer hSgy,

We sincerely thank you for your thorough review, thoughtful feedback, and positive remarks! We greatly appreciate your recognition of our novel reward designs, significant results, and clear writing. We respond to your concerns and questions below.

W1 & Q1: Two-stage RL setup.

R1: Thank you for raising this excellent point. It gives us the opportunity to clarify the core motivation behind the two-stage design of CoRL, which we view as a key strength of our framework rather than a contradiction. Specifically, our two-stage CoRL framework is easily understood through a Foundation-then-Specialization paradigm, which is both principled and practical for building powerful and versatile unified MLLMs.

• Stage 1 (Unified RL): The Foundation. The main goal of this stage is not to outperform our task-specific method or existing leading MLLMs on the target benchmarks, but to build a powerful generalist foundation. By synergistically optimizing both understanding and generation tasks, this stage endows the unified MLLMs with balanced and comprehensive capabilities. As shown in our ablation and pilot experiments, this unified RL training provides an overall accuracy boost across the board.

• Stage 2 (Task-specific/Refined RL): The Specialization. This second stage is an effective and crucial specialization step, demonstrating the flexibility and extensibility of our CoRL framework across a wide variety of diverse target tasks. The Math$^\text{We}$ benchmark (with text-rich geometric figures in images) is a perfect example, as it differs significantly from the unified RL data in terms of image types and reasoning types.

  Method	Math$^\text{We}$	Hints
  Baseline	5.9	Starting point
  + Unified RL	14.0 (+8.1)	Stage 1 builds a strong foundation.
  + Task-specific RL	15.8 (+9.9)	Specialization without a strong foundation is limited.
  + Unified RL & Task-specific RL	21.1 (+15.2)	Specialization on a strong foundation yields the best results.

  $\blacktriangleright$ The results clearly show that our two-stage paradigm is far superior to task-specific methods. This demonstrates that the unified RL foundation provides a much higher starting point for task-specific adaptation.

In summary, we argue that this two-stage paradigm effectively resolves the tension between general capability and specialized excellence. It offers a scalable and practical pathway for extending unified MLLMs to a diverse and continually expanding range of applications. We will revise our paper to make this motivation clearer.

W2: Ablation study on $\mathcal{R}_{\text{cycle}}$.

R2: Thank you for this valuable suggestion. We selected LPIPS for $\mathcal{R}_{\text{cycle}}$ primarily because it aligns better with human perceptual judgments than pixel-level metrics such as PSNR and SSIM. LPIPS measures image similarity in a feature space, making it more robust to minor, semantically irrelevant variations and better suited for rewarding high-level semantic consistency. Due to the short rebuttal period, we were only able to provide a comparison between LPIPS and MSE, but we will add a complete ablation study including PSNR and SSIM in the camera-ready version.

As shown in the table, LPIPS achieves better performance on both the DPG and WISE benchmarks compared to MSE, indicating its superiority in measuring semantic consistency.

W3: More related work.

R3: Thank you for pointing out these relevant works. We have updated the Related Work section to include them. Specifically, we have added MetaMorph, LMFusion, and BLIP3-o to AR-Diff, and OmniMamba to F-AR.

We hope these clarifications address your concerns. Thank you again for your constructive feedback and affirmation of our paper.

Dear Reviewer wJWf,

Thank you for your comprehensive review and for recognizing the intuitive nature and significance of our work. We are particularly grateful for your clear feedback and the opportunity to resolve your concerns, especially regarding the fairness of the SFT cold-start baseline. We respond to your concerns and questions below.

W1: Concerns on SFT cold-start baseline.

R1: Thank you for raising this important point. We would like to clarify that the SFT cold-start baseline was indeed trained on both understanding and generation tasks to ensure a relatively fair comparison.

• Training Data. As described in the caption of Table 3, the SFT cold-start data consists of x2x_rft_22k, mcot_r1_mcq (22K), and mcot_r1_vqa (22K). Among them, x2x_rft_22k is the same dataset used in our unified RL training and includes both generation and understanding tasks. Therefore, the SFT cold-start baseline does not only make use of understanding data. However, our initial attempts showed that using only x2x_rft_22k as the SFT cold-start data was almost ineffective. Even worse, it significantly degraded the model’s multimodal understanding capabilities. To address this, we ultimately used x2x_rft_22k, mcot_r1_mcq (22K), and mcot_r1_vqa (22K).

• Training Objective. During the SFT cold-start stage, we follow Janus-Pro's training objective and use the cross-entropy loss to simultaneously optimize the model's generation and understanding capabilities.

• Results. You made an astute observation. In practice, we did observe that SFT cold-start performs better than unified RL on certain understanding tasks, such as Math$^\text{We}$ and Logic$^\text{VT}$. This is likely due to task imbalance and the next-token prediction mode of autoregressive LLMs during SFT training, which is more direct and friendly for optimizing text generation than RL reward feedback.

W2: Scalability of CoRL.

R2: We agree that verifying scalability is crucial. To address this, we conducted a new experiment using Janus-1.3B [1] as the baseline, given the limited time during the rebuttal period. The results show significant and consistent improvements across different types of benchmarks. We will incorporate further validation on larger-scale models in our open-source project and the camera-ready version.

[1] Chengyue Wu, Xiaokang Chen, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In CVPR, 2025.

We hope these clarifications and additional results address your concerns. Thank you once again for your time and valuable feedback.

Dear Reviewer wJWf,

I hope this message finds you well. As the discussion period draws to a close, we would like to kindly confirm whether our last response has addressed your concern regarding CoRL's scalability to larger models. If you have any further feedback, we would greatly appreciate your insights.

Thank you once again for your time and thoughtful review.

Dear Reviewer 5YVq,

We sincerely thank you for your constructive and insightful review. We are delighted that you found our work to be compelling, novel, and important in multimodal AI, and we deeply appreciate your positive assessment and recommendation for acceptance. We respond to your identified weaknesses and suggestions below.

W0: Endogeneity in $R_\text{TIM}$ due to self-evaluation.

R0: This is a very perceptive and insightful point that touches upon an inherent challenge in self-rewarding RL. We are grateful you raised it. In our work, $R_\text{TIM}$ was indeed computed using the policy model itself, but the total unified reward is a composite signal. To a certain extent, the potential endogeneity is balanced by the other fully external rewards: $R_\text{cycle}$, $R_\text{Acc}$, and $R_\text{Format}$. This diverse reward structure inherently provides multi-faceted feedback and acts as a form of regularization, preventing the policy model from simply "gaming" the self-evaluated $R_\text{TIM}$ signal.

Furthermore, we conducted additional experiments comparing $R_\text{TIM}$ with external metrics such as the CLIP score on text-to-image benchmarks. $R_\text{TIM}$ demonstrates stronger alignment with dense prompts and better downstream performance, suggesting that it captures a more robust alignment signal beyond merely overfitting to internal representations.

W1: Discussion on reward scaling.

R1: Thank you for pointing out this lack of clarity regarding a critical implementation detail. In practice, each reward from $R_\text{cycle}$, $R_\text{TIM}$, $R_\text{Acc}$, and $R_\text{Format}$ is normalized to the range $[0, 1]$ before aggregation. This ensures that all rewards operate within comparable scales and prevents any single component from dominating due to scale differences. We will clarify this normalization process more explicitly in the revised version.

W2: Discussion on the sensitiveness of hyperparameter λ.

R2: Thank you for raising this point. The choice of λ = 0.8 was determined empirically. To provide a more formal justification, we have conducted the following ablation study on the hyperparameter λ.

The results show that λ = 0.8 yields the best generation performance while only slightly affecting understanding capabilities, whereas setting λ = 1.0 slightly boosts understanding at the cost of generation performance. Moreover, we agree that the model's sensitivity to λ warrants further analysis, and we will include an ablation sweep in the camera-ready version to explore this sensitivity more systematically.

Thank you once again for your constructive feedback and affirmation of our paper. We will incorporate these valuable suggestions and additional ablations in the camera-ready version.

Dear Reviewer 9e1d,

Thank you for your thorough and constructive review. We appreciate your positive feedback on the novelty and effectiveness of our work, the comprehensiveness of our experiments, and your kind suggestion regarding data anonymization. We respond to your concerns and questions below.

W1: Ablation study on $R_\text{TIM}$ and CLIP score.

R1: Thank you for raising this crucial point. Comparing the CLIP score with $R_\text{TIM}$ under our final RL training paradigm is an essential ablation. We have conducted additional experiments to complement and improve the ablation study in Table 4. Compared to the CLIP score, $R_\text{TIM}$ demonstrates better overall performance, especially on the DPG benchmark with dense, long-horizon prompts for image generation, highlighting its ability to better capture fine-grained semantic alignment.

W2: The main claim of this work is that “co-RL is better than task-specific RL.” Why, then, is task-specific RL still necessary?

R2: Thank you for this insightful question. Our core argument is not that CoRL (Unified RL) is better than task-specific RL, but rather that Unified RL provides a much stronger foundation upon which task-specific RL can build to achieve the expected results. Specifically, our two-stage CoRL framework is easily understood through a Foundation-then-Specialization paradigm, which is both principled and practical for building powerful and versatile unified MLLMs.

• Stage 1 (Unified RL): The Foundation. The main goal of this stage is not to outperform our task-specific method or existing leading MLLMs on the target benchmarks, but to build a powerful generalist foundation. By synergistically optimizing both understanding and generation tasks, this stage endows the unified MLLMs with more balanced/comprehensive capabilities that task-specific training alone cannot foster. As demonstrated in our ablation and pilot experiments, this unified RL training provides an overall accuracy boost across the board.

• Stage 2 (Task-specific/Refined RL): The Specialization. This second stage is an effective and crucial specialization step, demonstrating the flexibility and extensibility of our CoRL framework across a wide variety of diverse target tasks. The Math$^\text{We}$ benchmark (with text-rich geometric figures in images) is a perfect example, as it differs significantly from the unified RL data in terms of image types and reasoning types.

  Method	Math$^\text{We}$	Hints
  Baseline	5.9	Starting point
  + Unified RL	14.0 (+8.1)	Unified RL builds a strong foundation.
  + Task-specific RL	15.8 (+9.9)	Specialization without a strong foundation is limited.
  + Unified RL & Task-specific RL	21.1 (+15.2)	Specialization on a strong foundation yields the best results.

  $\blacktriangleright$ The results clearly show that our two-stage paradigm is far superior to task-specific methods. This demonstrates that unified RL provides a much stronger starting point for task-specific adaptation.

Therefore, task-specific RL is a necessary and important step in our CoRL framework, leveraging the powerful foundation built by Unified RL to push the boundaries of performance. We will revise our paper to articulate this motivation more clearly.

W3: Evaluation setup, and three-to-one merged model results.

R3: Thank you for the question. We are happy to clarify the evaluation setup and provide the corresponding results.

• Evaluation Setup. For multimodal understanding, only on the three mixed benchmarks (Math$^\text{VT}$, Math$^\text{VS}$, Math$^\text{Vis}$), whose test sets include both multiple-choice (MC) and open-ended (OE) QA formats, we merged the two task-specific models to obtain a final model capable of following both types of task instructions.

• Results of Three-to-One Merged Model. We report the results in the table below. As shown, a naive "three-to-one" merge of all task-specific models leads to significant performance degradation in text-to-image generation, likely due to task imbalance or instruction interference or ambiguity. In contrast, its influence on multimodal understanding is less pronounced.

  Benchmarks	Task Type	Three-to-One Merged	ULM-R1
  GenEval	image generation	0.75	0.77
  WISE	image generation	0.29	0.33
  DPG	image generation	83.20	83.92
  MMMU	MC	42.0	42.3
  MMStar	MC	47.5	47.6
  Math$^\text{We}$	MC	21.5	21.1
  MMVet	OE	43.0	43.9
  POPE	OE	88.7	88.9
  Logic$^\text{VT}$	OE	35.0	34.5
  Math$^\text{VT}$	MC & OE	42.4	42.5
  Math$^\text{VS}$	MC & OE	25.4	25.4
  Math$^\text{Vis}$	MC & OE	22.2	22.0

Q1: For $R_\text{OE-Acc}$, does this refer to Exact Match?

A1: In the OE tasks, the accuracy reward is based on the correctness of the final extracted answer, not on a strict exact match of the entire generated text (which includes the reasoning process). For instance, for a question whose answer is "3π", we parse the final output within the <answer>3π</answer> tags and check if it matches the ground truth. This is a binary reward: 1 for a correct final answer, 0 otherwise.

We hope these clarifications and additional results address your concerns. Thank you again for your time and valuable feedback.

The responses have addressed most of my concerns. I have increased my scores accordingly.

Dear Reviewer 9e1d,

Thank you again for your constructive review. As the discussion phase is approaching its end, we would like to confirm whether our responses have sufficiently addressed your concerns. If you have any remaining questions or concerns that require further clarification, please don’t hesitate to let us know.

We sincerely look forward to your feedback.