Vinci: Deep Thinking in Text-to-Image Generation using Unified Model with Reinforcement Learning

Thank you for your valuable feedback. We address each of your points individually below.

Q1. The results of SFT's model

Thank you for the insightful suggestion. We have accordingly supplemented Supervised Fine-Tuning (SFT) results, presented in the table below. While SFT yields a modest improvement (+7), the dominant gain (+15) emerges from the reinforcement-learning phase. This outcome substantiates that RL is significantly more effective than SFT in transferring the model’s visual-understanding capabilities into the image-generation process.

Q2. Compare with Best of N

We have additionally compared Vinci against Best-of-N baselines; Where Emu3(w.sf&Best-of-N) is fine-tuned with the same cold start data and generated with the Best-of-N (N=16) strategy. The results, summarized in the table below, show that Vinci retains a clear advantage. Best-of-N merely re-ranks fixed outputs without enhancing the generator itself, whereas our RL formulation transfers the model’s intrinsic understanding directly into the generation process, yielding sustained improvements in quality.

[1]A General Framework for Inference-time Scaling and Steering of Diffusion Models

Q3. The generalization capability

Thank you for the insightful suggestion; we have conducted the requested experiment, with results presented in the table below. The model achieves 50 % accuracy on color_attr which never explicitly seen during training, and the performance is close to Vinvi (54 %), demonstrating that the MCoT framework successfully generalizes its reasoning capabilities to novel visual properties.

Q4. More implementation details

To accommodate longer interleaved sequences, we extend the model's context length to 15,360 tokens. Each image is 512×512 in resolution, corresponding to 4,096 visual tokens.

The training is conducted on 16 A800 GPUs, and consists of two stages:

● In the cold start stage, we use a batch size of 64, a learning rate of 1e-5, and weight decay β=0.01. This stage lasts approximately 20 hours, allowing the model to acquire basic multimodal generation capabilities.

● In the reinforcement learning stage, we set the group size to 8 and train for an additional 60 hours, refining model performance through reward-based optimization.

Please refer to the supplementary material for more details, and we will also publish the code and data to ensure reproducibility of the work.

Thank you for your valuable feedback. We address each of your points individually below.

Q1. Expand the evaluation scope

We thank the reviewers for this suggestion. Owing to the tight rebuttal schedule, we engaged three undergraduate annotators to conduct a human evaluation on Geneval; the results are summarized in the table below. Despite the inevitable noise in automatic metrics, the human judgements confirm that Vinci still outperforms all baselines.

Additionally, we have now reported results on DPG-Bench and T2I-CompBench: Vinci ranks second on DPG-Bench—only marginally behind Janus-Pro-7B—and achieves the highest scores on T2I-CompBench, manifesting a substantial gain over Emu3. Taken together, these supplementary benchmarks further corroborate Vinci’s effectiveness.

Table 1. Human Evaluation

Table 2. DPG-Bench

Table 3. T2I-CompBench

Q2. About the MCoT data

We decompose data collection into sequentially verifiable sub-tasks. So we can interleave human inspection at each stage, yielding lower cumulative error than any single end-to-end model that lacks such intervention points.

Although we did not conduct a thorough manual inspection, we used GPT-4 to evaluate and filter out data that contains hallucinations or failures at the MCoT endpoints.
Empirically, sampling indicates that both Qwen-VL and GPT-4o exhibit strong and robust comprehension of the generated images, and we have not observed any systematic failures or notable shortcomings.

Due to space constraints we cannot include external links in this rebuttal, but the full dataset and codebase will be released concurrently with the camera-ready version to enable reproducibility and future extensions.

Q3. About the reward model

We acknowledge that detector failures can corrupt the reinforcement signal; however, the human evaluation reported in Q1 shows that such instances are not too much.

Developing more robust reward models is undeniably important, yet it lies beyond the scope of this paper and is therefore deferred to future research.

Q4. The hyperparameters for momentum reward

In our actual implementation, we have $\alpha = 0.5$, $\lambda = 0.5$, and $\gamma = 0.5$.
Here:

• $\alpha$ balances the single image quality score and momentum update;
• $\lambda$ is the time decay factor, encouraging improvement at earlier steps;
• $\gamma$ controls the sensitivity of $V_k$ to recent changes (smaller = more sensitive).

Due to time constraints, we additionally conducted two sets of sensitivity studies, shown below:

Table 4. Hyperparameters Sensitivity Studies

The results indicate that these hyperparameters have relatively little impact on the outcome.

Q5. The training cost and inference cost

• Cold-start stage: batch size = 64, learning rate = 1e-5, weight decay $\beta = 0.01$
  Duration: ~20 hours, to acquire basic multimodal generation capability.

• Reinforcement learning stage: group size = 8, training time = 60 hours
  Purpose: refine performance via reward-guided optimization.

Table 5. Inference Cost

Thank you for your valuable feedback. We address each of your points individually below.

Q1. The limitations of the application

We contend that this should not be regarded as a limitation of our work. Recently, the community has devoted considerable attention to unified autoregressive frameworks [1, 2, 3] that represent text and images within a single modality; consequently, a growing family of such unified models has emerged. We align with the position articulated in [1] that a truly unified autoregressive model should adopt a unified representation for both text and images—without which it cannot claim to be a foundational unified system. Through this paper, we also hope to further encourage the community to reflect on and refine the architectural design of unified autoregressive models.

[1] Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning

[2] Chameleon: Mixed-Modal Early-Fusion Foundation Models

[3] X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Again

Q2. Limited by GPT-4o's capabilities

We did not use GPT-4o to produce the MCoT annotations. High-quality human labels are prohibitively expensive to collect at scale; therefore, the current MCoT dataset pairs images generated by Flux with captions provided by Qwen-VL. GPT-4o is used to evaluate and filter out data that contains hallucinations or failures at the MCoT endpoints.
Empirically, sampling indicates that both Qwen-VL and GPT-4o exhibit strong and robust comprehension of the generated images, and we have not observed any systematic failures or notable shortcomings. The complete dataset will be released concurrently—please stay tuned.

Q3. The results based on other benchmarks

Following your suggestion, we have added results on DPG-Bench and T2I-CompBench. Vinci ranks second on DPG-Bench—only marginally behind Janus-Pro-7B—and achieves the highest scores on T2I-CompBench, demonstrating a substantial improvement. These additional benchmarks further corroborate the effectiveness of Vinci.

Table 1. DPG-Bench

Table 2. T2I-CompBench

Q4. Ablation on MCoT

Thank you for the suggestion. We have conducted an ablation on the MCoT data:

• w. MCoT: SFT with MCoT followed by RL.
• w/o MCoT & w. RL: Skip SFT and train directly with RL, using only a format reward to learn the interleaved output pattern.

Results show that the primary benefit of MCoT during cold-start is to support the interleaved output format; it yields only a modest improvement in generation quality (+7) compared with the larger gains delivered by RL (+15).
For Emu3—whose native architecture lacks interleaved text–image capability—learning the format solely through the format reward is difficult, leading to a significant drop in final performance (overall score: 64).

Table 3. MCoT Ablation Results

Q5. Compare with T2I-R1

We further benchmarked Vinci against T2I-R1 on both Geneval and T2I-CompBench. Vinci substantially outperforms T2I-R1 across all metrics. Notably, T2I-R1 is built upon Janus-Pro-7B—a stronger backbone than Emu3—underscoring the effectiveness of our approach.

Table 4. Geneval

Table 5. T2I-CompBench

Q6. The effectiveness of the momentum-based reward function

Due to space constraints in the rebuttal, we are unable to provide additional visualizations (they will be added in the subsequent version). Instead, we report aggregated statistics over three-round scenarios in the table below.

Results show that the vast majority of triple-round generations exhibit continuous improvement (Types 2 and 7 account for ≈ 80%). Furthermore, the same statistics after ablating $R_m$ drop to ≈ 63% for Types 2 and 7, which once again confirms that $R_m$ is instrumental in driving the model to iteratively refine its outputs.

Moreover, we will add the reward curve in the subsequent version to demonstrate the stable improvement.

Thanks for the authors' rebuttal. I found in Rebuttal Table 2. T2I-CompBench, the results of Emu3 are not provided. Since Emu3 is the base model, could you provide them to better validate the improvements?

Thank you for your valuable feedback. We address each of your points individually below.

Q1. About the cold-start

We disagree with the concern that cascading multiple models necessarily amplifies error. The staged pipeline is intentionally designed to constrain error, not propagate it. By decomposing data collection into sequentially verifiable sub-tasks, we can interleave human inspection at each checkpoint, yielding lower cumulative error than any single end-to-end model that lacks such intervention points. Moreover, the cold-start phase is scoped to format standardization—i.e., ensuring variable-length, interleaved text–image outputs conform to a strict schema—rather than boosting raw generation quality. Improvements in generation fidelity are eserved for the subsequent reinforcement-learning phase, whose quantitative gains are reported in Q4. Due to space constraints we cannot include external links in this rebuttal, but the full dataset and codebase will be released concurrently with the camera-ready version to enable reproducibility and future extensions.

Q2. Scalability of the method

Thanks to the rapid progress in the community, many multimodal large language model now support context windows of up to 128k tokens, enabling the generation of substantially longer multimodal sequences [1, 2]. Complementary efforts have simultaneously compressed visual representations—e.g., Anole reduces image tokens to only 1024. We argue, however, that the current bottleneck is not the length of the reasoning chain, but rather in how to achieve better reasoning during the image generation process.
This observation motivates the core contribution of our work: a novel end-to-end framework for interleaved text–image chain-of-thought reasoning.

[1] Thinking with Generated Images

[2] SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning

Q3. About the $R_i$ and the momentum reward function

Thank you for the valuable suggestion. $R_i$ is employed to assess the quality of a single generated image, whereas $R_m$—the momentum-based reward function—evaluates the quality of an entire image sequence by explicitly measuring whether the model makes consistent improvements across successive generation rounds. Following your recommendation, we have added a new ablation study (see the table below).

The quantitative results of this ablation, together with those reported in Q6, confirm that $R_m$ enables the model to perform effective self-correction toward higher-quality outputs.

Q4. The results of SFT's model

Thank you for your suggestion. We have incorporated the results of SFT's model as shown in the table below.

Thank you for the insightful suggestion. We have accordingly supplemented Supervised Fine-Tuning (SFT) results, presented in the table below. While SFT yields a modest improvement (+7), the dominant gain (+15) emerges from the reinforcement-learning phase. This outcome substantiates that RL is significantly more effective than SFT in transferring the model’s visual-understanding capabilities into the image-generation process.

Q5. Compare with the rewrite-based methods

We appreciate the suggestion and have now included a comparative experiment against prompt-rewriting baselines; the results, shown in the table below, demonstrate that our approach consistently outperforms existing rewriting methods.

Moreover, our interleaved text–image chain-of-thought framework eliminates the need for an extra rewrite model by leveraging self-reflection, thereby reducing both computational cost and latency while delivering a superior user experience.

[1] GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing

[2] T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT

Q6. The improvement across steps

Due to space constraints in the rebuttal, we are unable to provide additional visualizations (they will be added in the subsequent version). Instead, we report aggregated statistics over three-round scenarios in the table below.

Results show that the vast majority of triple-round generations exhibit continuous improvement (Type 2 and 7 account for ≈ 80%). Furthermore, the same statistics after ablating $R_m$ drop to ≈ 63% for Type 2 and 7, which once again confirms that $R_m$ is instrumental in driving the model to iteratively refine its outputs.

Moreover, we will add the reward curve in the subsequent version to demonstrate the stable improvement.

Thank you for your valuable feedback. We address each of your points individually below.

Q1. Compare with RL-based T2I models

We have added comparisons with recent RL-based text-to-image models (Tables 2 and 3 in [1]); the results are presented below. Vinci outperforms both RL-fine-tuned Stable Diffusion and FLUX, primarily because our pipeline incorporates a self-reflective mechanism during reinforcement learning that leverages the model’s own visual-understanding capabilities to enhance generation quality.

Q2. More details about MCoT data construction

For the object detection model, we use Mask2Former with a Swin-S backbone, following the official MMDetection configuration and pretrained on the COCO dataset. We apply a detection confidence threshold of 0.6 to remove low-confidence predictions. For computational efficiency and consistency, we retain at most 16 objects per class per image.

During data filtering, we introduce GPT-4o as a final quality filter to identify hallucinations or incoherent reasoning chains. Specifically, for each constructed MCoT sample, we query GPT-4o with a structured prompt to evaluate:

1. Whether the final image is semantically aligned with the original prompt;
2. Whether the reasoning steps are logically coherent and progressively improve the image’s consistency or visual quality.

GPT-4o provides a binary judgment ("pass"/"fail") along with a short explanation. We retain only the samples that pass this check and discard the rest. In practice, this filtering process removes approximately 8% of low-quality MCoT samples.

Q3. Derive theoretical guarantees for Equation 5’s stability

Below, we provide a comprehensive analysis of the reward function’s theoretical underpinnings.

1. Convergence Analysis

Finite Sequence Length ($n$):
For a finite sequence length $n$, all terms in the process reward function $R_m$ are finite sums, which guarantees that the value of $R_m$ remains finite and computable:

As $n \to \infty$:

a. The first term, which averages the quality scores $R_i(s_t)$, converges to the expected quality score, assuming a stable sequence:

b. The second term, which accounts for momentum-based improvement, converges depending on the decay factor $\lambda$:

\begin{cases} \lambda < 1: & \sum_{k=1}^{\infty} \frac{\Delta_k \lambda^{k-1}}{\sqrt{V_k} + \epsilon} \leq \frac{1}{\epsilon(1 - \lambda)} \quad \text{(absolutely convergent)} \end{cases}

\begin{cases} \lambda = 1: & \sum_{k=1}^{n-1} \frac{\Delta_k}{\sqrt{V_k} + \epsilon} \leq \frac{n - 1}{\epsilon} \quad \text{(bounded for finite $n$)} \end{cases}

Thus, the reward function $R_m$ guarantees convergence as $n \to \infty$, with the sum of improvements tending to a stable value based on the decay rate $\lambda$.

2. Edge Case Testing

The behavior of the reward function in edge cases demonstrates its adaptive nature and stability:

• No Improvement ($\forall k, \Delta_k = 0$):

  $$R_m = \frac{1}{n} \sum_{t=1}^{n} R_i(s_t) \quad \text{(degenerates to the average quality score)}$$

• Initial Large Improvement ($R_i(s_0) = 0$, $R_i(s_1) = 1$):

  $$V_1 = 0.1, \quad \frac{\Delta_1}{\sqrt{V_1} + \epsilon} \approx 3.162 \quad \text{(modest amplification of early improvement)}$$

• Low Variability Small Improvements ($\Delta_k = 0.01$, $V_k = 10^{-6}$):

  $$\frac{\Delta_k}{\sqrt{V_k} + \epsilon} \approx 10 \quad \text{(adaptive amplification of consistent small gains)}$$

• Constant Scores ($\forall t,\ R_i(s_t) = c$):

  $$\Delta_k = 0 \Rightarrow R_m = c \quad \text{(no momentum contribution)}$$

When the quality scores are constant, the reward function focuses entirely on the absolute quality score, with no momentumcomponent.

3. Mathematical Stability Conclusions

We formally summarize the stability guarantees of the momentum reward function $R_m$:

• Definition Completeness:

  $$\sqrt{V_k} + \epsilon > \epsilon > 0 \quad \text{ensures no division by zero}.$$

• Global Boundedness:

  $$R_m \in \left[0,\ 1 + \alpha \cdot \min\left( \frac{1}{\epsilon(1 - \lambda)},\ \frac{n - 1}{\epsilon} \right) \right],$$

  ensuring $R_m$ remains within a finite and interpretable range.

• Numerical Robustness:
  Setting $\epsilon = 10^{-8}$ avoids underflow and overflow. All inputs are bounded:

  $$R_i(s_t) \in [0, 1], \quad \Delta_k \in [0, 1], \quad V_k \geq 0.$$

• Convergence Guarantee:
  For $\lambda < 1$, the infinite sum of improvements is absolutely convergent:

  $$\lambda < 1 \Rightarrow \text{long-term stability and consistent reward computation}.$$

In conclusion, the momentum process reward function $R_m$ exhibits robust theoretical properties that ensure its stability and convergence during training.

Q4. The FLOPs per iteration

We measured the computational cost across different iterations; the results are shown in the table below. The overhead scales almost linearly with the number of generated images.

Thank you very much for your response. I have read your response to my comments and other reviewers. The authors address my concerns.