Q1 & Q3: The term "Token-Level CoT" for the standard autoregressive, patch-by-patch generation process can be seen as a loose analogy. While the authors define it based on its sequential and conditional nature the "thought" aspect is less explicit compared to the semantic-level CoT. Since there is no explicit thinking process, just a patch-by-patch image generation process. & How does this core aspect of 'thought' translate to the token-level visual generation? A deeper conceptual clarification would help solidify the paper's contribution in this regard.

Thank you for your valuable advice. We acknowledge that our terminology extends the CoT concept beyond its traditional textual reasoning context. We would like to clarify the concept of token-level CoT and address your concerns:

1. First, we believe there must exist a type of reasoning in the process of image generation, otherwise, we will not observe model with only token-level CoT optimized beats the baseline model on the same prompt or even in the setting where the top half image is given and the model is asked to generate the remaining ones.

2. Then, we will revisit the definition of CoT in the textual context, and lists where the autoregressive image generation bears the similarity.

   For the textual CoT, it is defined as the intermediate natural language reasoning steps that lead to the final output in [1]. The CoT contains a long reasoning process with problem analysis and logical inference, then followed by a short final answer. It is commonly acknowledged that the part of the long output corresponds to the part of extensive thinking. The 'thought' process is reflected in two ways:

   • Decomposition of the problem to steps
   • Logical coherence within the step

   Since the model outputs the tokens in the language space, this process can be easily interpreted by the human. The two aspects correspond to the logical path of solving and the correctness of each reasoning step.

   For the token-level CoT, the key insight is that the model's "thought process" or intermediate reasoning occurs simultaneously with final output generation, rather than as separate steps. The model reasons and generates in parallel, integrating its thinking directly into the creation process. Different from textual CoT, the token-level CoT is much longer compared and also holds a strict sequential and conditional nature. The two 'thought' process become:

   • The logical path of solving is determinant in the context of token-level CoT, which is the raster-scan order.
   • The correctness of each reasoning step corresponds to the choice of specific visual tokens, which is what our method tries to optimize. This correctness includes patch coherence, object appearance, lighting conditions, and other visual details. These intermediate steps could also be verified and interpreted by humans by decoding the visual tokens into images.

3. In summary, the modality difference between autoregressive text and image generation causes the "thought" happens in the different place. We draw an analogy between the two key aspects between the textual CoT and token-level CoT. The token-level CoT refers to the 'thought' process we identify in the autoregressive generation. We focus on optimizing the aspect on choosing the correct visual token.

We will include this clarification and comparison in our final version.

[1] Wei, Jason, et al. "Chain-of-thought prompting elicits reasoning in large language models."

Q2: About Semantic-level CoT: unambiguous prompts (e.g., 'a red apple'), could the authors discuss how the model avoids 'over-planning'—where the generated textual details might unnecessarily constrain the model's stochastic creativity in aspects like lighting, texture, and composition?

Thank you for this insightful question. We would like to address your concerns both qualitatively and quantitatively:

1. Qualitatively (Figure 13 to 14)

   Our experiment analysis in Figure 13 and 14 in the appendix reveals that our introduced semantic-level CoT actually enhances the diversity of the generated images. As shown in Figure 13, the base model Janus-Pro possess relatively low diversity. The generated images of Janus-Pro are basically the same: there is a tree branch on the bottom left corner and color of the origami cranes are all white with same zoom-in scale. While T2I-R1 generates much more diverse images with different backgrounds, cranes colors, and zoom-in scales. The same conclusion holds for Figure 14.

2. Quantitatively

   We further conduct an experiment to validate this conclusion. We generate 300 unambiguous prompts based on the example 'a red apple'. We enumerate 150 common objects and 10 colors and then randomly choose 2 colors to combine with each object. Then we use the base model Janus-Pro, the model trained only with the semantic-level CoT, and T2I-R1 to generate 10 images on each prompt. Here we include the model trained only with the semantic-level CoT to eliminate the potential influence on the diversity from the token-level CoT. The results are as follows:

   Model	Diversity
   Janus-Pro-7B	7.460
   Only Semantic-level CoT optimized	8.612
   T2I-R1	8.519

   The results clearly demonstrate that semantic-level CoT increases output diversity by 15.4% instead of harming it.

Q4: To further strengthen the analysis on the impact of BiCoT-GRPO on Visual Quality, could the authors provide more details on the human study protocol? The paper's primary contribution is enhancing compositional reasoning. Could the authors offer a more explicit discussion on how the BiCoT-GRPO framework specifically impacts aesthetics as a distinct objective? For example, is there a risk that the RL process might overly prioritize compositional correctness at the expense of aesthetic quality in some cases, and how does the ensemble reward system mitigate this potential conflict?

Thanks for this important question regarding the balance between the compositional generation and aesthetic quality.

1. Human Study Protocol

   We ask the annotators to judge only according to the visual appeal. We provide examples of an unsatisfactory generation, including different aspects:

   • Corrupted floating words
   • Distortion like melting effect of faces, hands, or other objects
   • Extra or missing hands, legs, tails, and etc.
   • Object merging, frequently observed for compositional prompts
   • Low-level aspects including over exposure or over saturatation
   • Blurred areas, especially in fine details like keyboard

   The instruction to the annotator highlights that the highly ranked images should show no or less problems above compared with others.

2. Aesthetics Quality

   We would like to address your concern of the aesthetics of the generated images:

   HPS score contains extra judgment based on the human aesthetic preferences, as shown in Fig. 4 and Line 167. For the training data of HPS [1], the model is trained on the human preference data, which contains both the alignment and visual appealing aspects. Therefore, the images generated by T2I-R1 actually shows better visual quality compared with the baseline model.

   We employ a different image quality judgement model to score the generated images from both Janus-Pro and T2I-R1 to further showcase the superior visual quality:

   Model	Aesthetic Score
   Janus-Pro-7B	5.41
   T2I-R1	5.76

   Specifically, we employ a CLIP-based aesthetic predictor, which receives the image and outputs a score. The predictor does not require prompt input so that the score is purely based on the visual appealing. We input all of the generated images when testing T2IComp-Bench to this predictor and record the average score. The result showcases that T2I-R1 improves the generation quality of the baseline model.

[1] Wu, Xiaoshi, et al. "Human preference score: Better aligning text-to-image models with human preference." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

Q1: The evaluation of obj detector cannot be fully showcased in T2I-CompBench. Need metrics of counting in Geneval to better illustrate.

Thanks for your advice. We provide the results comparison of T2I-R1 with the baseline model on the counting tasks in three different benchmarks: GenEval, T2IComp-Bench++, and GenAI-Bench:

As the results shown, T2I-R1 largely surpasses the baseline model on all of the three benchmarks. We believe this could validate the effectiveness of the object detector in counting tasks. Note that, we cannot reproduce the result of the baseline model Janus-Pro-7B in their paper. For fair comparison, we evaluated its performance on our own, ensuring that both the baseline model and T2I-R1 used the same random seed during generation.

Q2: In Fig. 5, token-level CoT is similar to the baseline output. It seems like the semantic-level CoT benefits more.

Sorry for the confusion. We would like to address your concerns from the following two aspects:

1. Qualitative Aspect (Figure 5)

   The prompt in Fig. 5 are sourced from WISE, which requires a strong reasoning capability. We do not inlcude this kind of data in our training dataset. Therefore, without the semantic-level CoT helping analyze the prompt, this prompt is out of distribution for the model, causing the baseline model and model with only token-level CoT optimised generating similar non-sense images.

   To observe the gain brought by the token-level CoT, we should include the semantic-level CoT when comparing, i.e., comparing the second and forth column in Fig. 5. The improvement brought by the token-level CoT is obvious. For example, with the top left prompt "Show a plant that is asymbol of good fortune inIrish culture, and is knownfor its three-lobed leaves":

   • Semantic-level only (Column 2): Correctly identifies "clover" but produces a flat, textureless image with poor lighting and plain white background
   • With token-level CoT (Column 4): Generates vibrant colors, natural lighting, realistic textures, and professional depth-of-field that makes the subject prominent

2. Quantitative Aspect (Table 4)

   We also illustrate the effectiveness of the token-level CoT quantitatively.

   First, for the compositional ability illustrated in Tabel 4, it can be shown that only optimising the token-level CoT brings large improvement, with 20% on T2I-CompBench and 5% on WISE benchmark.

   Second, we provide a quantitative metric to measure the improvement of visual quality brought by the token-level CoT. Specifically, we employ a CLIP-based aesthetic predictor, which receives the image and outputs a score. The predictor does not require prompt input so that the score is purely based on the visual appealing. We input all of the generated images when testing T2IComp-Bench to this predictor and record the average score.

   Model	Semantic-level CoT	Token-level CoT	Aesthetic Score
   Janus-Pro-7B			5.41
   -	$\checkmark$		5.58
   -		$\checkmark$	5.68
   T2I-R1	$\checkmark$	$\checkmark$	5.76

   The improvement from Line 2 to Line 4 and from Line 1 to Line 3 both demonstrate the effectiveness of token-level CoT in visual quality. We will include the illustration of Fig. 5 and the aesthetic score in the final version to make the effectiveness of token-level CoT more obvious.

Q3: Function of HPS and ORM is overlap

Sorry for the confusion. While both models evaluate prompt alignment, they also serve other distinct purposes:

1. ORM focuses exclusively on prompt-image alignment, whereas HPS incorporates human aesthetic preferences alongside alignment assessment, as demonstrated in Figure 4 and Line 167. For the training data of ORM, the label of an image is Yes whenever it is aligned with the prompt. For HPS, the model is trained on the human preference data [1], which contains both the alignment and visual appealing aspects.
2. ORM leverages multimodal large language models, thereby preserving extensive world knowledge, while HPS relies on CLIP-based architectures. With the knowledge from LLM, ORM is expected to provide a more faithful score when encourtering a unusual prompt or generated image.

We will clarify these distinctions in the text in our final version.

[1] Wu, Xiaoshi, et al. "Human preference score: Better aligning text-to-image models with human preference." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

Q1: Semantic-level CoT is to enrich the initial prompt by adding more details. However, this idea is closely related to prompt enrichment, which is not discussed.

Thanks for pointing out. After thoroughly reading the related papers mentioned, we acknowledge that the idea of semantic-level CoT is related to the prompt enrichment work. However, we would like to highlight some key differences and advantages between our method and the prompt enrichment technique:

1. The semantic-level CoT is generated from the image generation model itself, while the prompt enrichment leverages an extra LLM. Our design facilitates a joint-optimization of the prompt design and the image generation. On the contrary, the prompt enrichment is generation-model-agnostic, which means that although the prompt is enriched with more details, it is not necessary that the generation model could generate a better image based on this enriched prompt.

   To validate this claim, we conduct the following experiments on T2IComp-Bench:

   	Prompt Generation Model	Image Generation Model
   Exp1	Janus-Pro-7B	Janus-Pro-7B
   Exp2	Qwen2.5-7B-Instruct	Janus-Pro-7B

   Note that, T2I-R1 is obtained after Exp1 is trained with our proposed BiCoT-GRPO. The results are shown below:

   	Color	Shape	Texture	Spatial	Non-Spatial	Complex	Aesthetic
   Janus-Pro	0.6359	0.3528	0.4926	0.2061	0.3085	0.3559	5.41
   Exp1	0.6451	0.3773	0.5262	0.2298	0.3049	0.3578	5.31
   Exp2	0.7004	0.4111	0.5892	0.2522	0.3088	0.3597	5.30
   T2I-R1	0.8130	0.5852	0.7243	0.3378	0.3090	0.3993	5.76

   As shown, using the prompt enriched from either Qwen2.5 or Janus-Pro-7B itself still far lag behind T2I-R1 in both compositional abilities and aesthetics.

   Specifically, to evaluate aesthetics, we employ a CLIP-based aesthetic predictor, which receives the image and outputs a score. The predictor does not require prompt input so that the score is purely based on the visual appealing. We input all of the generated images when testing T2IComp-Bench to this predictor and record the average score.

2. Qualitative examples shown in Figure 9 to 12 also shows that our method goes beyond prompt enrichment. Although the zero-shot semantic-level CoT (enriched prompt) in Fig. 9 to 11 all correctly include all the key objects mentioned in the prompt and add more details, the model still cannot generate satisfying result. Simply enriching the prompt is not sufficient to bring notable improvements.

3. Since the semantic-level CoT and the image are generated from the same model, the deployment cost is much smaller than the prompt enrichment method:

   	Memory (GB)
   T2I-R1	16.657
   Exp2	30.501

We will include the discussion of prompt enrichment and cite all the works you mentioned in the related work in our final version.

Q2: Whether token-level CoT is meaningful under diffusion paradigms is unclear. Alternative strategies, such as VAR should be discussed.

Thank you for raising this important point. As illustrated in Line 27, the focus of our paper is the autoregressive text-to-image generation, where the images are generated token by token in a discrete space.

• For the diffusion-based models, they employ iterative denoising in continuous space that are totally different from our settings. While some notable parallels exist (e.g., the generation process of diffusion model resembles a type of CoT happening in the noise space instead of the token space), this would require a complete theoretical reformulation. We will leave this for future work and will further clarify our focus in the introduction.

• As for VAR, we believe our method could be directly applied. Techinically, VAR also adopts an autoregressive generation manner and the generation image encoder is also discrete. The only difference is that VAR first generates the coarse tokens then the fine-grained ones, but this does not conflict with our motiavation or method. In fact, it is interesting that this hierarchical reasoning from the coarse structure to fine details perfectly matches the spirit of CoT reasoning. We will also include this discussion into the future work part in our final version.

Q3: Role of RL should be justified. It's unclear whether RL is essential or whether similar gains could be achieved through supervised or hybrid optimization.

Thanks for your advice. We conduct a supervised finetuning (SFT) and a hybrid optimization (SFT then RL) to validate the advantage of direct RL training.

• SFT: We first source 200K high-quality image-text pairs. Then we use a MLLM to generate a detailed thinking process based on the image and original short prompt, which is used as the semantic-level CoT in the SFT training. During training, we jointly train two tasks together:

  1. Input: Image Prompt + Reasoning Instruction; Output: Semantic-level CoT
  2. Input: Image Prompt + Semantic-level CoT + <img_start>; Output: Token-level CoT.

  This design strictly follows the generation manner of T2I-R1 for a fair comparison. We train the model for one epoch with learning rate of 1e-5.

• Hybrid Optimization: After the SFT training, we follow the same setting of T2I-R1 to train the SFT model with reinforcement learning.

The results are shown below:

Key Findings:

• Both SFT and hybrid optimization fails to outperform training only with RL. This likely stems from the model's existing proficiency of generating semantic-level and token-level CoT. The additional SFT process forces the model to learn from data that is misaligned with its current capabilities, resulting in minimal or even detrimental improvements. Furthermore, SFT constrains the model's generation capabilities to match the training data distribution, reducing output diversity and limiting the exploration space during the subsequent RL phase.

• The training cost of SFT and hybrid optimization are also larger than exclusive RL training. We showcase the total training hours and the data that needs annotation. SFT requires annotating semantic-level CoT based on existing text-image pairs datasets while RL needs no data annotation.

  	Total Training Hours	Needed Annotation
  SFT	4.3	Semantic-level CoT
  Hybrid optimization	29.0	Semantic-level CoT
  T2I-R1 (RL)	14.5	None

We will include this experiment and the corresponding conclusion in our final version to stength the role of reinforcement learning.

Q4: Most performance comes from semantic-level CoT. It remains unclear how much of this improvement comes purely from prompt enrichment, without involving RL or token-level reasoning. A fairer comparison would include a model with enriched prompts but no further optimization, especially in the context of diffusion-based generators.

Thanks for your valuable advice. We want to address your concerns from three aspects:

1. Token-level CoT contributes substantially to performance improvement. We show the benefit obtained from token-level CoT from two aspects:

   1. Qualitative Aspect (Figure 5)

      The ablation visualizations in Figure 5 clearly demonstrate token-level CoT's crucial role. . Comparing columns 2 (semantic-level CoT only) and columns 4 (semantic-level and token-level CoT), the improvement brought by the token-level CoT is obvious. For example, with the top left prompt "Show a plant that is asymbol of good fortune inIrish culture, and is knownfor its three-lobed leaves":

      • Semantic-level only (Column 2): Correctly identifies "clover" but produces a flat, textureless image with poor lighting and plain white background
      • With token-level CoT (Column 4): Generates vibrant colors, natural lighting, realistic textures, and professional depth-of-field that makes the subject prominent

   2. Quantitative Aspect (Table 4)

      We also illustrate the effectiveness of the token-level CoT quantitatively.

      First, for the compositional ability in Tabel 4, it can be shown that only optimising the token-level CoT brings large improvement, with 20% on T2I-CompBench and 5% on WISE benchmark.

      Second, we provide a quantitative metric to measure the improvement of visual quality brought by the token-level CoT:

      Model	Semantic-level CoT	Token-level CoT	Aesthetic Score
      Janus-Pro-7B			5.41
      -	$\checkmark$		5.58
      -		$\checkmark$	5.68
      T2I-R1	$\checkmark$	$\checkmark$	5.76

2. Ablation of Prompt Enrichment.

   The qualitative results from Figure 9 to 12 in the appendix showcase that a valid prompt enrichment (zero-shot semantic-level CoT in the figure) does not necessarily produce aligned images. Without learned coordination between planning and generation, the model fails to follow the enriched prompt. In contrary, by our optimization, T2I-R1 could leverage and understand this enriched prompt to generate satisying results.

   The quantitative results from Exp1 and Exp2 have also shown that the improvement of our method has only partly come from the prompt enrichment.

3. While we focus on autoregressive models, we understand the reviewer's interest in broader applicability. However, testing prompt enrichment on diffusion models seems to provide very limited insights. There are huge differences including theoretical foundations, architecture, and training method between these two kinds of models. This cross-paradigm results wouldn't validate or invalidate our autoregressive findings.

   Nevertheless, we're open to discussing specific experimental designs if the reviewer believes certain diffusion experiments would strengthen our analysis.

Example 1:
Original Prompt: a blue backpack and a red book
Generated Semantic-level CoT: I have generated the following visualization as per your instructions:\n\n1. There is 1 blue backpack and 1 red book\n2. The blue backpack is a rectangular prism (spiked dots on its top side): the color blue\n3. The red book is a square shape (corner dots): the color red\n4. The items are placed side by side: a small distance between them

Example 2:
Original Prompt: "a painting on the top of a rabbit
Generated Semantic-level CoT: Below is a visualization of the prompt \"a painting on the top of a rabbit\":\n\n![A painting on a rabbit (top)](xxx/rabbit-painting-top-image).\n\nThe visual depicts a beautiful, small painting placed on the top of a rabbit. The painting features bright colors and intricate patterns, adding a touch of elegance and grace to the rabbit's head. The rabbit, with its soft fur and detailed whiskers, is pictured as an adorable, lifelike creature. The painting and rabbit create a visually appealing and harmonious piece, portraying a delightful and creative scenario.

Orignal Prompt: a diamond cufflink and a spherical shirt button
Generated Semantic-level CoT: A diamond cufflink sparkles next to a single spherical shirt button. The diamond cufflink is rectangular with a vivid sparkly cut, while the button is glossy and round, pure white with a smooth surface.

Orignal Prompt: A person is looking at a rainbow and marveling at its beauty.
Generated Semantic-level CoT: A person standing outdoors is gazing upwards at a vivid, multi-colored rainbow arching in the sky. The rainbow consists of seven distinct colors: red, orange, yellow, green, blue, indigo, and violet. The person's face, showing amazement, is partially illuminated by the sunlight filtering through the rainbow's colors.

Q1: Two-step generation requires semantic-level CoT + image tokens, which increases inference time. Doesn't compare a dedicated llm for planning + seperate image generator.

Thanks for you valuable advice. To address your concerns, we conducted additional experiments comparing our method against dedicated planning approaches and applying prompt enrichment on other backbones:

We show the results on T2IComp-Bench, the aesthetic score and the inference cost below:

Key Findings:

• Beyond Prompt Enrichment: While dedicated planning models (Exp1-3) show modest improvements over the baseline, all fall significantly short of T2I-R1's performance. This demonstrates that our method's effectiveness extends well beyond simple prompt enrichment and highlights the crucial role of our joint RL optimization framework.

• Aesthetic Enhancement: Our approach not only improves compositional capabilities across all categories but also enhances aesthetic quality (5.76 vs. baseline 5.41), indicating holistic image generation improvement.

• Excellent Trade-off: Our method demonstrates remarkable efficiency by adding only modest computational overhead—0.8 GB additional memory and 1.9 seconds per sample—when incorporating semantic-level CoT. With an average CoT length of just 97.1 tokens, our approach achieves a 16% average improvement across benchmarks. This efficiency significantly outperforms existing RL-enhanced reasoning methods that require thousands of reasoning steps to achieve comparable performance gains (as demonstrated in Figures 2 and 3 of the DeepSeek-R1 paper [1]).

We will include results from these three experiments in the final version to provide a more comprehensive comparison.

[1] Guo, Daya, et al. "Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning." arXiv preprint arXiv:2501.12948 (2025).

Q2: Ensemble of multiple expert models requires careful tuning of multiple reward models and their combinations. The importance of each reward model is not clear in Table 3.

Thanks for pointing out! In Table 3, the weight of all the rewards are set to 1. Our preliminary experiment shows the weight of the reward model causes little influence to the final training result. Here we provide a detailed study to illustrate the our method is stable to the weight of the reward model. We follow the training setting of T2I-R1, where three reward models, HPS, GroundingDINO, and GIT are employed.

In our experiments, we multiply the weight of each reward model by 5 respectively, while maintaing the weight of other two reward models. Apart from these three experiments, we also conduct an experiment where we compute the relative reward of each reward model inside the group, and then sum up the normalized reward as the final reward for the sample. This eliminates the mean and variance difference among the reward models (termed as Normalized Reward). The results are shown below:

The result showcases that the training procedure is relatively stable to the weights of the reward model, with an average range of 2.51% fluctuation. We acknowledege that the choice of the model weights are of great importance and we will study how to adaptively find the best weights in our future work.

Q3: Multi-stage training with RL makes reproduction challenging.

Sorry for the confusion. We would like to clarify that the stage in the paper does not mean different training phases. Our method requires only one stage of training. The step shown in the Fig. 3 refers to the sequential generation process of semantic-level CoT and token-level CoT. Detailedly, since the baseline model Janus-Pro-7B does not possess the ability to generate the image start token autonomously, we employ a two-step generation method. The two steps are:

1. First step: we input the image prompt and the reasoning insturction to generate the semantic-level CoT.

2. Second step: we input the image prompt, previously generated semantic-level CoT and the image start token to obtain the token-level CoT.

We will further clarify this concept and replace the word stage with step to avoid causing confusion in the final draft.

Q4: The work lacks insights into RL training and optimization. Overall, it appears to be a straightforward application of the GRPO algorithm to text-to-image tasks, lacking fundamental innovation.

We respectfully disagree with the assessment for the following reasons:

1. Our BiCoT-GRPO method provides the first insight into leveraging GRPO to jointly optimize the CoT from both the text and image modalities in a single iteration.
2. Besides, we propose a novel ensemble of reward models for the reward modeling in the GRPO training. Previous methods only adopt GRPO in the understanding tasks, where the reward functions are mostly rule-based and format rewards. We first study the reward functions in GRPO for the image generation tasks. Our ensemble approach combines multiple reward perspectives to deliver holistic evaluation while preventing reward hacking.
3. Finally, as illustrated in Line 2-9, the main focus of our paper is to explore how what types of CoT process could be utilized in the text-to-image generation task. We identify that RL is an ideal training method for eliciting the reasoning capability of the base model to generate two types of CoT. Here, GRPO is simply the tool that enables our investigation, much like how supervised learning enables many fundamental discoveries without being the focus itself.

Q5: The computational overhead and complexity of the two-step generation process and ensemble reward system, which may limit practical deployment and accessibility.

Thanks for your valuable advice. We address each aspect below:

1. The semantic-level CoT generation introduces minimal computational overhead, as demonstrated in the comparison below:

Given the substantial improvement across benchmark results, this minor overhead is practically acceptable compared to the gains achieved.

2. The ensemble of the reward model is only used in the training time and cost-free for the inference.
3. Compared to alternative two-step generation approaches that require separate models (e.g., a dedicated LLM for planning plus an independent image generator), our unified approach requires only a single model, effectively reducing the GPU memory requirements by half during inference.

Q6: The work lacks insights into RL training and optimization. Overall, it appears to be a straightforward application of the GRPO algorithm to text-to-image tasks, lacking fundamental innovation.

Please refer to our reponse to Q4.