Dear Reviewer 5UPe,

Thank you for your time and thoughtful feedback on Mask-GRPO. We are pleased that you recognize the novelty and effectiveness of our method. Please find our responses below:

Q1: Clarification on Transition Probability Formulations

Could the authors provide further $\cdots$ justify the final choice of $p_{\theta 1}$.

Thanks for your valuable question. Due to the new regulation this year, we are not allowed to provide any visual contents in rebuttal. However, we are happy to elaborate a more in-depth discussion of $p_{\theta 1}$ and $p_{\theta 2}$, in Equation (14) and Equation (15), respectively.

In brief, $p_{\theta 2}$ is a simplification of $p_{\theta 1}$, focusing solely on the newly unmasked tokens and ignoring others. While $p_{\theta 1}$ offers greater theoretical rigor (real transition probability) and lower bias in theory, we found them both effective in our experiments, resulting almost the same GenEval performance, as shown in Table 1 (in the paper).

A slight advantage $p_{\theta 1}$ has over $p_{\theta 2}$ is its robustness to the learning rate. While $p_{\theta 2}$ focuses solely on the newly unmasked tokens, it is more sensitive to the learning rate. The default learning rate we use is 3e-6 for $p_{\theta 1}$, and 2e-6 for $p_{\theta 2}$, respectively.

Thanks for your question. We hope we explain well under the visual content limitation.

Q2: Robustness to Reward Models

The current implementation heavily depends on CLIP $\cdots$ strengthen the generality claim.

Thanks for your valuable suggestion. We have conducted an additional experiment with two more reward models, ImageReward[1] and UnifiedReward[2]. We follow the same setting in default Mask-GRPO and the results on GenEval are shown below:

Table 8. Performence of Various Reward Models.

As shown, both ImageReward and UnifiedReward significantly outperform the baseline (by 32% and 23%, respectively), demonstrating the adaptability and effectiveness of Mask-GRPO across different reward models. Importantly, these results were achieved without any fine-tuning of hyperparameters, indicating potential for even higher performance with model-specific tuning.

We appreciate your suggestion and will include these results in the revised manuscript.

Q3: Generalization to Other MGMs

The paper uses Show-o $\cdots$ specific architecture or training schedule.

Thanks for your question. Actually, the decoding strategy of our base mdoel, Show-o, keeps the same with MaskGit. Moreover, they share a similary model architecture in terms of image generation. Therefore, we tend to believe that our Mask-GRPO can also work well in other MGMs like MaskGIT.

Furthermore, we are trying to apply Mask-GRPO on Meissonic. However, due to limited time and computational resource, we have not finished it yet. We promise we will keep updating it. Thanks for your understanding.

Thanks for your question.

Q4: Sample Efficiency and Compute Costs

The method introduces strategies like iteration reduction $\cdots$ how much performance was sacrificed?

Thanks for your valuable questions. To be brief, the unmasking reduction strategy achieves convergence using only 20% of the samples, resulting in a 25% performance improvement on GenEval while reducing training costs by 60%–80%; while sample filtering does not aim to reduce the training cost, instead it stablizes the training and gains a improved performance, as shown in Figure 4 (in the paper)

We begin with a brief recap of the definition of our unmasking reduction strategy: it reduces the total number of unmasking iterations during training (in our experiment, from $T$ = 50 to $T$ = 10), while maintaining the full unmasking schedule during evaluation. Importantly, in GRPO-based training (our method), one sample does not correspond to a single prompt or image prompt or one image, but rather to a single unmasking iteration, since the loss is computed iteratively at each step (see Equation (10)). Therefore, by applying the unmasking reduction strategy, we reduce both the number of unmasking iterations and the sample count to just 20%. Despite using only 20% of the samples, this strategy delivers a 25% performance gain onGenEval. Moreover, it leads to a 60%~80% reduction in training cost. To be more specific, it reduces the training hours from 230 A100 GPU hours to 80 hours in our experiments.

For sample filtering, we discard low-variance sample groups to better highlight relative advantages $A$. This enhances learning signal and stability, ultimately leading to improved performance.

Thanks for your question.

Q5: Theoretical Justification or Guarantees

Although the GRPO framework $\cdots$ expected to perform better than PPO or DPO in MGMs?

Thanks for your insightful questions. The theoretical advantages of GRPO over PPO primarily stem from reduced bias and improved training stability, as discussed in [3,4]. These properties are particularly valuable in the masked generative modeling (MGM) setting, where multi-step iterative generation introduces long-term credit assignment challenges.

We appreciate your interest in the theoretical underpinnings and plan to explore more rigorous justifications in future work.

Reference

[1] Xu, Jiazheng, et al. "Imagereward: Learning and evaluating human preferences for text-to-image generation." Advances in Neural Information Processing Systems 36 (2023): 15903-15935.

[2] Wang, Yibin, et al. "Unified reward model for multimodal understanding and generation." arXiv preprint arXiv:2503.05236 (2025).

[3] Yue, Yu, et al. "Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks." arXiv preprint arXiv:2504.05118 (2025).

[4] Yu, Qiying, et al. "Dapo: An open-source llm reinforcement learning system at scale." arXiv preprint arXiv:2503.14476 (2025).

Dear Reviewer 5UPe,

We have completed the experiment on Meissonic [1], and would like to share some updates related to the experiment as well as other aspects of our work.

Update on Q3

We are pleased to report that we have applied Mask-GRPO to Meissonic [1] and evaluated it on GenEval benchmark. The results are summarized in the table below:

Table 9. Mask-GRPO's GenEval Performance on Meissonic[1].

Note that we experimented with three different reward models, all of which led to improvements over the baseline Meissonic [1], achieving up to a 33% increase in overall performance. In all experiments, we used a learning rate of $3e-6$ and set the iteration steps to 64. These results further demonstrate Mask-GRPO's generalizability across different MGM architectures.

Update on Limitations

Thank you for your thoughtful feedback. We will incorporate these into the Limitations section. For the misalignments with human judgment, we will integrate this with our response to Q2 (Robustness to Reward Models). For the potential reward hacking, we will elaborate a deeper discussion based on what we have discussed in l.292.

Thank you again for your time and constructive review. We look forward to your further feedback!

Reference

[1] Bai, Jinbin, et al. "Meissonic: Revitalizing masked generative transformers for efficient high-resolution text-to-image synthesis." The Thirteenth International Conference on Learning Representations. 2024.

[2] Xu, Jiazheng, et al. "Imagereward: Learning and evaluating human preferences for text-to-image generation." Advances in Neural Information Processing Systems 36 (2023): 15903-15935.

[3] Wang, Yibin, et al. "Unified reward model for multimodal understanding and generation." arXiv preprint arXiv:2503.05236 (2025).

Dear Reviewer 5UPe,

We would like to sincerely thank you once again for your valuable feedback, insightful comments, and the time and effort you’ve dedicated to reviewing our work. Your constructive suggestions have been immensely helpful in improving our paper.

As the discussion period is coming to a close in about 24 hours, we wanted to kindly check if our response has fully addressed your concerns. If there are any remaining questions or points for discussion, we would be more than happy to engage further.

If our response has resolved your concerns, we would be truly grateful if you might consider adjusting your score. Your support would mean a great deal to us!

Limitations

The discussion of potential limitations, such as biases inherited from reward models like CLIP or risks of reward hacking during long training, could be more explicitly elaborated to acknowledge societal and ethical considerations.

Thank you for raising this important point. We have expanded the discussion of limitations in the 'Conclusion and Futhur Work' Section. Due to the regulation, we are not allowed to upload an updated manuscript in Openreview, but we we will incorporate these revisions in the next version. Here is the detail.

• Before:

While achieving superior results, Mask-GRPO’s potential is still not fully revealed. On the one hand, applying such methods on larger base models to explore possible scaling-up will be interesting work. On the other hand, Mask-GRPO’s potential on video generation is valuable, while text-to-video generation is always more complicated as it has to generate coherent outputs that are both contextually and temporally consistent.

• After:

While achieving superior results, Mask-GRPO’s potential is still not fully revealed. First, unlike RL post-training in LLMs, applying RL to T2I generation relies on effective reward models to evaluate generated images. In this work, we use CLIP as the reward model, which proves effective. However, like other reward models (e.g., ImageReward, UnifiedReward), CLIP is still narrow in scope and not accurate enough for further improvements in RL post-training, especially considering the societal biases inherited from the pre-training data. Developing more robust and fair reward models, or jointly training them during RL, may help mitigate these issues. Second, although our out-of-domain results (on GenEval and MSCOCO) suggests that Mask-GRPO successfully mitigates reward hacking, ensuring long-term resistance to such behavior remains an open challenge. Over extended training, models may learn to exploit weaknesses in the reward function, potentially leading to degraded or unsafe outputs. Incorporating human feedback or adversarial training may be necessary to guard against this risk. Third, applying Mask-GRPO to larger base models to explore possible scaling-up will be an interesting work, as RL post-training may benefit from improved model capacity. Finally, Mask-GRPO also holds potential for video generation, where the challenge lies in maintaining both temporal coherence and semantic consistency — a more complex objective than in still-image generation.

We believe this revision better acknowledges the societal and ethical considerations, especially regarding reward models and reward hacking. Thank you for helping us improve this important section.

Conclusion

Thank you once again for your time, constructive feedback, and encouragement throughout the review process. Your feedback and engagement has significantly improved the clarity and quality of our work. We hope our responses have addressed all your concerns, and we welcome any further discussion or questions you may have.

If all your questions have been satisfactorily addressed, we would be deeply grateful if you could consider increasing your score to reflect a more positive assessment. Your support would be a tremendous encouragement to us.

Thank you!

Dear Reviewer 5UPe,

Thank you again for reading our rebuttal and for the valuable suggestions. We noticed that you have submitted final score. We would like to kindly ask whether our rebuttal has resolved your concerns and whether there are any remaining issues you would like us to clarify. We sincerely hope that you will consider both our rebuttal content and our original paper. We would appreciate any additional feedback.

Dear reviewer Ydx6,

Thank you for your comprehensive review of our paper. We greatly appreciate your recognition of the clear motivation and strong empirical results of our method. Please find our detailed responses to your concerns below.

Q1: More Details on Equation (14)

Can you offer more details on equation (14)?

Thank you for your question. We would like to offer a more in-depth explanation of Equation (14), in conjunction with Figure 2 (in our paper). In brief, Equation (14) computes the sum of probabilities over all possible sampling paths that result in the same transition. The first term of the right-hand side represents the product of probabilities for newly unmasked patches (tokens), while the second term corresponds to the product of probabilities for newly remasked patches.

Let us first revisit the definition. In Figure 2, we compute the transition probability for iteration $t = 0$, which is $p_\theta (s_{1} | s_0, a_0)$ in Equation (14). For simplicity, assume the image consists of only 4 patches (index from 1 to 4), and let $i$ denote the patch index. Also assume that the visual codebook size is 4, with token ids from 1 to 4, denoted by $k$. We define $p_t^i$ as the softmax distribution at index $i$ at iteration $t$, and $(p_t^i)^k$ as the probability of sampling token id $k$ at index $i$. At $t = 0$, the initial state $s_0$ is fully masked, and only one patch needs to be unmasked.

As shown in Figure 4, assume the softmax distributions are:

• $p_0^1$ = $[0.1, 0.2, 0.6, 0.1]^T$

• $p_0^2$ = $[0.5, 0.2, 0.1, 0.2]^T$

• $p_0^3$ = $[0.3, 0.4, 0.2, 0.1]^T$

• $p_0^4$ = $[0.15, 0.2, 0.15, 0.5]^T$.

The key insight is that as long as the patch $i = 1$ samples token $k = 3$ with probability 0.6, regardless of what other patches sample, the resulting transition is always the same - first patch remains unmasked while the others are remasked. This is because when patch $i = 1$ samples token $k = 3$, its confidence score $cs_0^1$ = 0.6 is the highest among all $cs_0^i$, regardless of the other samples. According to Equation (5), this leads to the following Choose and Move (CaM) operations:

• $CaM_0^1 = 1$

• $CaM_0^2$ = $CaM_0^3$ = $CaM_0^4 = 0$

In other words, the first patch is unmasked while the others are remasked.

Returning to Equation (14), we compute the $p_\theta (s_{1} | s_0, a_0)$ based on Figure 2. Here we have:

• $Y_0^{CaM}$ = {1}, which means the newly unmasked patch index with $CaM_0^i = 1$

• $Y_0^M \backslash Y_0^{CaM}$ = {2, 3, 4}, which means the newly remasked patch index with $CaM_0^i = 0$

Then the first term of Equation (14) is simply $cs_0^1$ = 0.6, representing the 'probability product of newly unmasked patches'. For the second term, we can rewrite it as $\prod_{i = 2, 3, 4} (\sum_{k} (p_0^i)^k)$, where k needs to be $(p_0^i)^k < min (cs_0) = cs_0^1$ = 0.6. However all $k = 1 , 2, 3, 4$ meet this requirment, no matter what $i$ is. Therefore, the second terms equals to $(0.5+ 0.2 + 0.1 + 0.1) \times (0.3 + 0.4 + 0.2 + 0.1) \times (0.15 + 0.2 + 0.15 + 0.5) = 1$, which is the 'probability product of newly remasked patches'. Putting it all together we have $p_\theta (s_{1} | s_0, a_0) = 0.6 \times 1 = 0.6$, which is obvious illustrated in Figure 2.

To summrize, Equation (14) accurately computes the real transition probability, which AR-style (see Equation (13)) cannot capture. It explicitly distinguishes between newly unmasked and newly remasked patches and calculates their probabilities accordingly.

We sincerely appreciate your question and apologize for the lack of detail of Equation (14). These clarifications will be incorporated into the revised version.

Q2: Other Benchmarks

Can you provide more evaluations on T2I tasks other than Geneval?

Thank you for this valuable suggestion. We have conducted additional experiments on WISE[1], The results are as follows:

Table 7. Model Comparison on WISE[1].

As shown, Mask-GRPO achieves substantial gains in WISE, with up to 29% improvement, demonstrating the generalizability of our method beyond GenEval.

However, due to time and computational constraints, we have not yet completed the evaluation on DPG-Bench. We will update the results in the second phase of the rebuttal. Thank you for your understanding.

We will include these new benchmarks in the revised version of our paper. Thank you again for the helpful suggestion.

W1: Font size

The font size in the figures $\cdots$ match the font size of the main text.

Thank you. We will revise all figures to ensure the font sizes are consistent with the main text in the updated manuscript.

W4: Spelling Errors

There are some spelling errors $\cdots$ in line 212.

Thank you for pointing this out. We will thoroughly proofread the manuscript and correct all typographical and grammatical issues. We sincerely apologize for any inconvenience this may have caused in your reading experience.

References

[1] Niu, Yuwei, et al. "Wise: A world knowledge-informed semantic evaluation for text-to-image generation." arXiv preprint arXiv:2503.07265 (2025).

Update on Q2

Dear reviewer Ydx6,

We are pleased to report that we have completed the evaluation of Mask-GRPO on DPG-Bench[2], and the results are summarized in the table below:

Table 8. Model Comparison on DPG-Bench[2].

As shown in Table 8, integrating Mask-GRPO with the base Show-o model leads to a significant performance gain, improving the overall score on DPG-Bench from 67.48 to 85.63 — a 27% increase. Notably, our method outperforms many other state-of-the-art approaches, demonstrating its strong and consistent effectiveness. We hope this additional evidence clarifies the strength of our approach.

Thank you once again for your time and thoughtul review. We look forward to hearing from you!

Dear Reviewer Vae1,

Thank you very much for your comprehensive and detailed review of our paper, as well as for recognizing the novelty and effectiveness of our work. Please find our point-by-point responses below.

Q1: Sample Filter

Are there any regularities in the types of prompts $\cdots$ Is it repeatedly resampled?

Thank you for your detailed question. Yes, if a sample group is rejected, it will repeatdly resample itself until it either passes the varience threshold or reaches the maximum number of attempts (set to 3 in our experiments to balance computation costs).

In our experiments, we observed that certain prompts consistently fail to meet the variance threshold within the allowed number of attempts. Below are several representative examples:

• '6ad74e6708fc4 Dresses For 14 Year Girls Online Shopping | Dresses For 14 Year ...'
• 'Halloween-Images-For-Kids-Clip-Art.png'
• 'Tags: K-Pop, Kim Chung-ha, Serious, Make Up, Lemon, Blue Background, Nail Polish, Fruits, Yellow Outfit, Yellow Dress, Blooming Blue'
• '6947 Cooper Point Rd NW, Olympia, WA 98502 (#1486235) :: Platinum Real Estate Partners'
• 'How to beat travel stress on family holidays'
• 'Round Trivet - #25 - Tree of Life (3 colour option)'
• 'Beautiful 1970s boho lace maxi dress'
• 'How to Determine Your Body Type | Jalisa's Fashion Files'
• 'Palm tree clip art'
• 'Watercolor Holiday Characters - image 4 - student project'
• '2018 Transit 150 Low Roof, Empty Cargo Van #T80223 - photo 5'

We found that these prompts share some common characteristics:

• Non-Descriptive or Unstructured Prompts: Many appear to be taken directly from web page titles or article headers (e.g., ’6ad74e6708fc4 Dresses For 14 Year Girls Online Shopping | Dresses For 14 Year ...‘), often including irrelevant or noisy content, and lacking clear visual instruction.

• Lack of Visual Specificity and High Ambiguity: Some prompts are abstract or stylistically vague. They do not clearly specify the visual information, which causes high ambiguity. For instance, 'How to beat travel stress on family holidays' is conceptually broad with no clear visual target; 'Palm tree clip art' suggests a style but does not describe the image clearly; and 'Halloween-Images-For-Kids-Clip-Art.png' leaves many details undefined (e.g., number of characters, composition, setting).

To summarize, these 'hard-pass' prompts lack of specific visual guidance, which leads to high ambiguity and frequent rejection by our variance-based filter.

Thank you again for the question. We hope we clarify the details well.

Q2: CLIP Model

What mechanism do $\cdots$ only relying on CLIP as the reward model?

Thank you for this insightful question. It is well known that the CLIP model has limited ability in counting and attribute binding. However, we believe the success of our method stems from the design of GRPO, where we do not rely on absolute correctness, but rather on relative correctness within a sample group.

In GRPO-based methods (ours), we do not use raw rewards from the reward model directly. Instead, we transform them to group-level normalized advantage $A_t^j$ (see Equation (12)), which is effectively comparing the relative correctness among samples within a group [1]. Therefore, even if CLIP model cannot provide an accurate 'absolute correctness' for counting or attribute correctness, as long as it capture the relative trend, it can still drive the model to improve.

To verify this, we conducted an additional experiment. We randomly selected 10 GenEval-counting-style prompts (e.g. 'a photo of $N$ items') and generated 10 correct images accordingly. We then modified each prompt to 'a photo of $\frac{N}{2}$ items' and 'a photo of $2N$ items', while keeping the originally generated (correct) images. We computed the CLIP scores for each prompt-image pair using our reward model. The results are as follows:

Table 5. Additional Experiments on Relative Correctness

As shown, CLIP consistently assigns higher scores to prompts matching the correct count, demonstrating that it can indeed capture 'relative correctness'. This justifies the use of CLIP within our GRPO framework and explains why Mask-GRPO can improve performance in counting and attribute-binding tasks.

We appreciate your valuable question and hope this additional experiment provides further clarity.

Q3: Text Prompts of Figure 5

Could the authors provide the text prompts $\cdots$ the comment about diversity l.299.

Thank you for your careful suggestion. Below are the text prompts corresponding to each image in Figure 5 (each line, from left to right):

Line 1:

• Create an image of a cat as a gardener, wearing a straw hat, gardening gloves, and surrounded by colorful flowers

• Anime illustration of Princess Mononoke from Studio Ghibli, by artgerm, stunning artwork

• Futuristic cyberpunk city at night, neon lights, high-tech car, vibrant colors, cinematic lighting, highly detailed, sci-fi atmosphere, 8k resolution, unreal engine

• A whimsical pink cloud-shaped building with minimalist windows and doors, floating above a vibrant blue sky with cotton-like clouds, Studio Ghibli-style animation movie texture

Line 2:

• A white puppy sitting playfully in autumn leaves, surrounded by fallen red apples, soft natural lighting

• Heroic elf warrior, golden glowing background, detailed fantasy armor, cinematic lighting, epic fantasy art, high detail

• Vibrant city skyline during sunset, modern skyscrapers, colorful abstract style, warm gradient sky, digital art, urban landscape, vivid colors

• Mystical forest with glowing mushrooms and a babbling brook

Line 3:

• Futuristic metallic humanoid robot, highly detailed face, sci-fi background, cinematic lighting, dystopian cityscape, 4K resolution

• A cute duck wearing a chef uniform covered in cookie batter, unreal engine render 8k

• A realistic Venusaur animal among the trees, forest lake, moss, cold weather, dark teal and amber, Sony A7 IV

• 4d photographic image of full body image of a cute little chibi boy realistic, vivid colors octane render trending on artstation, artistic photography, photorealistic concept art, soft natural volumetric cinematic perfect light, UHD no background

We sincerely apologize for the omission and will include these prompts in the revised version.

W1: Unclear Presentation

I find the paper not very clearly presented $\cdots$ Clarifying this distinction would avoid confusion.

Thanks for your suggestion. We agree that the phrasing in l.37 may be misleading. We will revise this sentence to clarify the distinction and avoid potential confusion. We sincerely apologize for any inconvenience caused.

W2: Qualitative Presentation

The paper claims that low reward variance $\cdots$ showing sets of images of different variance.

Thank you for this valuable suggestion. We agree that visualizing image sets with different reward variances would better convey our ideas - particularly in combination with the discussion in Q2. We will include such examples in the revised version.

W5: Typos

There seems to be quite a few typos $\cdots$ inside a sentence.

Thank you for pointing this out. We will thoroughly proofread the manuscript and correct all typographical and grammatical errors in the revision. We sincerely apologize for any disruption this may have caused to your reading experience.

References

[1] Guo, Daya, et al. "Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning." arXiv preprint arXiv:2501.12948 (2025).

I thank the authors for providing the requested details. I find them very insightful and hope they can be included in the paper or in the appendix.

The CLIP model analysis is particularly nice, although N/2 and 2N might be a bit dramatic. A larger experimental campaign, possibly in future work, could verify if the preference provided by CLIP is statistically meaningful for smaller changes in numbers of objects.

I would recommend the authors to provide more details about how they intend to clarify the wording, so that we can have more confidence that in will indeed be fixed.

In any case, the rebuttal gives food for thoughts and I will reconsider the paper in light of these new elements.

Typos

We sincerely apologize for the inconvenience caused by the typos in our submission. After carefully reviewing the manuscript, we have corrected the following issues. Due to the regulation, we are not allowed to upload an updated manuscript in Openreview, but we we will incorporate these revisions in the next version.

• l.79

Before: $\cdots$ first to in-troduce $\cdots$

After: $\cdots$ first to introduce $\cdots$

• l.196

Before: $\cdots$ sampling the $k$-th at position $i$ at iteration $t$

After: $\cdots$ sampling the $k$-th token at position $i$ at iteration $t$

• l.229

Before: $\cdots$ during training, e.g., from $T=50$ to $T=10$. while keeping $\cdots$

After: $\cdots$ during training (e.g., from $T=50$ to $T=10$), while keeping $\cdots$

• l.296

Before: $\cdots$ are presneted in $\cdots$

After: $\cdots$ are presented in $\cdots$

• l.212

Before: $\cdots$ effectively setting$\beta$=0 $\cdots$

After: $\cdots$ effectively setting $\beta$=0 $\cdots$

• l.235 : Added the figure for reward instability.

• l.522 : Corrected reference formatting.

• Figure 1 : Adjusted figure size and font size to match the main text.

• Figure 4: Converted the figure content into a table, as detailed in our response to Reviewer QEdg (Q4).

We once again apologize for these errors and greatly appreciate your attention to detail.

CLIP Experiment

A larger experimental campaign, possibly in future work $\cdots$ in numbers of objects.

Thank you for acknowledging our additional CLIP experiments and for your insightful suggestion. We will incorporate these into our paper.

We are indeed exploring this line of work about reward models. Unlike RL post-training in LLMs, applying RL to text-to-image generation requires effective reward models for evaluating generated images. However, current options (e.g., CLIP, HPS, UnifiedReward $\cdots$) are either too narrow in scope or insufficiently accurate. Therefore, currently we are focusing on jointly training the reward model during RL post-training, rather than treating it as a frozen evaluator. We are excited to further explore this promising avenue and hope to report meaningful progress in future work.

Conclusion

Once again, thank you sincerely for your time, constructive feedback, and encouragement throughout the review process. Your suggestions have been truly insightful and have helped us improve our work significantly. We hope that our clarifications have addressed your concerns. We would be very grateful if you would consider increasing your support for our paper — your support would be a great motivation for our continued research.

Thank you!

Dear Reviewer Vae1,

Thank you very much for your time and effort in reviewing our paper. We are glad that you found our rebuttal useful, and we sincerely appreciate your further suggestions. Please find our detailed responses regarding the wording, typos, and CLIP experiments below.

Wording

I would recommend the authors to provide more details about how they intend to clarify the wording $\cdots$ indeed be fixed.

Thank you for your valuable suggestion. We realized that much of the confusion in our presentation stemmed from the ambiguous use of the term 'newly predicted tokens' (e.g., l.21). To improve clarity, we have revised our manuscript as follows. Due to the regulation, we are not allowed to upload an updated manuscript in Openreview, but we we will incorporate these revisions in the next version.

• l.21-l.24

Before: they predict all masked tokens simultaneously in parallel at each iteration, defined as newly predicted tokens, but only keep the most confident ones, defined as newly unmasked tokens. The rest of the newly predicted tokens will be re-masked for the next iteration, which we define as newly re-masked tokens.

After: they predict all masked tokens simultaneously in parallel at each iteration, but only keep the most confident ones, defined as newly unmasked tokens. The rest will be remasked for the next iteration, which we define as newly remasked tokens.

• l.37-l.39

Before: Since MGMs predict all unmasked tokens in parallel, a naive solution is to use the joint probability of all newly predicted tokens at each iteration as the transition probability $\cdots$

After: Since MGMs predict all masked tokens in parallel, a naive solution is to use the product of their probabilities at each iteration as the transition probability $\cdots$

• l.47-l.51

Before: Building on this, we propose two candidate definitions: (1) the joint probability over all newly unmasked tokens at each iteration. (2) the joint probability over both newly unmasked and remasked tokens $\cdots$

After: Building on this, we propose two candidate definitions: (1) the product of probability over all newly unmasked tokens at each iteration. (2) the product of probability over both newly unmasked and remasked tokens $\cdots$

• l.104-l.106

Before: At each iteration t, the model predicts all tokens in $Y^M_t$ simultaneously, which we defined as newly predicted tokens in Section 1, and moves a subset of them, $\cdots$

After: At each iteration t, the model predicts all masked tokens in $Y^M_t$ simultaneously and moves a subset of them, $\cdots$

• l.173-l.174

Before: The newly predicted tokens (those with $CaM_t^i = 1$ in Equation (5)), play the most critical role $\cdots$

After: The newly unmasked tokens (those with $CaM_t^i = 1$ in Equation (5)), play the most critical role $\cdots$

• l.190-l.192

Before: The sampling results for the newly predicted tokens $Y_t^{CaM}$ remain the same. The remaining tokens in $Y_t^M \backslash Y_t^{CaM}$, which will be remasked, have lower confidence scores $cs_t^i$ than any of newly predicted tokens in $Y_t^{CaM}$.

After: The sampling for the newly unmasked tokens $Y_t^{CaM}$ remain the same. The remaining tokens in $Y_t^M \backslash Y_t^{CaM}$, which will be remasked, have lower confidence scores $cs_t^i$ than any of newly unmasked tokens in $Y_t^{CaM}$.

• l.200-l.201

Before: $\cdots$ focusing solely on the newly predicted tokens $Y_t^{CaM}$ and ignoring others.

After: $\cdots$ focusing solely on the newly unmasked tokens $Y_t^{CaM}$ and ignoring others.

We hope that these wording revisions help clarify our presentation by removing the ambiguity surrounding the term 'newly predicted tokens'.

Once again, thank you for pointing this out.

Dear Reviewer Vae1,

We would like to sincerely thank you once again for your valuable feedback, insightful comments, and the time and effort you’ve dedicated to reviewing our work. Your constructive suggestions have been immensely helpful in improving our paper.

As the discussion period is coming to a close in about 24 hours, we wanted to kindly check if our response has fully addressed your concerns. If there are any remaining questions or points for discussion, we would be more than happy to engage further.

If our response has resolved your concerns, we would be truly grateful if you might consider increasing your support. Of course, we fully respect your decision either way and deeply appreciate your thoughtful review. Your support would mean a great deal to us!

Dear Reviewer QEdg,

We sincerely appreciate your thoughtful feedback and encouraging recognition of our novel idea and well-supported methodology. Your careful reading and insightful questions have been invaluable. Below we address each of your concerns in detail.

Q1: Table Statistics Error

The paper claims a 38% $\cdots$ be incorrect?

Thank you for your careful review. The reported 38% improvement in GenEval performance is corret, as shown in Figure 4. However, we sincerely apologize for an error in Table 2 of the original submission. Specifically, we mistakenly reported an incorrect baseline score for Show-o. The correct GenEval score of Show-o, as observed in our experiments, is 0.53, and our method achieves 0.73, resulting in the 38% improvement. Below is the corrected Table 2:

Table 2. Model Comparison on Geneval. Results are from [1] or their original papers.

Note that our experimental result is consistent with [1]. Similar variations are also found in other text-to-image (T2I) works[2,3]. We apologize for any confusion this may have caused and will revise Table 2 accordingly in the updated manuscript.

Q2: Reducing Strategy

In the ablation study $\cdots$ effective or meaningful?

Thank you for raising this point. The GenEval scores for the two reduction strategies are:

• Computation Reduction Strategy: 0.56
• Unmasking Reduction Strategy: 0.66

Both outperform the baseline (0.53), although they are below the Mask-GRPO (0.73). However, we emphasize that these strategies, especially the unmasking reduction strategy, are effective and meaningful due to better sample effiency and lower training cost. To be brief, the unmasking reduction strategy achieves convergence using only 20% of the samples, resulting in a 25% performance improvement on GenEval while reducing training costs by 60%–80%.

We begin with a brief recap of the definition of our unmasking reduction strategy: it reduces the total number of unmasking iterations during training (in our experiment, from $T$ = 50 to $T$ = 10), while maintaining the full unmasking schedule during evaluation. Importantly, in GRPO-based training (our method), one sample does not correspond to a single prompt or image prompt or one image, but rather to a single unmasking iteration, since the loss is computed iteratively at each step (see Equation (10)). Therefore, by applying the unmasking reduction strategy, we reduce both the number of unmasking iterations and the sample count to just 20%. Despite using only 20% of the samples, this strategy delivers a 25% performance gain onGenEval. Moreover, it leads to a 60%~80% reduction in training cost. For these reasons, we conclude that the unmasking reduction strategy offers superior sample efficiency and is indeed effective.

The computation reduction strategy also reduces training cost by computing loss over only a subset of the total iterations. However, the unmasking conduction strategy is more effective than the computation reduction strategy, as detailed in Section 4.3.

In conclusion, both strategies provide meaningful trade-offs between efficiency and performance, especially in online reinforcement learning (RL) settings。

We appreciate your insightful question and hope these clarifications help.

Q3: Category Discrepancy

Among the various evaluation $\cdots$ discrepancy be explained?

Thank you for pointing this out. First of all, we apologize again for the errors in Table 2. After correcting the baseline statistics (Q1), we confirm that our method improves GenEval performance across all categories, including colors. Please find the updated results below:

Table 2. Model Comparison on Geneval. Results are from [1] or their original paper.

However, we acknowledge that the improvement in the 'Colors' category is relatively smaller, while our reward model, clip model, has a strong ability to perceive color. We attribute this to the limitation of the base model, which is the nature of RL-based post training.

Mask-GRPO is a RL-based post-training method in T2I generation. As discussed in recent works [4,5,6] on large language models, the power of RL-based post training is limited by the base model. RL is improving the performance by unlocking the potenial of the base model, 'teaching' it to 'climb higher' to where it is capable but does not know how to reach. However, RL cannot create new capabilities beyond the base model’s inherent capacity. Therefore, we believe the smaller gains in 'Colors' category reflect a performance ceiling of the base model in handling color semantics, which our method can optimize but not fundamentally surpass. We notice that [3] has similary Geneval results after applying RL to the same base model, which supports our explainations.

Thank you again for your question. We hope we address the concern.

Q4: Typos and Figure 4

There are some typos $\cdots$ be more effective.

Thank you for the suggestion. We will thoroughly review and correct all typos in the revision to ensure clarity and readability. We sincerely apologize for any inconvenience caused to your reading due to typos.

Regarding Figure 4, we agree that presenting the results as a table improves clarity. We propose the following revised table:

Table 4. Ablation performence on Geneval.

References

[1] Yan, Zhiyuan, et al. "Gpt-imgeval: A comprehensive benchmark for diagnosing gpt4o in image generation." arXiv preprint arXiv:2504.02782 (2025).

[2] Guo, Ziyu, et al. "Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step." arXiv preprint arXiv:2501.13926 (2025).

[3] Liu, Jie, et al. "Flow-grpo: Training flow matching models via online rl." arXiv preprint arXiv:2505.05470 (2025).

[4] Gandhi, Kanishk, et al. "Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars." arXiv preprint arXiv:2503.01307 (2025).

[5] Yue, Yang, et al. "Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?." arXiv preprint arXiv:2504.13837 (2025).

[6] Shah, Darsh J., et al. "Rethinking reflection in pre-training." arXiv preprint arXiv:2504.04022 (2025).

Dear Reviewer QEdg,

We would like to clarify a small error in our response to Q3. The reference in the sentence 'We notice that [3] has similary Geneval results after applying RL to the same base model' should be [2], not [3]. We sincerely apologize for any confusion this may have caused.

Thank you once again for your time and thoughtful review. We look forward to hearing from you!

Dear Reviewer QEdg,

Thank you for your valuable time and continued engagement with our work. We are pleased to hear that our previous response addressed your questions, and we sincerely appreciate you taking the extra step to provide suggestions on presentation. We have carefully addressed the point you raised. Please find the detailed revisions below.

Typos

We sincerely apologize for the oversights in our initial submission and thank you for your meticulous review. We have corrected all identified issues. While the regulations prevent us from uploading an updated manuscript in Openreview, we assure you that all changes will be incorporated into the next version.

• l.79

Before: $\cdots$ first to in-troduce $\cdots$

After: $\cdots$ first to introduce $\cdots$

• l.196

Before: $\cdots$ sampling the $k$-th at position $i$ at iteration $t$

After: $\cdots$ sampling the $k$-th token at position $i$ at iteration $t$

• l.229

Before: $\cdots$ during training, e.g., from $T=50$ to $T=10$. while keeping $\cdots$

After: $\cdots$ during training (e.g., from $T=50$ to $T=10$), while keeping $\cdots$

• l.296

Before: $\cdots$ are presneted in $\cdots$

After: $\cdots$ are presented in $\cdots$

• l.212

Before: $\cdots$ effectively setting$\beta$=0 $\cdots$

After: $\cdots$ effectively setting $\beta$=0 $\cdots$

• l.235 : Added the figure for reward instability.

• l.522 : Corrected reference formatting.

• Figure 1 : Adjusted figure size and font size to match the main text.

• Figure 4: Converted the figure content into a table, as detailed in our response to Q4.

We are grateful for your attention to detail, which has helped us improve the quality and clarity of our manuscript.

Conclusion

Once again, thank you sincerely for your time, constructive feedback, and encouragement throughout the review process. Your insightful suggestions have been invaluable in strengthening our manuscript. We hope our responses and revisions have fully addressed your concerns. If we have have successfully done so, we would be very grateful if you would consider increasing your support in your final evaluation — Your support would be a tremendous encouragement to us.

Thank you!

Dear Reviewer QEdg,

Thank you once again for your response, your continued engagement, and your commitment to helping improve our paper. We truly appreciate the time and effort you have dedicated to reviewing our work! Until now it seems like we have addressed all concerns, and we will carefully clarify all of them in the revised version.

Dear Reviewer QEdg,

We would like to sincerely thank you once again for your valuable feedback, insightful comments, and the time and effort you’ve dedicated to reviewing our work. Your constructive suggestions have been immensely helpful in improving our paper.

As the discussion period is coming to a close in about 24 hours, we wanted to kindly check if our response has fully addressed your concerns. If there are any remaining questions or points for discussion, we would be more than happy to engage further.

If our response has resolved your concerns, we would be truly grateful if you might consider increasing your support. Of course, we fully respect your decision either way and deeply appreciate your thoughtful review. Your support would mean a great deal to us!