We sincerely appreciate your time and effort in reviewing our paper. Below are our point-by-point responses:

1. Details & More experiments about Figure 3 (c): Details: Figure 3(c) is based on the LlamaGen-stage1 model and computed on the full COCO2017-val dataset at a resolution of 256. The CFG scale is set to 7.5, with no top-k or top-p sampling applied. Except for the temperature, all other settings follow the official GitHub repository’s parameters.

For temperature perturbation, as described in the main text, our temperature perturbation strategy involves computing the entropy of each token during generation. Tokens are categorized into five predefined entropy intervals ([0,2), [2,4), [4,6), [6,8), [8,∞)), and for each interval, we apply a specific temperature to adjust the token distribution. The overall image quality is then assessed on the dataset.

More experiments: Additionally, we have added the evaluation metrics for STAR model on COCO2017 (due to limited time, we will provide evaluation on more models and more datasets in the next version), as shown in the table below:

2. Why masking order helps: In mask-based generation models, each step selects which random tokens to keep based on a predefined rule—typically by adding noise to the confidence scores and keeping the top-k tokens (see Equations 4 and 5 in mainpaper). This can be viewed as a confidence-based top-k sampling strategy, where higher-confidence tokens at a given timestep are more likely to be selected, and the noise level controls the randomness.

From our observations, due to the mask scheduler’s design, low-entropy tokens (which tend to have higher confidence) are selected earlier in generation, while many high-entropy tokens are accepted together in later stages (as shown in Figure 4). This can hurt image quality. By adjusting the noise level based on entropy—using lower noise in high-entropy regions and more randomness in low-entropy ones—we encourage more cautious token selection where needed and allow more exploration in simpler areas, which leads to better image quality.

3. Speedup Ratio: Our proposed acceleration strategy reduces inference cost by an additional ~15% on top of existing acceleration method [1], achieving around 2.3× speedup compared to the original model. Although the current performance gain is modest, this approach highlights the potential of exploiting image non-uniformity for faster generation. We are actively investigating ways to further improve its effectiveness.

Reference:

[1] Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding

Thank you for your valuable feedback! Below are our responses to your concerns:

1. Novelty: Thank you for your concern regarding novelty. While entropy has been extensively studied in autoregressive text generation (e.g., EDT [1]), we observe that there are key differences between autoregressive image generation and text generation, which make it inappropriate to directly apply these methods to image generation. In fact, EDT proposes reducing the entropy in low-entropy tokens to sharpen the distribution and increasing randomness in high-entropy regions to improve diversity. In contrast, our work focuses on the sampling strategy for autoregressive image generation and proposes increasing randomness in low-entropy regions, which is opposite to EDT’s approach and brings better image generation quality.

2. Generalizability: Due to resource limitations, it is challenging for us to systematically evaluate a wide range of scenarios. Besides the COCO dataset, we also report results on DPG and HPS—where DPG assesses the model’s ability to follow complex text prompts, and HPS evaluates the aesthetic quality of generated images. These metrics are commonly used to evaluate multiple aspects of image generation models. We believe they provide meaningful insights, and we plan to conduct more comprehensive evaluations in future work.

3. Failure analysis: Due to policy restrictions, we are unable to provide visualizations. In some models, such as Lumina-mGPT, high-entropy regions may actually reflect good diversity, and lowering temperature there can reduce visual quality. Encouraging diversity in low-entropy regions may also occasionally introduce artifacts. We will include further analysis and visual cases in future versions. For a brief discussion of limitations, please refer to Section E.3 in the supplementary material.

4.Theoretical foundation: Thank you for pointing this out. Indeed, the current entropy-based temperature adjustment is empirically designed. We believe this originates from the fact that continuous image information is discretized into token indices, and the information distributions of images and text differ significantly. Since generation quality is highly sensitive to these distributions and the learned distribution is often imperfect, we apply further adjustment based on token-level entropy. As we currently lack a suitable theoretical framework to rigorously analyze this phenomenon, we have instead provided empirical evidence in the main text and hope to establish a stronger theoretical foundation in future work.

5. Statistical significance: Thank you for pointing this out. We found that generation-related metrics are indeed sensitive to random seeds. To further analyze this, we randomly selected 10 seeds from the range $[0, 1e6]$, ran the generation model 10 times conditioned on these seeds, and computed the mean and standard deviation of the results, as shown in the table below, the rand seeds have little impact on the performance improvement brought by our method, further supports the positive impact of the proposed method on performance.

LlamaGen:

Meissonic:

STAR:

Note that due to time constraints, we were unable to complete this analysis for all models on all metrics, but we will include the full results in a future version.

6. Clarification on entropy calculation & Implementation details: The entropy described in Section 3.2 is computed over all spatial tokens—specifically, we calculate the entropy of the probability distribution used when generating each token, without applying any denoising or spatial smoothing strategies. And our work will be open source later.

7. Hybrid AR and continuous models: This is a interesting question. The concept of entropy is specific to discrete token-based generation, where each token has an associated probability. This issue does not naturally arise in continuous generation frameworks.

Of course, we believe there can be some alternative ideas in continuous visual generation. For example, the sampling process in discrete models is somewhat analogous to the noise injection process in continuous methods. In multi-step SDE-based models, it may be possible to apply different noise levels to different regions of the image based on content complexity, which could potentially improve generation quality. However, since entropy is not defined in continuous space, other indicators like variance may be needed to guide such adjustments.

For hybrid discrete-continuous models, it depends on how the two parts are combined. Our method is designed to balance diversity and stability in discrete generation. If the discrete part is responsible for generating fine details (as in models like HART [2]), our approach may still be helpful. Conversely, if the discrete AR component is only used to generate high-level (like BLIP3-o [3]) or abstract information, our method may not be effective in that case. The effect of noise in the latent space on entropy remains unclear and would require further investigation.

Reference

[1] EDT: Improving Large Language Models’ Generation by Entropy-based Dynamic Temperature Sampling

[2] HART: Efficient Visual Generation with Hybrid Autoregressive Transformer

[3] BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset

Thank you for your recognition and correction. Here are my answers to your concerns:

1. Token distribution of speculative strategy: Thanks for pointing out. Our proposed speculative decoding modification does not fully preserve the original model’s token distribution. In fact, we found that appropriately adjusting the token probability distribution of the autoregressive generation model—such as with the entropy-aware temperature method described in our approach—helps produce higher-quality images. Specifically, the scale term in Equation 8 in mainpaper encourages the acceptance of low-entropy tokens by increasing randomness in their distribution, while the random term sharpens the sampling distribution for high-entropy tokens by suppressing excessive randomness.

2. COCO2014val dataset evaluation: Thank you for your comment. Here, we provide FID and CLIPScore results on the COCO2014val dataset, shown in the table below:

Note that * means result from a subset with around 14,000 images.

Due to time constraints, the Lumina-mGPT model has not completed the standard 30k-image inference; we report its metrics on a subset (~14,000 images) of images and commit to including the full COCO-2014 results in a future version.

Additionally, since the Meissonic model’s generated images differ significantly in COCO style, we provide its performance on the MJHQ-30k dataset: our proposed method reduces the FID on MJHQ from 20.50 to 17.96 and improves CLIP-score from 27.33 to 27.61.

3. Entropy and NLP models: Thank you for your insightful comment. While entropy has been extensively studied in the context of NLP, especially in works such as EDT [1] and SED [2], our study focuses on the entropy distribution of tokens and the corresponding sampling strategies in autoregressive visual generation—a perspective that, to the best of our knowledge, has not been thoroughly explored in prior work. For instance, EDT recommends reducing the entropy of low-entropy tokens to sharpen distributions and increasing randomness in high-entropy regions to promote diversity, while SED emphasizes that high-entropy tokens are more likely to lead to confusion and require careful handling. Moreover, several recent GRPO strategies also incorporate entropy-aware mechanisms. We believe that our visual generation setting poses distinct challenges that motivate a different design choice, as discussed in our work.

Reference:

[1] EDT: Improving Large Language Models’ Generation by Entropy-based Dynamic Temperature Sampling

[2] SED: Self-Evaluation Decoding Enhances Large Language Models for Better Generation

Thank you for your rebuttal. This rebuttal has solved most of my problems. Experiments have proven the proposed method works well across multiple baselines. Considering it lacks a theoretical proof, I'll hold a Borderline accept score.

Thanks for your affirmation. I respond to your concerns as follows:

1. Typo in equation: Thank you for your pointing out. We agree that Equation 6 (b) may cause confusion, we correct it here as:

$\text{(Eq. 6a)} \quad T_s = T \cdot \left[1 - \beta \cdot \left(s - \left\lfloor S/2 \right\rfloor\right)\right]$

$\text{(Eq. 6b)} \quad \tilde{p}_s = \frac{p_s}{T_s}$

2. Proposed method on histogram: After applying the proposed entropy-aware sampling method, we observed a more concentrated entropy variance—that is, the distribution of token entropy across the dataset becomes more focused, with fewer extremely high or low-entropy tokens and more tokens falling in the middle range.

While we cannot provide visualizations due to policy constraints, similar examples are shown in Figure 6 of the supplementary material. Compared to the baseline, which tends to generate images with flatter backgrounds and less detailed foregrounds (with entropy concentrated in certain regions), our sampling strategy leads to a more balanced spatial distribution of entropy, resulting in more visually diverse and structured outputs. We will add more related analysis in future versions to further support our observations.

3. What happens in foreground: Our method not only increases randomness in low-entropy regions to promote diversity, but also reduces it in high-entropy regions to ensure stability. According to Equation 2 in the main text:

$\text{(Eq. 2)} \quad T=T_0 e^{-\frac{\epsilon}{\alpha}}+\theta,$

when entropy $\epsilon$ is large, the temperature $T$ approaches $\theta$, which is typically less than 1. This lowers randomness and encourages more confident sampling in complex (foreground) areas. Conversely, when entropy is small, $T \approx T_0 + \theta$, usually greater than 1, introducing more randomness to help generate diverse content in simpler regions. Thus, for these simpler regions, the model’s stochastic exploration is encouraged to enhance the richness of the generated image content; while for more uncertain regions (those with higher entropy), the model adopts a more conservative sampling strategy to ensure stable generation. Together, this approach improves the consistency between image and text while maintaining generation diversity.

4. Comparison with SOTA diffusion methods: Thanks for pointing out, here we provide quantitative results of several diffusion and flow models, including SD v2.1, PixArt-Alpha, SDXL, and SD v3 on FID, CLIP-score (coco validation 2017), DPG and HPS, for comparison, as shown in table below:

I thank the authors for their responses, and based on this additional info, I am happy to inc my score to 5. Thanks for the good work!