Injecting Vision Language into Autoregressive Image Generation

1. Image Quality and Subject Consistency: As shown in Figure 4, the generated images are not visually appealing, and maintaining consistency between the generated objects and those in the subject images remains challenging.
2. Reliance on BLIP for Text Encoding: In the implementation details and Figure 2, the authors mention that they “utilize the pre-trained FLAN-T5 XL as the text encoder and precompute text embeddings from descriptions generated by BLIP.” However, since BLIP is an older image captioning model, its reliability may be insufficient, potentially limiting the quality of the text-conditioned image generation.
3. Limited Capability in Handling Multiple Objects: The input images used in the experiments contain only a single object, making it unclear how well the model would perform when processing images with multiple objects.

• The proposed method lacks novelty. Using BLIP to generate prompts is common.
• The proposed attention mechanism and classifier-free guidance are commonly used.
• The paper is not well-presented such as an empty line in Line 211, section 3.2 is very confusing, and "Suppl" in Line 361.
• More up-to-date methods should be compared in Table 1.
• Experimental results are very limited.
• The layout of Fig 5 and Fig 6.

• Limited motivation, many current image-generation models, such as ControlNet, already support both image and text as condition.
• Insufficient experiments and analysis. The generation results lack strong metrics and thorough analysis to convincingly support the proposed approach.
• Incomplete comparison with existing methods, please conduct a more comprehensive literature review.

I am not entirely persuaded by some components of the model design.

1. Given the robust visual condition extraction pipeline, I question the necessity of extracting textual conditions for this "image in, image out" task. Specifically, in the visual condition pipeline, there is already a pre-trained VQ model, an object mask, and a trainable MLP, which together provide a strong framework for extracting visual representations. At the very least, I would expect the authors to clarify their motivation for including textual conditions and to present experimental results demonstrating how these conditions contribute.

2. The hierarchical attention mechanism is not explained in sufficient detail. Specifically, it is unclear how $Hier$ is derived, which seems essential to the new hierarchical mechanism. Aside from a brief mention of it being “derived from label-specific descriptions,” there is little explanation. Additionally, $Hier$ is not explicitly shown in Figure 3, which depicts the model pipeline.

3. I think the “hierarchical attention mechanism” is two standard attention mechanisms, with the visual attention values weighted according to their relevance to the textual features of objects. Although the mechanism itself is straightforward, this specific design choice lacks clear motivation. Why not apply the same weighting mechanism to the attention of textual conditions? And why are “the textual features of objects” used as a target to guide $Hier$ (based on my understanding from the dotted line in Figure 3) rather than, for example, “the visual patches after the object masks”?

The experimental results are insufficient to justify the model design, and as a result, many components seem arbitrary.

4. If the authors claim novelty in their textual & visual condition extraction components, I would expect more ablation studies on these. For instance, what if only the visual pipeline is used? What if a pre-trained VAE is used instead of vector quantization?

5. If the authors claim novelty in their hierarchical attention component, I would like to see more ablation studies on that as well. Some examples are mentioned in point 3.