LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integration

General Responds: We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged that multiple reviewers found the paper well-structured, technically sound, and easy to follow (9VLN, f8N8, ytVC). Reviewer ytVC noted that the proposed framework is "well-designed and conceptually sound" and appreciated the "novel and thoughtfully constructed" nature of our approach, recognizing that it demonstrates "meaningful contributions." Reviewer zjmb similarly highlighted the generality of our framework, praising the "good layout planning and reasoning" and "flexible conditioning and efficient attention mechanism" in our integration model. Reviewers also acknowledged the good performance on T2I-CompBench (9VLN) and the practical applications of our batch editing and collage editing features (f8N8). We are pleased that these aspects of our work were well-received, and we address the remaining concerns with detailed clarifications in the following responses.

W1: Regarding Dependency on GPT and FLUX.1 dev

A: While we leverage GPT-4o and FLUX.1 dev as foundation models, our contributions lie in the novel architectural design and integration methodology. The whole design of LayerCraft system and the parameter reuse dual-LoRA architecture of OIN should be highlighted. Notably, LayerCraft outperforms all compared methods, including GPT-4o, both quantitatively and qualitatively.

W2: Pixel Format without Layer Separation

A: We respectfully clarify that LayerCraft actually maintains full layer separation throughout the generation process. Our intermediate representations include Layout, reference images, and the intermediate results at each layer, this is precisely what enables flexible post-generation editing.

Q1: Impact of CoT on Inference Time

A: The CoT reasoning adds approximately 80% to inference time compared to direct generation. However, this overhead is justified by substantial quality improvements that the spatial accuracy improves from 0.2831 to 0.6432 (127% improvement). Moreover, our overall runtime remains faster than that of the GPT-4o API, which typically requires several minutes for comparable generation and editing tasks.

Q2: Why Not Use Original FLUX-Fill Model

A: Our OIN addresses fundamental limitations that FLUX fill cannot overcome. FLUX infill does not support reference image processing, which is crucial for subject-driven inpainting. Our dual-LoRA architecture introduces a new approach: LoRA weights handle conditions while maintaining FLUX's generation quality, enabling consistent style in both generation and editing. OIN also outperforms other concurrent methods (see Figure 6 in the supplementary materials).

Q3: How soon can native GPT catch up to paper's result quality? If in a short time, what's the impact of this work?

A: We agree that generative LLMs like GPT-4o are rapidly improving, and it is possible that future versions may natively support more controllable layout-to-image generation. However, LayerCraft is designed as a modular framework that leverages LLM capabilities rather than replacing them. Our work demonstrates how LLMs can be orchestrated with visual modules to achieve structured, editable, and interpretable generation. These capabilities remain challenging for end-to-end diffusion models or monolithic GPT pipelines.

General Responds: We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged that multiple reviewers found the paper well-structured, technically sound, and easy to follow (9VLN, f8N8, ytVC). Reviewer ytVC noted that the proposed framework is "well-designed and conceptually sound" and appreciated the "novel and thoughtfully constructed" nature of our approach, recognizing that it demonstrates "meaningful contributions." Reviewer zjmb similarly highlighted the generality of our framework, praising the "good layout planning and reasoning" and "flexible conditioning and efficient attention mechanism" in our integration model. Reviewers also acknowledged the good performance on T2I-CompBench (9VLN) and the practical applications of our batch editing and collage editing features (f8N8). We are pleased that these aspects of our work were well-received, and we address the remaining concerns with detailed clarifications in the following responses.

W1:Compare CoT-enabled ChainArchitect to vanilla GPT-4o prompting without reasoning steps

A: Our ablation study (Table 2) provides direct comparison with simpler strategies. The configuration without CoT (last row) effectively approximates vanilla GPT-4o prompting, showing spatial accuracy degradation from 0.6432 to 0.2831. This substantial gap validates our CoT approach.

W2: No Hard Spatial Relations

A: We acknowledge that including more explicit examples of occlusion and 3D viewpoint shifts would strengthen the paper. While we cannot add new results during the rebuttal phase, we kindly refer reviewers to the following existing results: the photo collage in Figure 1, Figure 5 in the main paper, and Figure 2 in the supplementary materials, which showcase instances of cloth deformation and 3D viewpoint variation. Regarding occlusion, our framework handles it naturally through the use of overlapping bounding boxes and sequential object injection during composition. We will include more comprehensive and focused examples in the camera-ready version.

W3: Similarities with LLM-Blueprint

A: We appreciate the pointer to LLM Blueprint [1]. While both works use LLMs for layout generation, LayerCraft differs in three key ways: (1) We employ modular agents with role-specific prompts and memory, enabling more interpretable and debuggable scene planning. (2) We integrate layer-wise reasoning and spatial anchoring based on background cues, which is not present in LLM Blueprint and eliminates the cascading errors due to "generate-then-edit" pipeline. (3) Our pipeline supports batch editing with global consistency thanks to our Intermediate Representations and Object-Integration Network (OIN), which is beyond the scope of LLM Blueprint. We will include a detailed discussion in related work.

W4: Lack of Human Evaluation and Multi-turn Editing Results

A:

User Study

Given the limited time for rebuttal, we conducted a small-scale user study (20 participants) to evaluate interactive complex prompt generation. A larger-scale study will be included in the camera-ready version. In this preliminary study, we assessed the generation quality of five systems: LayerCraft, GPT-4o, LLM-Blueprint, FLUX.1 Dev, and GenArtist, using 15 challenging prompts. A total of 20 participants rated each generated image on a five-point Likert scale (1 = poor, 5 = excellent) across four criteria: (1) prompt consistency, (2) naturalness, (3) visual appeal (color, composition, style), and (4) overall quality.

As shown in the table below, LayerCraft achieved the highest average scores in three out of four criteria. In contrast, LLM-Blueprint, FLUX.1 Dev, and GenArtist showed noticeably lower scores across all categories, especially in prompt consistency and visual appeal, indicating limitations in structured layout generation or visual coherence. These results demonstrate that LayerCraft not only matches GPT-4o in generation quality but also surpasses it in prompt fidelity and naturalness, reinforcing the value of our structured, multi-agent pipeline for complex, multi-turn editing scenarios.

Mean (μ) and standard deviation (σ) of user ratings (1–5) for each criterion. Higher values indicate better performance.

Additionally, we conducted a quantitative evaluation on multi-turn editing using 1/5 of the MagicBrush benchmark, where the ground-truth(GT) target exists. For the comparisons, we copied the reported results from GenArtist. L1 and L2 are for measuring the average pixel-level difference between the edited image and GT targets. CLIP-I and Dino measures the cosine similarity between the image embeddings of GT and generated results. CLIP-T demonstrates the cosine similarity results between text and results. The results are shown below:

Multi-turn quantitative comparison on MagicBrush. Lower is better for L1/L2; higher is better for CLIP-I, DINO, and CLIP-T.

LayerCraft achieves the best performance across most metrics, including CLIP-I, DINO, and CLIP-T, indicating stronger visual-textual alignment and perceptual similarity. While GenArtist shows slightly better L1 and L2 scores, our method excels in semantic alignment, which is more critical for realistic, context-aware edits.

W5: Failure Cases

A: Across our experiments, we did not observe notable failure cases in terms of layout accuracy, spatial reasoning, and generation quality. However, one limitation we did observe is that the scale of inserted objects can vary depending on the background image. This variation arises because the ChainArchitect is designed to inject objects in a contextually natural way, but the prompts or input images often lack explicit size specifications for the target objects.

Q1: Using any other LLMs

A: Due to the limited time available during the rebuttal phase, we were unable to conduct extensive experiments with additional LLMs. However, we tested Claude Sonnet 3.7 and Gemini 2.5 and observed minimal performance differences in most cases. We also experimented with Gemma-3 and LLaVA, whose results were comparable. Notably, we found that once the prompt is decomposed into structured representations, the performance bottleneck is no longer tied to model size. This aligns with recent findings, e.g. "Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Model Parameters, ICLR 2025", which highlights that test-time compute and inference-time reasoning are more critical to leveraging pre-trained LLMs effectively. We will include the additional experiments and findings in the camera-ready version.

General Responds: We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged that multiple reviewers found the paper well-structured, technically sound, and easy to follow (9VLN, f8N8, ytVC). Reviewer ytVC noted that the proposed framework is "well-designed and conceptually sound" and appreciated the "novel and thoughtfully constructed" nature of our approach, recognizing that it demonstrates "meaningful contributions." Reviewer zjmb similarly highlighted the generality of our framework, praising the "good layout planning and reasoning" and "flexible conditioning and efficient attention mechanism" in our integration model. Reviewers also acknowledged the good performance on T2I-CompBench (9VLN) and the practical applications of our batch editing and collage editing features (f8N8). We are pleased that these aspects of our work were well-received, and we address the remaining concerns with detailed clarifications in the following responses.

W1: Incremental novelty of core components.

A: While LayerCraft uses established tools, its core innovation lies in how these components are combined to enable structured, controllable generation. And we respectfully disagree with the characterization of our contributions as merely incremental. Specifically:

• Our Object Integration Network (OIN) uses dual-LoRA modules not for weight adaptation like conventional LoRA finetuning, but for conditional understanding, allowing multiple control while preserving the original generative ability. Additionally, our parallel attention streams with attention score mixing mitigate the quadratic memory cost typically caused by long control sequences. This design significantly outperforms concurrent methods, as shown in Supplementary Figures 4 and 5, which compare our object injection results against a naive extension from OmniControl, ACE++, and EasyControl.
• Our Layer-wise Progressive Generation avoids error cascades common in generate-then-edit pipelines.
• We introduce background-guided layout planning and 3D-aware scene graphs to enhance spatial reasoning and editing precision. In contrast, previous work arranges layouts without considering the content or perspective of the background image.
• Our intermediate reference images enable consistent batch collage editing. This is an important capability for collage editing where GPT-4o and other existing methods fall short (see Figure 5 and Supplementary Figure 6).

W2: Lack of interactivity evaluation

A: Given the limited time for rebuttal, we conducted a small-scale user study (20 participants) to evaluate complex prompt generation. A larger-scale study including detailed user interaction study will be included in the camera-ready version. In this preliminary study, we assessed the generation quality of five systems: LayerCraft, GPT-4o, LLM-Blueprint, FLUX.1 Dev, and GenArtist, using 15 challenging prompts. A total of 20 participants rated each generated image on a five-point Likert scale (1 = poor, 5 = excellent) across four criteria: (1) prompt consistency, (2) naturalness, (3) visual appeal (color, composition, style), and (4) overall quality.

As shown in the table below, LayerCraft achieved the highest average scores in three out of four criteria. In contrast, LLM-Blueprint, FLUX.1 Dev, and GenArtist showed noticeably lower scores across all categories, especially in prompt consistency and visual appeal, indicating limitations in structured layout generation or visual coherence. These results demonstrate that LayerCraft not only matches GPT-4o in generation quality but also surpasses it in prompt fidelity and naturalness, reinforcing the value of our structured, multi-agent pipeline for complex, multi-turn editing scenarios.

Mean (μ) and standard deviation (σ) of user ratings (1–5) for each criterion. Higher values indicate better performance.

W3: Complexity and potential for error propagation

A: We incorporate three levels of error mitigation in our pipeline:

• Background Generation Validation: During intial background synthesis, our LayerCraft performs viewpoint suitability assessment to ensure the generated perspective accommodates subsequent object placements. When inadequacies are detected, the system automatically reformulates prompts and regenerates backgrounds.
• Visual-Grounded Layout Generation: Unlike text-only methods that struggle with spatial accuracy, our approach uses background images as visual guides to generate layouts. This grounding makes object placement more precise and predictable, and it also supports flexible adjustments through user and agent interactions.
• Context Handling: LayerCraft stores intermediate images and spatially analyzed layouts in a structured JSON format, which can be retrieved and reused when needed.

Q1: Did you test simpler pipelines like a single LLM agent or GPT-4o? What were the failure modes?

A: Yes, we conducted an ablation study (Table 2) to evaluate simplified versions of our pipeline, including single-agent setups that remove key components of our Chain-of-Thought (CoT) reasoning. Specifically, we tested variants without (i) generation order, (ii) object relationship modeling, (iii) both, and (iv) all CoT reasoning. These simplified pipelines are conceptually aligned with approaches based on a single LLM agent or direct GPT-4o usage.

Our results show that removing CoT-driven reasoning, especially layout ordering and inter-object relationships, leads to significant degradation in spatial coherence, object count accuracy, and visual realism. The full removal of CoT results in the steepest drop in performance, with frequent placement conflicts and incoherent scene compositions.

These failure modes underscore the limitations of simpler pipelines. Without structured, iterative reasoning, such models struggle to handle compositional complexity and fail to support consistent multi-object placement.

Q2: How often does the pipeline fail on complex prompt?

A: Across our experiments, we did not observe notable failure cases in layout accuracy, spatial reasoning, or generation quality. One limitation we did notice is that the scale of inserted objects can vary depending on the background image. This variation arises because ChainArchitect is designed to inject objects in a visually natural way, yet prompts or input images often lack explicit size constraints for target objects. Overall, the system performs robustly across diverse prompts, and most inconsistencies stem from VLM limitations rather than LayerCraft's design.

Q3: How does LayerCraft handle context during long interactions?

A: LayerCraft maintains and updates structured scene layouts and intermediate images. These are serialized and passed between turns, enabling both agents and users to make informed editing decisions over time. During Batch Editing, the ChainArchitect will be reinitialized every time for new images to ensure the max performance.

General Responds: We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged that multiple reviewers found the paper well-structured, technically sound, and easy to follow (9VLN, f8N8, ytVC). Reviewer ytVC noted that the proposed framework is "well-designed and conceptually sound" and appreciated the "novel and thoughtfully constructed" nature of our approach, recognizing that it demonstrates "meaningful contributions." Reviewer zjmb similarly highlighted the generality of our framework, praising the "good layout planning and reasoning" and "flexible conditioning and efficient attention mechanism" in our integration model. Reviewers also acknowledged the good performance on T2I-CompBench (9VLN) and the practical applications of our batch editing and collage editing features (f8N8). We are pleased that these aspects of our work were well-received, and we address the remaining concerns with detailed clarifications in the following responses.

Q1: Evaluation on Broader Benchmarks (GenEval).

A: We now include GenEval results in Table 1. LayerCraft achieves 0.84 overall score, outperforming or matching GPT-4o across most subcategories (e.g., 1.0 on Single Object, 0.94 on Two Objects, 0.89 on Color), while significantly surpassing non-agent methods like Show-O, SDXL, DALL-E 3, and SD3.5. These new results will be included in the camera-ready version.

Comparison on GenEval Results

Q2: Missing Results on Complex T2I-CompBench.

A: We now include Complex T2I-CompBench results in Table 2. LayerCraft scores 0.4631 on the complex subset, the highest among all methods. The result confirms LayerCraft's robustness on multi-object, compositional, and instruction-heavy prompts.

Comparison of complex on T2I-Compbench

Q3: Lack of Comparison with GPT-4o and Other Advanced LLMs.

A: Visual comparisons (Figures 4–6) demonstrate that LayerCraft performs better than GPT-4o in spatial reasoning, controllability, identity preservation, and batch collage consistency. While GPT-4o struggles with maintaining identity, ensuring consistency across images, and handling fine-grained edits, LayerCraft overcomes these challenges through layer-wise control, background-guided spatial reasoning, and intermediate representations. This advantage stems from LayerCraft's unified conditional mechanism, which contrasts with GPT-4o's less integrated structure that can cause mismatches and brittle editing behavior.

Q4: Missing Comparison in Layout Generation with Prior Work.

A: We appreciate the request for layout generation comparisons. Direct quantitative comparison is challenging because existing methods like LLM-Grounded Diffusion do not use background-guided reasoning or support layer-wise generation. Instead, we provide an ablation study: removing our CoT-based layout strategy reduces spatial accuracy from 0.6432 to 0.2831, showing the clear benefit of our approach. These results highlight how our method offers more precise and coherent layouts than conventional layout predictors.