CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-step

Thank you for your thoughtful and encouraging feedback. We have clarified the CoT scope, addressed technical details, and discussed limitations as suggested.

Weakness

Weakness 1: The method addresses only spatial disambiguation rather than broader forms of reasoning (e.g., temporal, causal, or functional). The "CoT" framing may be misleading as it only affects layout.

Answer: Thank you for your insightful comment. We framed our method as "CoT-lized" because it adopts the step-by-step reasoning paradigm from LLMs, using an iterative "plan-evaluate-refine" loop to guide generation. This describes the process of our method, not just its application domain. Furthermore, our framework can indeed handle broader inferential tasks, leveraging the MLLM's knowledge. For instance, given the prompt "Einstein's favorite musical instrument", our MLLM reasons that it is a violin and rewrites the prompt into a generatable form like "a beautiful violin." This demonstrating its capacity for factual and conceptual reasoning. We will clarify this point in our revised manuscript. Thank you for your valuable feedback.

Weakness 2: This paper lacks of user study or human evaluation, despite the task involving semantics and perception. The evaluation relies solely on LLM scorer and visual examples.

Answer: Thank you for the valuable suggestion. In response, we have conducted a human evaluation to assess our method's semantic and perceptual performance. We asked six human evaluators to rate images generated for 200 complex prompts from 3DSceneBench. They scored our method (CoT-Diff) against four key baselines on a 1-5 scale across two criteria: 3D Layout Accuracy and Visual Quality.

As shown in Table, CoT-Diff significantly outperforming all baselines. Notably, our method achieves the highest scores in both Layout Accuracy (4.08) and Visual Quality (4.12), demonstrating a unique ability to enhance complex spatial control without the degradation in image fidelity seen in methods like Image-CoT.

Weakness 3: Some details are missing in formulation. For instance, vt in Eq. 4 is undefined, which is important. Moreover, the loss optimization procedure is vaguely described.

Answer: We thank the reviewer for pointing out these omissions and apologize for the lack of clarity.

To address your first point, v_t in Equation (4) represents the velocity field predicted by our FLUX model at timestep t. Formally, $v_t = v_Θ(z_t, t, C)$, where $C$ denotes the combined conditional inputs (e.g., depth map and text embeddings) and $Θ$ represents the model's parameters.

Regarding the loss optimization procedure, we train our two LoRA modules (Semantic and Depth) independently using a flow-matching loss. The specific loss functions are as follows:

1. For the Semantic LoRA module (L_SL): $L_{SL} = E_{t,z,\epsilon}[||v_{\theta}(z, t, P, P_1, ..., P_k) - u_t(z|\epsilon)||^2].$ Here, $v_θ$ is the velocity field predicted by the Semantic LoRA-augmented model, $P, P1, ..., Pk$ are the global and local prompt embeddings, and $u_t(z|ε)$ is the target velocity field.
2. For the Depth LoRA module (L_DM): $L_{DM} = E_{t,z,\epsilon}[||v_{\Phi}(z, t, C_D, P_{emb}) - u_t(z|\epsilon)||^2].$ Here, $v_Φ$ is the velocity field predicted by the Depth LoRA-augmented model, $C_D$ is the depth condition, and $P_emb$ is the prompt embedding.

We will update Section 3 with these precise formulations to improve clarity and reproducibility.

Weakness 4: FLUX.1-schnell is a powerful backbone. It is unclear whether the baselines use the same backbone or use others, e.g., SDXL. RPG is training-free, which can be applied to the same backbone.

Answer: Thank you for this crucial point on experimental fairness.

You are correct. In our initial experiments, we aimed to showcase each method at its best by using their officially recommended backbones (e.g., RPG with SDXL, EliGen with FLUX.1-dev). However, for the most direct comparison, we highlight our results against EliGen in Tables 2 and 3 of our main text, a training-based method that also utilizes a FLUX backbone. As shown in Tables 2 and 3 of our main text, CoT-Diff already demonstrates a significant performance advantage. This provides compelling evidence that our framework's superiority will hold when compared against a training-free method like RPG on the same architecture.

For the sake of complete academic rigor, we agree that an explicit experiment on the FLUX.1-schnell backbone is essential for isolating the specific advantages of our CoT-Diff framework. Unfortunately, adapting RPG from its native SDXL (U-Net architecture) to FLUX (DiT architecture) is a non-trivial engineering task due to significant architectural disparities. This adaptation is already in progress, but its complexity prevented us from finalizing the results within the brief rebuttal period.

Therefore, we commit to including this crucial experiment in the camera-ready version of the paper. We will adapt RPG to the FLUX.1-schnell backbone for a direct and fair comparison. This will further solidify the superiority of our CoT-Diff framework.

Weakness 5: Qualitative failure in core claim: As seen in Fig. 4 (last column), the image for “pointing at a distant sailboat” fails to depict the pointing action. This suggests the pipeline may not faithfully capture verb-level intent or fine-grained semantics.

Answer: We acknowledge the limitation shown in this example. This is likely due to two factors:

1. Coarse Control Granularity: our method emphasizes controlling the 3D layout of full objects, but the hand is only a partial component of the human body. Guiding such fine-grained regions (e.g., hand gestures) via whole-body 3D bounding boxes is inherently difficult, and precise control at this level remains a challenging open problem;
2. Backbone Limitations: the backbone itself has limited capacity for faithfully capturing verb-level semantics—none of the compared methods successfully rendered the pointing action, indicating a general limitation in current diffusion models for action grounding.

Question

Question 1: Overclaiming CoT: Could the authors clarify in what way their method performs reasoning beyond spatial layout decomposition? Do you claim any general CoT behavior or just layout disambiguation?

Answer: As stated in W1, our main claim is the CoT-style generation paradigm, where MLLMs guide the diffusion process step by step, rather than claiming general-purpose CoT reasoning.

Question 2: Backbone bias: Have you tested the method on a weaker or standard backbone like SDXL?

Answer: Thank you for this question regarding the generalizability.

Our initial experiments used FLUX.1-schnell to show our framework's maximum potential. We agree that testing on SDXL is crucial for demonstrating generalizability.

However, implementing and training our entire framework on SDXL is a significantly more demanding task than adapting a training-free method like RPG (as discussed in our response to W4). This training process is already underway, but given the substantial computational resources and time required, we were unable to obtain fully converged results and complete the evaluation within the brief rebuttal period.

We note that prior work (e.g., MIGC [1]) has successfully applied semantic and 2D layout control to the SDXL backbone. This suggests that our 3D-aware approach would also be effective on SDXL. We commit to including this analysis in a future version of our paper.

Question 3: Qualitative correctness: Could you discuss failure cases or limitations of action grounding?

Answer: As stated in W5, our pipeline faces challenges in fine-grained action grounding such as "pointing," due to both the coarse 3D control granularity and limitations of the diffusion backbone.

Limitation

Limitation 1: The paper does not explicitly address limitations like over-reliance on 3D layout accuracy, lack of end-to-end differentiability, or societal risks from generation failures (e.g., hallucinations or spatial understanding biases).

Answer: Indeed, our method exhibits a degree of reliance on the accuracy of upstream 3D layouts. If these input 3D layout representations themselves contain errors or lack robustness, the quality and spatial precision of the final generated images may be impacted, even with CoT-Diff's effective utilization. Furthermore, the iterative reasoning process of the MLLM in the current framework is not end-to-end differentiable, which limits the potential for large-scale holistic optimization and deeper learning. More broadly, as the reviewer mentioned, all large generative models face risks of hallucination and bias, and our model is no exception. This could include generating "hallucinated" content inconsistent with semantics, or exhibiting biases in spatial understanding derived from training data.

However, CoT-Diff's strength lies in its explicit 3D layouts and iterative MLLM refinement, allowing for improved spatial understanding and self-correction during generation, thus mitigating spatially related hallucinations and error propagation. Future work will focus on improving the robustness of the 3D layout generation, exploring differentiable reasoning frameworks, and actively addressing biases and societal risks in models.

[1] Zhou, Dewei, et al. "Migc: Multi-instance generation controller for text-to-image synthesis." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024.

Dear Reviewer sj9U,

We sincerely thank for your insightful suggestion (Weakness 4) regarding a direct comparison on the same backbone architecture. And we have conducted a new experiment to address this.

Initially, we committed to adapting RPG to the FLUX backbone. During this process, we discovered that porting RPG from its native U-Net (in SDXL) to the DiT architecture (in FLUX) is non-trivial and results in a significant performance drop. This is likely because RPG's guidance mechanism is highly optimized for U-Net's specific structure.

Therefore, to provide a more rigorous and truly fair evaluation, we pivoted to compare our CoT-Diff against a much stronger and more relevant baseline: RAG-Diffusion [1]. This recently accepted work (ICCV 2025) presents a state-of-the-art training-free method, conceptually similar to RPG, but is explicitly designed for and demonstrates excellent performance on the FLUX.1-dev backbone.

The results are as follows:

As shown, these results offer conclusive evidence that the performance gains originate from our novel CoT-Diff framework—not merely the backbone—by decisively outperforming a more formidable and relevant baseline on the same FLUX architecture, thus fully addressing the reviewer's concern. We will incorporate these findings into our revised manuscript.

We sincerely thank you for your recognition of our work. Based on your suggestions, we have added further analysis, clarified key experimental details, and included comparisons with recent CoT-based methods to strengthen the paper.

Question

Question 1: Is the design of some prompts problematic? For example, in Figure 4 with the prompt, "a car behind a bench on a snowy landscape," the spatial structure is correct in the results from both CoT-Diff and EliGen. The perceived correctness in both is due to different viewing angles, as the prompt only specifies the front-behind relationship without left-right positioning. How should such cases be judged during the actual labeling process, and could this ambiguity affect the other evaluation metrics?

Answer: Thank you for raising this question. In our evaluation, spatial relations such as "behind" are defined with respect to the camera (observer) viewpoint, which provides a consistent and objective reference frame. Without this convention, viewpoint rotation could arbitrarily flip spatial relations (e.g., “behind” becoming “front” or “right”), making evaluation unreliable. This camera-centric assumption is also adopted by established benchmarks such as T2I-CompBench[1].

Question 2: Could the image size have an impact on the results?

Answer: Our current implementation generates images at a resolution of 512×512. Since our method relies on accurate 3D layout modeling, low-resolution settings (e.g., 128×128) may introduce errors in rendered depth, leading to hallucinations. However, we observe that at 512×512 or higher resolutions, the depth maps are sufficiently accurate to guide high-quality image generation.

Limitation

Limitation 1 a): The LC method incorporates layout guidance, yet it performs worse than traditional diffusion models in the reported metrics. A more detailed analysis is needed to explain this counterintuitive result.

Answer: The seemingly counterintuitive performance of LC arises from several limitations in its design:

1. Backbone Choice: LC uses Stable Diffusion 1.5 as its backbone, which has relatively limited image quality and spatial understanding capabilities compared to more advanced models (e.g., FLUX). This constrains the upper bound of its spatial reasoning performance.

2. Conditioning Mechanism: Although LC accepts 3D layout input, it does so through a sparse-depth LoRA module trained on top of a ControlNet pretrained for dense depth. The sparse layout signal is thus weakly injected into the model through an indirect path (LoRA → ControlNet → SD), which reduces its effectiveness in enforcing correct spatial structure. Moreover, this training approach is also known to be challenging to converge and can even degrade the backbone's inherent generative capabilities.

3. Lack of Semantic Disentanglement: LC lacks a semantic module like the Semantic LoRA in CoT-Diff, making it more prone to object confusion when generating scenes with multiple entities. For example, in Fig. 4 (column 2), both the car and the bench are generated as benches, failing to represent the intended spatial relationship (“car behind a bench”).

Together, these factors explain why LC, despite incorporating layout input, underperforms in spatially grounded generation tasks.

Limitation 1 b): The results analysis lacks fine-grained interpretation. For example, in the ablation study, the "Front" category shows a performance drop after adding the "+Semantic Layout" module. The authors should briefly explain this decline.

Answer: Thank you for your meticulous review of our ablation study. You have correctly identified the slight performance dip in the "Front" category after adding "+Semantic Layout," which is an excellent question. We believe this phenomenon is not an anomaly and can be attributed to the following factors:

1. Conflict between Semantic and Geometric Cues: The Semantic Layout module injects entity-level semantic information into specific image regions (as described in lines 143–144). However, in the absence of corresponding depth maps to enforce geometric structure, this strong semantic bias may conflict with the model’s pre-existing handling of simple spatial relations such as “front.” This can temporarily interfere with the model’s generation patterns and lead to slight performance drops.

2. Importance of Depth Information: This decline further confirms the necessity and effectiveness of adding the Depth Map module. As shown in Table 4, once depth information is introduced on top of the semantic layout, the model not only resolves occlusions and object ordering issues but also significantly improves performance in the “Front” category (from 44.5 to 51.3). This highlights the complementary nature of semantic and geometric guidance: while semantics guide content, depth provides spatial anchoring in 3D space.

3. The characteristics of the dataset: Our training data is constructed based on annotations from EliGen, and a similar pattern was observed in EliGen's results (as seen in Table 2 of our main text), where front/behind performance also dipped when only semantic layouts were introduced. Therefore, the decline we observe may be partially inherited from the limitations of the source dataset.

Limitation 1 c): The paper lacks an analysis of how many inference steps are needed to produce spatially coherent images, which is important given that high inference cost is a stated limitation.

Answer: We evaluates the performance with a fixed number of MLLM optimization steps at varying total steps. It can be observed that as the number of steps increases, the system's performance progressively improves, tending to converge after 3 steps.

Accordingly, we designed an adaptive early-stopping strategy: once the model determines the layout is correct, it ceases further evaluation, with a maximum of 5 evaluation steps. This strategy balances performance and time overhead, achieving considerable performance with an average of 2.9 steps.

Limitation 1 d): Recent works have also explored applying CoT to image generation. The paper should cite and compare with these methods, clarifying the key differences and advantages of the proposed approach.

Answer: Thank you for bringing these relevant recent works to our attention. We have studied them and added experimental comparisons.

Key Differences and Advantages: The core difference between our CoT-Diff and these methods lies in the fundamental mechanism of the "Chain-of-Thought." Other methods (e.g., Image-CoT[2], T2I-R1[3]) implement CoT at the semantic or token level, refining text prompts or internal features iteratively. In contrast, CoT-Diff introduces explicit 3D spatial reasoning, using an MLLM to generate and refine a physically interpretable 3D layout, which serves as precise guidance for the diffusion model. This structured approach excels at handling complex spatial relationships and ambiguities.

Experimental Comparison: We evaluated these methods on our 3DSceneBench benchmark. The results clearly show CoT-Diff's superior performance in spatial reasoning, particularly in challenging categories like "Front/Back Left/Right," "Multi Relation," and "Complex."

We will include these works in our "Related Work" section and add the comparison to the "Experiments" section in the final version to demonstrate the novelty and effectiveness of our method.

[1] Huang, Kaiyi, et al. "T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation." Advances in Neural Information Processing Systems 36 (2023): 78723-78747.

[2] Guo, Ziyu, et al. "Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step." arXiv preprint arXiv:2501.13926 (2025).

[3] Jiang, Dongzhi, et al. "T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot." arXiv preprint arXiv:2505.00703 (2025).

We sincerely thank you for the valuable feedback. Our main contribution is a CoT-lized framework that activates the powerful reasoning abilities of MLLMs to guide diffusion, effectively addressing the limitations of generative models in spatial reasoning and the rigidity of one-time layout planning.

Weakness

Weakness 1: Lack of comparison with unified understanding-and-generation models.

Answer: We have added comparisons with Janus-Pro[1], VILA-U[2], Image-CoT[3], and T2I-R1[4]. The results demonstrate CoT-Diff’s superior performance in spatial reasoning and text-image alignment, suggesting that current unified generation-understanding models still lack explicit reasoning capabilities compared to our CoT-lized framework.

Weakness 2: As the authors discussed, CoT-Diff will cause high computational overhead: using GPT-4o or Gemini 2.5 Pro at each step is time-consuming.

We thank the reviewer for this crucial point on computational overhead. We acknowledge that using a powerful MLLM introduces a time cost. To provide full transparency, we have conducted a detailed quantitative analysis.

1. Comparison with Existing Methods First, we compare CoT-Diff with both layout-based and other CoT-based methods. The baseline methods FLUX, RPG, and Eligen use 50 inference steps, and our CoT-Diff use 20 steps. The search number of Image-CoT is 20.

The data shows that while CoT-Diff's Feedback Time adds to the overhead compared to simpler layout-based methods, its total inference time is comparable to other CoT-based frameworks like Image-CoT. This suggests the observed overhead is characteristic of the complexity required for step-by-step generation.

2. Adaptive Strategy for Efficiency More importantly, we have designed an adaptive early-stopping strategy to explicitly manage this cost without sacrificing quality. The MLLM guidance is not fixed but applied dynamically until the layout is deemed correct (up to a maximum of 5 steps). The table below shows how runtime scales with the number of MLLM feedback steps:

Our experiments show that this adaptive strategy achieves considerable performance in just 2.9 steps on average, striking an effective balance between generation quality and computational cost.

Weakness 3: The pipeline relies on intermediate images for feedback, but no visualizations of these images. The early steps are usually very blurry, leaving it unclear how useful they are for the MLLM.

Answer: We use FLUX-schnell as the diffusion backbone, which is a distilled version of FLUX-dev. Thanks to its improved denoising capability, even the early-stage intermediate images are relatively clean and informative. As shown in Figure 2 (Image 1 and Image 2), the visual quality at each step is sufficient for the MLLM to perform reliable evaluations and provide meaningful feedback.

Weakness 4: The proposed condition-aware attention mechanism closely resembles the one in EliGen, yet this similarity is not discussed.

Answer: We thank the reviewer for pointing out the similarity between CoT-Diff's condition-aware attention mechanism and EliGen. We acknowledge that both methods indeed share common ground in their core mechanisms: both leverage global and local textual prompts to guide image content generation, and both employ similar isolation strategies by masking regions corresponding to different prompts (e.g., P1 and P2) during attention calculation to prevent semantic interference.

However, CoT-Diff extends upon this foundation in two key aspects, endowing it with more powerful conditional control capabilities:

1. Introduction of Multimodal Conditions: Beyond textual prompts (such as p, p1, p2), CoT-Diff additionally incorporates a Depth Condition as a novel geometric input. This allows the model to simultaneously utilize both the semantic information from text and the precise spatial-geometric information from depth maps, enabling richer and more refined conditional control.

2. Extended Attention Mechanism for Heterogeneous Condition Fusion: To effectively fuse these two inherently heterogeneous conditional information types—text and depth—and to prevent potential conflicts and interference between them, we specifically extended the attention mechanism. By applying custom-designed attention masks, we ensure that information flow between different modal conditions (i.e., textual conditions and depth conditions) remains independent, thereby guaranteeing the precision and autonomy of each guiding information type.

Question

Question 1: How does performance change if the custom attention mask is omitted?

Answer: We have demonstrated the performance of the model without optimization. It can be observed that omitting the custom attention mask leads to a significant performance degradation in dimensions other than "front" and "behind". This is because the absence of the mask causes conditions from different objects to act on the same region, leading to issues such as attribute leakage and confusion.

Question 2: Section 4.5 mentions runtime efficiency comparison, but the table lacks these numbers; how does CoT-Diff’s inference time compare to other methods?

Answer: As stated in W2, we adopt a quantitative comparison. The result shows CoT-Diff’s runtime is acceptable when compared against layout-based methods and is comparable to other CoT-based frameworks.

[1] Chen, Xiaokang, et al. "Janus-pro: Unified multimodal understanding and generation with data and model scaling." arXiv preprint arXiv:2501.17811 (2025).

[2] Wu, Yecheng, et al. "Vila-u: a unified foundation model integrating visual understanding and generation." arXiv preprint arXiv:2409.04429 (2024).

[3] Guo, Ziyu, et al. "Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step." arXiv preprint arXiv:2501.13926 (2025).

[4] Jiang, Dongzhi, et al. "T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot." arXiv preprint arXiv:2505.00703 (2025).

Dear Reviewer UhfL,

We would like to sincerely thank you again for your insightful review comments.

To address the main concerns you raised, we have provided a detailed response. Specifically, we have now included comprehensive comparisons with state-of-the-art unified models, conducted a thorough analysis of the computational overhead, and further clarified the novelty of our condition-aware attention mechanism. We hope these additions and clarifications have resolved your questions.

As the author-reviewer discussion period is ending soon, we would be very grateful if you could take a moment to look over our rebuttal. Please let us know if our responses have adequately addressed your concerns.

Thank you again for your time and valuable feedback.

We sincerely thank you for your positive and encouraging feedback. We are glad that you recognize the novelty of our CoT-lized Diffusion framework and its contribution to integrating MLLM-driven reasoning into the diffusion process. Based on your suggestions, we have incorporated several additional experiments into the paper to further showcase the effectiveness of our approach.

Weakness

Weakness 1: The framework's main drawback is its significant computational overhead. The core mechanism requires invoking a powerful MLLM at each denoising step to evaluate the image and refine the layout.

Answer: We thank the reviewer for their concerns regarding computational overhead. CoT-Diff's MLLM is not invoked at every denoising step; its primary cost is concentrated in the initial "thinking" phase for layout refinement.

Inspired by this, we designed an adaptive early-stopping strategy: the MLLM ceases evaluation once the layout is deemed correct (max 5 steps), averaging just 2.9 steps. This effectively balances performance and efficiency, as shown in the second time cost table.

We argue this front-loaded computational cost is a reasonable trade-off for the significant improvements in generation quality and spatial accuracy. CoT-Diff's value in layout-critical applications like game asset creation, virtual staging, and film pre-visualization justifies this expense.

Weakness 2: The system's "reasoning" capability is entirely outsourced to a large, proprietary MLLM.

Answer: We acknowledge CoT-Diff's reliance on external MLLMs, potentially impacting reproducibility. However, our core contribution lies in designing a novel framework that effectively bridges MLLM's high-level cognitive reasoning with diffusion models' pixel synthesis. Our key innovation demonstrates how to activate and dynamically inject powerful spatial and logical reasoning from external MLLMs into the diffusion process in a step-by-step, corrective manner, representing a significant advance in controllable generation. Furthermore, our framework is model-agnostic, and while we showcased maximum potential with a proprietary model, CoT-Diff can readily adapt to powerful open-source MLLMs (see Q2 response), solidifying its reproducibility.

Weakness 3: Lacking a sufficiently analysis for the system's failure modes or its robustness to such errors.

Answer: While CoT-Diff operates as a multi-step cascade with theoretical error propagation risks, its core iterative feedback mechanism is inherently designed for self-correction.

Even if an error occurs at step t, the MLLM re-evaluates and corrects it at step t+1. A prime example is Figure 2 in our main paper, where the MLLM's initial flawed layout (Image 1) is detected and rectified in the subsequent loop, leading to a corrected generation (Image 2) and a successful final output. The system is designed to recover from its own intermediate mistakes.

We provide a detailed qualitative analysis and case study in our Q3 response explicitly demonstrating the system's recovery process, and experimental results empirically validate that increasing MLLM evaluation and correction steps consistently improves performance. This indicates CoT-Diff is not a brittle cascade but a robust mechanism leveraging its iterative nature for error correction and progressive refinement.

Question

Question 1: Could you quantify CoT-Diff's inference time and computational cost against baselines (e.g., FLUX, EliGen)? Have you explored the trade-off between performance and efficiency by reducing MLLM feedback frequency, and is continuous feedback essential?

Answer: We will include a detailed quantitative comparison of inference time and cost against baselines in the revised manuscript. The results are presented in the table below. Our results show that while CoT-Diff introduces overhead, our adaptive strategy keeps it manageable. The baseline methods FLUX, RPG, and Eligen use 50 inference steps, and our CoT-Diff use 20 steps. The search number of Image-CoT is 20.

We believe that continuous feedback, especially early on, is crucial for timely and accurate layout correction. This is because the initial denoising steps establish the foundational structure of the image, where any modifications have a far more significant and decisive impact on the final output. A sparse feedback schedule (e.g., every N steps) risks missing critical windows for correction, potentially leading to errors that cannot be fixed in later stages. Therefore, our adaptive approach, which focuses intense guidance at the beginning and stops when the layout is stable, strikes a vital balance between ensuring high-quality results and managing computational cost.

Question 2: How does CoT-Diff perform when paired with smaller, open-source MLLMs (e.g., Qwen family)?

Answer: We thank the reviewer for this important question regarding the framework's robustness with different MLLMs. We compared CoT-Diff's performance when guided by proprietary Gemini-2.5-Pro versus open-source Qwen2.5-VL-72B-Instruct, with results summarized below:

The results clearly indicate that the framework's performance degrades gracefully when switched to the open-source MLLM; critically, the system does not collapse and still outperforms other baseline methods. This demonstrates our framework's inherent value and robustness, supporting our W2 response that our approach is not fundamentally tied to a single proprietary MLLM and ensures reproducibility.

Question 3: Could you provide a qualitative analysis of the system's failure modes?

Answer:

We thank the reviewer for their question on system failure modes. CoT-Diff's core strength lies in its inherent self-correction capability.

1. Qualitative Analysis: Our framework is inherently designed to recover from initial errors. Figure 2 in the main text serves as a prime case study of this self-correction mechanism.

   In that example:

   (1) The MLLM initially generates a flawed 3D layout, misinterpreting the spatial relationship.

   (2) This leads to an incorrect intermediate generation (Image 1), where the relationship between woman and desk is wrong.

   (3) Crucially, in the next feedback loop, the MLLM assesses Image 1, detects the discrepancy between the generated image and the text prompt, and rectifies its own initial layout plan.

   (4) This corrected layout then guides the diffusion model to produce a spatially accurate intermediate image (Image 2), leading to a successful final output (Image N).

While the system is robust, it can occasionally fail if the MLLM repeatedly makes incorrect assessments or if the diffusion model fails to follow the guidance. We will add a detailed qualitative analysis of such failure cases to the revised manuscript.

2. Quantitative Validation: To validate this, we evaluated performance at varying optimization steps. As the table below shows, performance consistently improves with more steps, saturating around 3 steps.

This trend indicates that the iterative process is effective at correcting errors and that the system reliably recovers and improves with more "thinking" time.