1. The paper may not introduce many novel insights compared to previous methods. The dynamic optimization strategy of CompGS and the compositional optimization strategy in GALA3D[1] appear to be similar. Both alternate between composition-level optimization and entity-level optimization. GALA3D optimizes a single entity with a dynamic camera radius of $\frac{3}{4}\left\|\left(h_i, w_i, l_i\right)\right\|_2$ , which has an effect similar to the Volume-adaptively Optimizing Entity-level Gaussians in CompGS. Furthermore, the initialization of 3D Gaussians with 2D compositionality, compared to methods like REPARO[2] and ComboVerse[3], seems to add only an additional step. This step involves using 2D diffusion models to generate a well-composed image from a given prompt and obtaining entity-level prompts via LLM.

2. Semantic drift and pattern leakage: In Figure 5 of the appendix, the chair and hat exhibit leaf-like patterns similar to those of the ficus in a pot.

References:

[1] Zhou, Xiaoyu, et al. "Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting." arXiv preprint arXiv:2402.07207 (2024).

[2] Han, Haonan, et al. "REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment." arXiv preprint arXiv:2405.18525 (2024).

[3] Chen, Yongwei, et al. "Comboverse: Compositional 3d assets creation using spatially-aware diffusion guidance." arXiv preprint arXiv:2403.12409 (2024).

(1) My concern about this paper is its limited novelty. While the results appear promising, I feel this paper may not provide substantial new insights for the readers.

(2) There is a lack of discussion regarding object interactions. In scenes with multiple objects, interactions are inevitable; however, the paper does not address how to manage complex interactions. For instance, if one object is obstructed by another in the image generated by T2I method, how should we effectively initialize and optimize the obstructed object?

(3) Some blurriness is noticeable in the results (for example, "An artistis painting on a blank canvas" in Fig 3, "a student is typing on his laptop" in Fig 4). The image quality falls short compared to other image-to-3D works, such as DreamCraft3D and DreamCraft3D++.

Unclear components

• LLM: Using LLM to provide priors for compositional generation is currently a common approach (the most similar use case to this paper is SceneWiz3D ). However, some details are still desirable. For examples, what is the prompt? What is the usual success rate? Any failure case?
• Editing: Is every component trained or only the added part? If they are all trained, how to keep the existing content unchanged? If only the added part trained, how to make sure it is compatible to existing components (as some editing might change existing part)?

Comparison to Gala3D

• The paper mentioned the code/weight are not available. However, Gala3D does provide the code/weight via application.
• In L1061, the paper states "(Gala3D) fails to incorporate mutual interaction". I have reservations about the statement. For example, their teaser sample where an astronaut sitting on a couch, the interaction between the people and the couch is not trivial.
• One straightforward adaptation and ablation is to replace the LLM-provided 3D bounding boxes with the 2D+img-to-3D bounding boxes, w/ and w/o using the 3D as initialization.

Limitation/Issues of the pipeline

There are some evident limitations not discussed.

• Semantic leak due to 2D model

1. Fig 9, last row: the leaf leaks to the hat, panda, and the couch

• 3D inconsistency and blurry boundary, due to image-to-3D and optimization

1. Fig 9, the rabbit: The blurring stuff under the table. And it looks very 3d inconsistent.
2. Fig 4, top-left: the left-most viewpoint, the boundary between the cake and the gift is blurred. Also, it looks very 3d inconsistent.

• Occlusion due to 2d/3d ambiguity

1. Fig3, last row: what does two silken look like separately?

• Number of objects: The samples provided have number of objects less the five. In comparisons, Gala3D showed quite some samples with a larger amount of objects. Are there any hard constraint preventing the author to try the same prompts?

Contribution:

• Explicit representation: Replacing the representation itself it not a contribution.

1. For example, how would set-the-scene perform if we replace its representation to GS?
2. L162 mentions SceneWiz3D is based on NeRF. However, they use NeRF for background and use meshes for foreground objects.

• local-global optimization: It is not new in composition-aware generation, for example, set-the-scene adopt similar strategy. Some discussions and comparison are desirable.

MISC:

• Runtime in table 2: Are these results calibrates by number of object? Moreover, some of these methods include background, which is definitely an important factor to distinguish object-level and scene-level. This factor should also be considered in the table.

1. Object scale. This paper introduces a reasonable improvement by using rescaling during the entity-stage optimization. However, it does not address the scale relationship of the final generated objects. For instance, in Figure 1, the soccer ball and butterfly appear disproportionately large in the scene. I am curious if this is a bias introduced during training, as such issues do not typically arise in the inialization of text-to-image models. This could indicate a potential limitation in the current approach’s handling of object scaling.
2. Decomposing optimization: The decomposition approach does not consider the relationships between objects. The decompositional text embedding v_l may conflict with the actual 3D scene. For example, in Figure 4, the use of “a girl” as the text prompt to guide the SDS optimization generally results in a standing figure, which does not align with the curled posture of the girl in the scene. This discrepancy might impact the quality of the generated output, as evidenced by the slightly unnatural appearance of the girl in Figure 4.
3. Evaluation metrics: I think that CLIP can only evaluate the alignment between the generated output and the input text, but it does not adequately reflect the quality of the generation. The authors might consider incorporating aesthetic evaluation metrics, e.g., LAION Aesthetics Predictor [1] and PickScore [2], to provide a more comprehensive assessment of the aesthetic quality of the generated results.

[1] https://laion.ai/blog/laion-aesthetics. [2] Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation.

1. The paper lacks experiments that test small objects generation, such as creating a scene with a castle and a figure standing on it, where the model captures fine details in smaller objects.
2. The paper does not address scenarios involving scenes with a high density of entities, such as a garden with hundreds of entities. While generating scenes with only a few entities can be done manually by positioning objects individually, testing whether the model can generate large-scale scenes that align with the text description would better demonstrate its robustness.