HFDream: Improving 3D Generation via Human-Assisted Multi-view Text-to-Image Models

Dear Reviewer T5e1,

We sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.

---

[W1] Collection cost of the multi-view human feedback dataset.

We compiled a dataset with 5,000 human-labeled annotations by examining up to 10k images, a process that took about 8 hours in labeling. If we had used Amazon Mechanical Turk for this task, the cost would have been roughly \$0.02 per annotation, totaling \\$200. Moreover, considering the importance of a larger and more diverse dataset, acquiring human labels for synthetically generated images is much more feasible than having experts create new, high-quality 3D assets. Collecting human labels for generated images is particularly advantageous because creating new 3D assets, which accurately reflect a given caption, is significantly more difficult and time-consuming than generating images from captions using a text-to-image model.

---

[W2] Challenges in identifying ambiguous view directions.

In practice, it is not problematic to label objects with ambiguous definitions of view directions. Specifically, we instructed the human annotators to skip the images with ambiguous view definitions and to only label the most certain ones. The detailed instructions can be found in Appendix B (Labeling Instruction). We also observed that the objects you referred to, which look the same from different angles, are already handled well by existing text-to-image models. Therefore, we chose to focus on objects with unambiguous view definitions, as the model struggles more with these objects to generate view-aligned images. We believe that human-in-the-loop learning is an effective solution for addressing view-dependent image/3D generation.

---

[W3] Lack of experiments with Stable Diffusion.

The proposed pipeline of fine-tuning text-to-image models with reward models can be used orthogonally with any text-to-image diffusion model. Still, following your suggestion, we included the results for Stable Diffusion in Appendix G. As shown in Figure 17, our method significantly improves view-alignment of Stable Diffusion 2.1, showing the generality of our approach.

---

[Q1] Over-saturation of text-to-image samples generated with HFDream

For your information, we use the guidance scale of 20 for 3D generation (same as DreamFusion-IF), and 7.5 for 2D image generation. We believe that the oversaturation issue is due to the supervised fine-tuning method (as observed in [1, 2]), and it can be alleviated by using sophisticated approaches such as reinforcement learning, e.g., DPOK [2] and DDPO [3]. To support our hypothesis, we additionally conducted an experiment by applying DDPO with our proposed reward models, and found that this approach enhances view-aligned image generation without a degradation in visual quality. The experimental details and qualitative results are shown in Appendix G and Figure 18.

[1] Lee et al., 2023. "Aligning Text-to-Image Models using Human Feedback." arXiv 2023.

[2] Fan et al., 2023. "DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models." NeurIPS 2023.

[3] Black et al., 2023. "Training Diffusion Models with Reinforcement Learning." arXiv 2023.

Dear Reviewer T5e1,

Thank you very much again for your time and efforts in reviewing our paper.

We kindly remind that we have only two days or so in the discussion period.

We just wonder whether there is any further concern and hope to have a chance to respond before the discussion phase ends.

Many thanks,
Authors

Dear Reviewer L7ZF,

We sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.

---

[W1] Limited novelty of the proposed method.

We emphasize that our key contribution is not limited to simply applying a human-in-the-loop framework to text-to-3D generation. Although the overall pipeline is intuitive and straightforward (this is rather the strength of our framework), it poses non-trivial challenges including data collection formulation and algorithmic design choices. To address them, we propose several techniques such as collecting human feedback as directions instead of preferences (Section 3.2) or using categorically normalized reward values as training weights (Section 3.3). Notably, a naive approach such as collecting human preferences similar to various prior works [1,2,3] fails to yield a high-quality reward model, leading to trivial predictions (e.g., consistently assigning low scores for rarer directions such as "back"). We believe that our work provides valuable insights to the research community.

[1] Xu et al., 2023. "ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation". NeurIPS 2023.

[2] Kirstain et al., 2023. "Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation." NeurIPS 2023.

[3] Wu et al., 2023. "Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis." arXiv 2023.

---

[W2] Lack of comparison to SOTA text-to-3D baselines.

We emphasize that our method is compatible with any text-to-3D techniques. We improve the text-to-image model, which serves as a backbone for text-to-3D methods, thereby enhancing the geometric consistency of the generated text-to-3D results. Nonetheless, in response to your suggestion, we evaluate our method in conjunction with Magic3D. Our approach, added to Magic3D (termed Magic3D-HFDream), is compared quantitatively and qualitatively against the default Magic3D-IF-SD setup. As presented in Table 2, we measure CLIP-R precision score and normalized reward score values, finding significant improvement in both metrics. Moreover, we conducted a comparative analysis against several SOTA baselines, including MVDream, ProlificDreamer, Magic3D, TextMesh, and DreamFusion-IF. This comparison was done using demos from the MVDream project website. For instance, in the 'Corgi riding a rocket' example in Figure 13, we observed that while baseline methods compromise either prompt alignment (MVDream, in which the rocket is missing) for geometric consistency or vice versa (DreamFusion-IF, Magic3D-IF-SD, TextMesh, ProlificDreamer), our Magic3D-HFDream demonstrates consistently better results in both aspects. The visual comparison can be found in Appendix E.3, and the rendered video is available in the updated supplementary materials and in this Google Drive link.

---

[W3] Concern about the human-labeled dataset and the generalization of the fine-tuned text-to-image model.

We would like to clarify your potential misunderstanding. Our dataset is distinct from ImageReward-v1.0 and is not a subset of it. We created our dataset by generating images with text-to-image models and then collecting human feedback on valid view direction, as described in Section 3.1. We labeled about 5k images in total.

Regarding generalization, we remark that both the reward function and the fine-tuned text-to-image model are evaluated on unseen text prompts (describing unseen objects). As shown in Figure 6, the reward model shows great reliability for all directions, and also generalizes very well, with minimal accuracy drop even for unseen objects. Table 1 also shows the clear improvements in view-dependent image generation for unseen objects. Additionally, we use LoRA [1] to help retain the generalization of the fine-tuned text-to-image model.

[1] Hu et al., 2021. "LoRA: Low-Rank Adaptation of Large Language Models." ICLR 2022.

---

[W4] Small evaluation prompt set size.

Our main goal is to evaluate the generalization of the text-to-image model systematically. Specifically, the reason we evaluate two different sets of prompts is to verify HFDream generalizes well to prompts with unseen objects. Additionally, we believe that running 244 generations total with 61 different prompts is enough to test the effectiveness of our method, given our goal was to demonstrate that our method can consistently perform reliable generation with each of the random seeds. Nevertheless, following your suggestion, we evaluate HFDream on DreamFusion prompts as well, and present the results in Appendix F. We chose 50 different prompts among the 411 prompts, due to the tight time schedule, on top of additional prompts that were used in MVDream.

Dear Reviewer L7ZF,

Thank you very much again for your time and efforts in reviewing our paper.

We kindly remind that we have only two days or so in the discussion period.

We just wonder whether there is any further concern and hope to have a chance to respond before the discussion phase ends.

Many thanks,
Authors

Dear Reviewer PzSD,

We sincerely appreciate your review with thoughtful comments. We have carefully considered each of your questions and provide detailed responses below. Please let us know if you have any further questions or concerns.

---

[W1] Concerns about motivation.

It is important to note that our framework (i.e., learning from human feedback) is time- and resource-efficient approach for collecting high-quality data compared to alternative approaches; (a) Training a camera-pose aware diffusion model requires 3D asset data which is known to sacrifice prompt fidelity at the cost of geometric consistency, due to distribution shift. (b) Adopting a ControlNet-like approach relies on additional priors and is limited to face landmarks only [1]. Finally, we remark that our method is orthogonal to other approaches and can be used together for even better performance.

[1] Han et al., 2023. "HeadSculpt: Crafting 3D Head Avatars with Text." arXiv 2023.

---

[W2] Unsatisfactory quality of the text-to-3D generation results.

We emphasize that our method is compatible with any text-to-3D techniques since we fine-tune the text-to-image model, which serves as a backbone for text-to-3D methods. Nevertheless, to address quality concerns, we applied our method to the more advanced backbone, Magic3D (termed Magic3D-HFDream), and compared against the default Magic3D (termed Magic3D-IF-SD) by following the suggestions of Reviewer L7ZF. Additional qualitative and quantitative results using Magic3D-HFDream are available in Appendix D of the revised draft. As shown in Table 4, our method consistently improves the CLIP-R precision score and the normalized reward score, and alleviates the Janus problem as shown in Figure 8. We also provide qualitative results using prompts and demos from the MVDream demo website. As shown in Appendix E.3, Figure 13, Figure 14, and Figure 15, the baseline methods either sacrifice prompt alignment (MVDream, in which the rocket is missing) or geometric consistency (DreamFusion-IF, Magic3D-IF, TextMesh, ProlificDreamer), while our result (Magic3D-HFDream) achieves better results in both aspects. The rendered video file is attached in the updated supplementary material and can also be found in the Google Drive link.

---

[W3] Over-saturation and over-smoothness of the text-to-3D generation results.

It is important to note that our main focus is on addressing the multi-face problem, which is the most critical challenge in current text-to-3D generation methods, as highlighted by many prior works [1, 2, 3]. While we agree that over-saturation and over-smoothness are also important issues, we think that these issues are not of our major focus and can be mitigated by combining our method with more advanced methods, such as Magic3D. For example, when combined with Magic3D, our method generates 3D outputs with reduced over-saturation and over-smoothness (see Figure 8, Appendix E.3 and Google Drive link).

[1] Poole et al., 2023. DreamFusion: Text-to-3D using 2D Diffusion.

[2] Shi et al., 2023. MVDream: Multi-view Diffusion for 3D Generation.

[3] Wang et al., 2023. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation.

Dear Reviewer PzSD,

Thank you very much again for your time and efforts in reviewing our paper.

We kindly remind that we have only two days or so in the discussion period.

We just wonder whether there is any further concern and hope to have a chance to respond before the discussion phase ends.

Many thanks,
Authors