Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image

We sincerely thank you for your time and valuable comments, some of which are fundamental and in-depth suggestions that help us greatly improve our paper. To address your concerns, we present the point-to-point response as follows.

Comment 1: Unrigorous technical presentation

We appreciate your observation regarding the clarity of the notations and explanations in Equations (1) and (5). In response to your feedback, we will make the revisions in the updated manuscript to enhance understanding.

For Equation (1), here $d(i,j)$ denotes the value of coordinate (i,j) in the depth map, while $x$ is the integration process along the vertical line $y=j$, $\vec{n}(x)$ denotes the vector of the input normal field at the position $x$. $n_x$ is the component of the normal field $\vec{n}$ along the direction of the x-axis (which is a scalar function). The confusion might arise because the letter $x$ is used in two different contexts. To clarify, we’ll make the following adjustments: We will use the formula $d(i,j)=∑t=0inx(t,j)d(i,j)=∑t=0inx(t,j)$, where $d$ and $n_x$ are considered as matrices in the discrete version. This change aligns better with how the variables are handled in the code.

For Equation (5) (ExplicitTarget). Let $Avg(V, W) = \frac{\sum_i{V_i W_i}}{W_i}$ represent the weighted average function, and $V_{M}(v, i): (\mathbb{N}^+, \mathbb{N}^+) \rightarrow \{0, 1\}$ represent the visibility of vertex $v$ in mesh $M$ under view $i$. $Col_M(v,i)$ Indicate the color of vertex v in viewpoint i. We compute the ExplicitTarget $ET$ of each vertex in mesh $M$ as

$ ET_{M}(v) = \\begin{cases} Avg(Col_M(v,i), V_{M}(v, i) W_M(v, i)^2) & \text{, if } \sum_{i} V_{M}(v, \mathcal{i}) > 0 \\\\ \mathbf{0} & \text{, otherwise, } \\end{cases} $

where $W_M(v, i) = -\cos(N_{v}^{(M)}, N_{i}^{(view)})$ is a weighting factor, $N_{v}^{(M)}$ is the vertex normal of $v$ in mesh $M$, and $N_{i}^{(view)}$ is the view direction of view $i$.

$ET_M$ computes the results with view direction weighting so that each viewpoint does not introduce significant errors for surfaces that are too skewed in the current viewpoint (because predictions in these places are usually inaccurate).

Comment 2: Missing ablation studies

We appreciate the depth of your feedback. To address your concern, we will enrich our paper with the following additional ablation studies to elucidate the contributions of each component within our model:

(a) ISOMER Module Analysis: We will incorporate a comparative experiment of the ISOMER module to better demonstrate its superiority over existing reconstruction algorithms.

(b) Explicit Target Algorithm Analysis: We will show qualitatively how the ExplicitTarget algorithm improves the final performance.

(c) Robustness analysis: we will quantitatively study the performance and differences of our method under non-front inputs.

(d) Resolution Impact Analysis: We will expand our study to include a qualitative comparison across various resolutions in order to demonstrate the differences between different resolutions.

Question 1. Thanks for your insightful question. The ground truth mask is determined based on the normal map predicted by the model. Predicted pixels with an RGB magnitude far from 1 are considered background.

Question 2. Thanks for your insightful question. To resolve your concerns, we add a new test with rotated objects in Table 2 to test robustness in non-front-facing views. The test results show that our method still performs well in this case, and even the geometry prediction is more accurate.

Question 3. We appreciate your attention to this matter. In response, we will enhance the clarity and details of our explanation in the revised version as follows:

• Edge Collapse: This operation is used to avoid and heal defects in the mesh. It involves selecting an edge within a triangle and collapsing it to the other edge, effectively merging the two triangles into a single triangle. This process can help to eliminate narrow triangles that might be causing issues in the mesh, such as those that are too thin to accurately represent the surface they are approximating. It prevents the creation of topological artifacts and maintains mesh quality.

• Edge Split: This is the opposite of edge collapse. In the edge split, an edge that is longer than a specified maximum length is divided into two, creating new vertices at the midpoint of the edge. This operation is used to refine the mesh, ensuring that the local edge length is kept close to the optimal length. It helps to maintain the quality of the mesh by avoiding edges that are too long, which could lead to an inaccurate representation of the surface.

• Edge Flip: Edge flip is an operation that adjusts the connectivity of the mesh to improve its quality. It involves flipping an edge within a triangle to connect two non-adjacent vertices, effectively changing the triangulation of the mesh. This can help to maintain the degree of the vertices close to their optimal value, which is typically six for internal vertices (or four for boundary vertices).

These operations aim to improve mesh quality, avoid defects, and ensure an accurate representation of the target geometry.

Remark 1. Thanks for pointing out the issue. We will modify the statement in our revision.

Remark 2. Thanks so much for your great suggestions. We are definitely considering adopting a name that is more straightforward and easily comprehended.

We appreciate your detailed comments. We will thoroughly revise our paper. Thanks again for your time and in-depth suggestions.

Sorry for the messed up Latex in Equation (1) above, here is a corrected version.

Here $d(i,j)$ denotes the value of coordinate (i,j) in the depth map, while $x$ is the integration process along the vertical line $y=j$, $\vec{n}(x)$ denotes the vector of the input normal field at the position $x$. $n_x$ is the component of the normal field $\vec{n}$ along the direction of the x-axis (which is a scalar function).

The confusion might arise because the letter 'x’ is used in two different contexts. To clarify, we’ll make the following adjustments: We will use the formula $d(i,j)=\sum_t=0^i n_x(t,j)$, where d and n_x are considered as 2D arrays in the discrete version. This change aligns better with how the variables are handled in the code.

Dear Reviewer,

Thank you for your patience and thorough review. We have addressed the issues you raised with detailed responses. We welcome you to engage in further discussion with us. Your insights are invaluable, and we are eager to clarify any points or provide additional information as needed.

Looking forward to your feedback and any further questions you may have.

Best regards, Authors

We sincerely thank you for your time and valuable comments, some of which are fundamental and in-depth suggestions that help us greatly improve our paper. To address your concerns, we present the point-to-point response as follows.

Comment 1: Limited Qualitative Results to Frontal View, Worse Side Views

Thanks for bringing this comment up, and we acknowledge the presence of this issue. Our investigation indicates that it stems from occasional inconsistencies in the multi-view predictions, which is a challenge inherent in all current multi-view reconstruction solutions. We are actively exploring ways to mitigate these inconsistencies to enhance the robustness and reliability of our reconstruction process.

Comment 2: Potential Bottleneck by the Two-view Initialization of First Step

Thanks for your insightful question. As introduced in our main text, the initialization process is specifically designed to guarantee the correct overall topological structure and is unrelated to reconstruction details. The majority of artifacts are primarily derived from inconsistencies in multi-view predictions rather than the reconstruction process. As shown in Figure 9, our method can yield satisfactory results even with a sphere as the initial shape. Furthermore, Figure 5(a) demonstrates that the ISOMER reconstruction step is capable of correcting some of the artifacts introduced by multi-view predictions. This showcases the resilience and corrective capabilities of our approach in refining the final reconstruction output.

Comment 3: Narrow Evaluation Scope, Limited Geometry Complexity

Thanks for your insightful question. For a consideration of fairness, the samples we have chosen for qualitative evaluation are largely based on previous methods, thus not incorporating diverse real-world objects. However, in our quantitative evaluation using the GSO dataset, a set of real-world objects scanned by Google, our method significantly outperforms existing methods. Our approach excels in capturing fine details, showcasing our proficiency in reconstructing complex geometries with high fidelity.

Regarding geometry complexity, our method is comparable to existing works such as Wonder3D, OpenLRM, and InstantMesh. It's important to note that, like these methods, we cannot generate unseen features such as "a solid bottom and an empty interior of a mug" when these aspects are not present in the input views. This limitation is inherent to the field and represents a challenge for all methods dealing with incomplete view data.

Comment 4: Pose Sensitivity and Distortion Issues

Thanks for pointing out the issue. To resolve your concerns, we add a new test with rotated objects in Table 2 to test robustness in non-front-facing views. The test results show that our method still performs well in this case, and even the geometry prediction is more accurate.

Comment 5: Oversimplified Texture Handling of Occluded Region

We appreciate your detailed comments. We agree that our current method for handling colors in non-visible areas is quite straightforward. The simplicity of this approach allows for extremely efficient processing times, ensuring the overall efficiency and stability of our workflow. We are committed to exploring more sophisticated coloring techniques in future work to enhance the visual output of our reconstructions.

Comment 6: Small Evaluation Set

Thanks for your thorough comment. We chose 30 random objects in alignment with the standards set by previous work (CRM, SyncDreamer, Wonder3D). Given that SyncDreamer takes up to 50 hours to generate 30 objects and results from other papers show that SyncDreamer performs significantly worse than existing methods, we exclude SyncDreamer and conduct an experiment on the entire GSO dataset. The results for all methods are provided in Table 1.

Comment 7: Lack of Quantitative Ablation Studies

We are grateful for your comprehensive comments. Recognizing the need for a more robust ablation study, we have taken the following steps to enhance our original analysis:

(a) ISOMER Module Analysis: We will incorporate a comparative experiment of the ISOMER module to better demonstrate its superiority over existing reconstruction algorithms.

(b) Explicit Target Algorithm Analysis: We will show qualitatively how the ExplicitTarget algorithm improves the final performance.

(c) Robustness analysis: we will quantitatively study the performance and differences of our method under non-front inputs.

Question 1:

We greatly appreciate your insightful feedback. In light of your suggestions, we will revise the introduction in the updated paper to clearly delineate our approach. Similar to 3D generation methods like Wonder3D and InstantMesh, our work draws inspiration from ImageDream and Zero123++ for multiview image generation. However, we employ two denoising models and a super-resolution model to accomplish multi-view generation with notable distinctions as follows:

(a) Orthogonal View Generation: Our first denoising model, similar to ImageDream, generates four orthogonal views instead of perspective views, which simplifies the subsequent reconstruction process and enhances multi-view consistency. The choice of orthogonal views facilitates a direct geometric correlation between pixels. Moreover, we integrate learnable class embedding to encode view information, enhancing the model's ability to understand and process multi-view inputs.

(b) IP-Adapter Utilization: Contrary to being used in the Multi-view Image Generation step, our method employs an IP-Adapter in the Multi-view Image Upscale step. We incorporate a controlnet model with IP-Adapter to allow for the enhancement of multi-view details and achieve the targeted resolution.

(c) Normal Prediction Setting: In the Normal Prediction module, we adopt a denoising model with a reference U-Net, akin to the concept in Zero123++. However, unlike Zero123++, which shares all weights between the reference U-Net and the main network, we utilize an independent pre-trained reference U-Net. We freeze all parts within the reference U-Net except for self-attention (Lines 544-545) to preserve the generalization ability from the pre-trained model.

Question 2:

Thanks a lot for pointing out! Here $d(i,j)$ denotes the value of coordinate $(i,j)$ in the depth map, while $x$ is the integration process along the vertical line $y=j$, $\\vec{n}(x)$ denotes the vector of the input normal field at the position $x$. $n_x$ is the component of the normal field $\\vec{n}$ along the direction of the x-axis (which is a scalar function).

The confusion might arise because the letter ‘x’ is used in two different contexts. To clarify, we’ll make the following adjustments: We will use the formula $d(i,j)=\\sum_{t=0}^i n_x(t,j)$, where d and n_x are considered as 2D arrays in the discrete version. This change aligns better with how the variables are handled in the code.

Question 3:

Thanks for pointing out the issue. We will correct these mistakes in the revision.

Question 4:

We thank the reviewer for raising this question, and we deeply regret the lack of detailed comments on the appendix's algorithm which has caused confusion. The variable cnt is crucial as it determines the number of while loop iterations at Line 11 (L11). It is used to record the number of iterations before reaching Line 21 (i.e., all colors are applied), and afterward, the iteration repeats cnt times to guarantee the color completion process is completed. Besides, we acknowledge the mistake at Line 3 (L3), where the initialization of 'colored' should indeed be based on the Inv. In the invisible vertex coloring, a flood fill operation is applied on the array 'colored' to fill the colors, while an ongoing laplacian smoothing is performed on the array C[i] to achieve a smoother color transition for a more aesthetically pleasing result. We appreciate your correction and will ensure future versions have clearer documentation to prevent misunderstandings.

Question 5:

The "Expansion" in our model serves as a regularization technique directly applied to the parameters, similar to weight decay, rather than functioning as a loss term. At each step, vertices are moved a small distance in the direction of their normals.

Question 6:

Thanks for pointing out the issue. We will modify this statement in our revision following your suggestion.

Question 7:

Thanks for your thorough comment. The entire ISOMER process takes approximately 10 seconds. Within this timeframe, the Mesh Initialization step accounts for about 2 seconds, the preliminary Mesh Reconstruction takes around 3 seconds, the Mesh Refinement step consumes approximately another 5 seconds, and the final Mesh Colorization is completed in less than 0.1 seconds.

We appreciate the reviewer for pointing out those issues. We will thoroughly revise our paper. Thanks again for your time and in-depth suggestions.

Dear Reviewer,

Thank you for your patience and thorough review. We have addressed the issues you raised with detailed responses. We welcome you to engage in further discussion with us. Your insights are invaluable, and we are eager to clarify any points or provide additional information as needed.

Looking forward to your feedback and any further questions you may have.

Best regards, Authors

We sincerely appreciate your constructive and thorough comments. Your main suggestions on the experimental setting and ablation studies help us refine our paper. To address your concerns, we present the point-to-point response as follows.

Comment 1: Narrow Evaluation Scope

Thanks for your thorough comment. GSO (Google Scanned Objects) is a set of real-world objects scanned by Google using specialized equipment for research. We chose 30 random objects in alignment with the standards set by previous work (CRM, SyncDreamer, Wonder3D). Given that SyncDreamer takes up to 50 hours to generate 30 objects and results from other papers show that SyncDreamer performs significantly worse than existing methods, we exclude SyncDreamer and conduct an experiment on the entire GSO dataset. The results for all methods are provided in Table 1. Additionally, following your suggestions, we will add more experiments on other real-world object datasets, such as MVImageNet, and provide a detailed evaluation in the revised paper.

For the user study, as shown in Appendix F, we rendered 360-degree videos of subject-driven 3D models and presented each volunteer with five samples of rendered videos from a random method. Volunteers rated the samples on four aspects: 3D consistency, subject fidelity, prompt fidelity, and overall quality on a scale of 1-10, with higher scores indicating better performance. We collected results from 30 volunteers, as shown in Table 2 in the Appendix. Our findings indicate that users significantly preferred our method across these aspects.

Comment 2: Insufficient Analysis of ISOMER

We appreciate your insightful comment. Addressing the concern about the novelty in the multi-view generation aspect, we will provide a unified response in the subsequent question. To resolve your concern regarding ISOMER, we will revise its introduction to emphasize the key insight of each component within ISOMER. The first step of ISOMER ensures topological consistency, while the second step efficiently reconstructs rough geometries, and the third step improves surface reconstruction accuracy by handling multi-view inconsistencies with ExplicitTarget.
Our experiments indicate that ISOMER is capable of enhancing consistency as demonstrated in Figure 5. We include a new comparative evaluation experiment that contrasts Wonder3D with and without ISOMER to provide a clearer understanding of the benefits in Table 1.

Comment 3: Lack of Novelty in Multi-view Generation

We thank the reviewer for raising this concern. The main focus of our paper is on addressing current limitations within current multi-view reconstruction approaches, exemplified by Wonder3D. Existing methods often suffer from low resolution, multi-view inconsistency, and low generation speed. Our methodology is specifically tailored to address each of these issues within the existing framework, rather than proposing an entirely new pipeline.

Here's a concise summary of our approach:

• Noval Reconstruction Algorithm: For the first time in such a pipeline, we have introduced a mesh-based reconstruction algorithm, which has gained significant improvements in speed and quality.

• Quality Enhancement: We have developed techniques to improve the resolution of reconstructed models, providing higher fidelity and detail.

• Consistency Improvement: Our approach includes mechanisms to reduce inconsistencies across different views, leading to more coherent and reliable 3D reconstructions.

Comment 4: Lightweight Ablation Studies

We are grateful for your comprehensive comments. Recognizing the need for a more robust ablation study, we have taken the following steps to enhance our original analysis:

(a) Quantitative Evaluation of the ISOMER Module: We have incorporated a detailed quantitative assessment of the ISOMER module to better demonstrate its contribution to overall performance.

(b) Resolution Impact Analysis: We have expanded our study to qualitative comparisons across various resolutions to see how different resolutions impact the final 3D reconstruction outcomes.

(c) Explicit Target's Effect on Texture: Mirroring the approach used for geometry, we will include additional experimental results that illustrate the impact of the Explicit Target method on texture quality in the revised manuscript. We show the difference in Figure 1.

Comment 5: Advantage of Generating Dense Meshes

We thank the reviewer for raising this concern. We believe that detailed and sharp geometry does require a sufficient number of surfaces. As illustrated in Figure 6 of the main text, the intricacies of text engraving cannot be achieved without enough surfaces. For scenarios that prefer more compact meshes, such as in gaming or other graphics-intensive applications, various existing mesh decimation and retopology techniques can be applied to achieve the desired outcome. Since this process is quite engineering-oriented, we did not include it within the scope of our paper. It is worth noting that while simplifying a high-precision dense mesh model to a more compact one is feasible, the inverse—enhancing a model with fewer details to a high-precision one—is significantly more challenging. This consideration is a primary reason our paper emphasizes the optimization of generation accuracy and surface count, aiming to address the complexities involved in achieving high-fidelity geometric detail.

Comment 6: Lack of Detailed Explanations

Thanks for pointing out the issue. We will modify our statement in the revision.

We will add more details about edge collapse, edge split, and edge flip in the revision as follows.

Edge Collapse: This operation is used to avoid and heal defects in the mesh. It involves selecting an edge within a triangle and collapsing it to the other edge, effectively merging the two triangles into a single triangle. This process can help to eliminate narrow triangles that might be causing issues in the mesh, such as those that are too thin to accurately represent the surface they are approximating. Edge collapse can prevent the creation of topological artifacts and maintain the quality of the mesh.

Edge Split: This is the opposite of edge collapse. In edge split, an edge that is longer than a specified maximum length is divided into two, creating new vertices at the midpoint of the edge. This operation is used to refine the mesh, ensuring that the local edge length is kept close to the optimal length. It helps to maintain the quality of the mesh by avoiding edges that are too long, which could lead to an inaccurate representation of the surface.

Edge Flip: Edge flip is an operation that adjusts the connectivity of the mesh to improve its quality. It involves flipping an edge within a triangle to connect two non-adjacent vertices, effectively changing the triangulation of the mesh. This can help to maintain the degree of the vertices close to their optimal value, which is typically six for internal vertices (or four for boundary vertices).

The goal of these operations is to improve the mesh quality while avoiding defects and ensuring that the mesh accurately represents the target geometry.

Question 1:

Thanks for your questions. The edges of sudden jumps will have a steep normal, and integrating over this normal gives a large depth difference. So this does not have an observable negative impact on the algorithm.

Question 2:

Thanks for your insightful questions. The 256-to-512 model is specifically tuned to integrate information from multiple views, which is crucial for ensuring consistency across different perspectives. The 512-to-2048 model, on the other hand, is optimized to concentrate on the finer details of the reconstruction, enhancing the overall quality of the output. Since the multi-view aware 256-to-512 bears a relatively higher computational load compared to the 512-to-2048 model, it is part of our strategic design to balance computational load with multi-view consistency and accuracy. This dual-model strategy is crafted to enhance efficiency in both training and inference, making our method more practical for real-world applications.

Question 3:

Yes. Since it is computed as a weighted sum, we rarely encounter inconsistency issues. However, It is indeed a promising direction to explore more advanced coloring methods in future work to further enhance the robustness and quality of our results.

Question 4:

Thanks for pointing this out, we'll be more detailed in the revised version! For a noisy latent with shape [B, C, H, W], we further add a [1, C, 1, 1] shaped N(0, 0.1) gaussian noise to it, thus enhancing the generalization of the network and avoiding the zero terminal SNR problem.

Question 5:

We thank the reviewer for raising this concern. For a legitimate normal field, any closed line integral should be zero, which is the meaning of being irrotational. However, predictions from neural networks cannot possibly meet this condition.

Question 6:

Thanks for pointing out the typo. We will correct it to Figure 5(b).

Question 7:

Thanks for your valuable comments. We will add more details as follows.

Because it is four views that are in the same plane, there is an obvious pairwise polar geometric relationship, i.e., any pixel in one view corresponds to a horizontal straight line in the neighboring view. Thus if a non-null pixel, does not have any non-null pixels on the corresponding straight line in the neighboring view, then the data is ilegal. This problem is usually caused by objects in the data that have no thickness, i.e., they are observable in one view and happen to be invisible in another view.

Question 8:

It refers to Multi-view Image Upscale, with a detailed description in Appendix B.

We thank the reviewer for pointing out these typos. We will carefully proofread the manuscript and sincerely hope that you will find the revision satisfactory. We appreciate your time and insightful comments.

Thank you for the detailed response and additional experiments.

After reviewing your comments, I have the following questions and concerns:

1. It appears that Wonder3D+ISOMER performs slightly better than Wonder3D alone but still falls short (a lot) of Unique3D. Does this imply that the primary improvement in Unique3D comes from the multi-view generation (2D diffusion models) rather than the reconstruction model (ISOMER)? Could you explain the significant performance gap between Wonder3D+ISOMER and Unique3D? Additionally, would combining Unique3D's multi-view prediction with Wonder3D's reconstruction model result in better performance than "Wonder3D+ISOMER"? I'm asking because I want to understand whether the improvement is due to the reconstruction method or the multi-view prediction. Based on the current results, it's difficult to determine.

2. The authors claim that their multi-view module offers higher resolution, better multi-view consistency, and faster generation speed. However, the last two points are not supported by experiments. How do you quantitatively measure multi-view inconsistency?

3. When directly calculating the point color as a weighted sum of projected 2D pixels, why doesn't this method suffer from inconsistency, especially at the boundaries or overlaps between multiple views?

4. I still don't fully get the point. When there is a sudden depth change (e.g., due to occlusion), how can simply integrating the normals yield the correct depth? I don't think there would be steep normals; rather, there should be multiple segments of normals reflecting the surface properties of each region separately.

5. Regarding the generation of dense meshes, I agree that generating sharp features requires a sufficient number of faces. However, the ability to export a large number of triangles doesn't necessarily indicate that the method can generate sharp and detailed geometry. For instance, existing methods can increase their resolution to 512 or 1024 when using the Marching Cubes algorithm, which will produce many more triangles, but the underlying geometry remains unchanged. My point is that you should only claim the generation of sharp details (with verification) as an advantage, not just the generation of a large number of triangles.

6. Where is the "Quantitative Evaluation of the ISOMER Module"? I couldn't locate it in the rebuttal PDF.

7. Regarding the "irrotational normal", what is the motivation behind "introducing a random rotation to the normal map before integration? The process is repeated several times, and the mean value of these integrations is then used to calculate the depth, providing a reliable estimation." How does this solve the problem?

8. In the rebuttal, the qualitative examples provided are limited to one or two instances, which is not very convincing or helpful for understanding. Please avoid this and include more examples in your revision.

We sincerely thank you for your time and valuable comments, some of which are fundamental and in-depth suggestions that help us greatly improve our paper. To address your concerns, we present the point-to-point response as follows.

Comment 1: Multi-view Consistency Analysis

We appreciate the reviewer bringing this comment up. To answer your concern, first, our choice to adopt a sequential design is based on the following observations: 1) Integrating the image diffusion model with two super-resolution models into a 2048-resolution diffusion model would result in an enormous computational load due to the high resolution. 2) We noticed that methods combining both image and normal diffusion models (e.g., Wonder3D) frequently yield normal results lacking sufficient details, and the guidance scale does not effectively control the output. Our experiments on this hybrid architecture were in alignment with these observations. By decoupling the two models, we have notably enhanced the accuracy of the normals, as the normal diffusion model does generations based on clear images rather than noisy image latents. Therefore, we have chosen to use separate image and normal diffusion models to improve the fidelity of the normals and the subsequent mesh reconstruction. ISOMER can even be used to improve the consistency of other methods. For example, in Table 1, we replaced Wonder3D's reconstruction method with ISOMER, which is not only faster but also of higher quality.

Comment 2: Clarity in "ExplicitTarget Optimization" Subsection

We value your insightful suggestions. To address your concern, we will revise the explanation and equations in "ExplicitTarget Optimization" subsection for better comprehension. Here is our improved version:

(ExplicitTarget). Let $Avg(V, W) = \frac{\sum_i{V_i W_i}}{W_i}$ represent the weighted average function, and $V_{M}(v, i): (\mathbb{N}^+, \mathbb{N}^+) \rightarrow \{0, 1\}$ represent the visibility of vertex $v$ in mesh $M$ under view $i$. $Col_M(v,i)$ Indicate the color of vertex v in viewpoint i. We compute the ExplicitTarget $ET$ of each vertex in mesh $M$ as

$ ET_{M}(v) = \\begin{cases} Avg(Col_M(v,i), V_{M}(v, i) W_M(v, i)^2) & \text{, if } \sum_{i} V_{M}(v, \mathcal{i}) > 0 \\\\ \mathbf{0} & \text{, otherwise, } \\end{cases} $

where $W_M(v, i) = -\cos(N_{v}^{(M)}, N_{i}^{(view)})$ is a weighting factor, $N_{v}^{(M)}$ is the vertex normal of $v$ in mesh $M$, and $N_{i}^{(view)}$ is the view direction of view $i$.

$ET_M$ computes the results with view direction weighting so that each viewpoint does not introduce significant errors for surfaces that are too skewed in the current viewpoint (because predictions in these places are usually inaccurate).

Comment 3: ISOMER Boundary Inconsistencies and Non-front-facing Views

Thanks for your thorough comment. We agree that the key insight of ISOMER could be better clarified. Following your suggestions, we will revise our paper to emphasize that ISOMER directly handles the case where the global normal of the same vertex is inconsistent across viewpoints.

As demonstrated in our ablation study (Figure 5), previous reconstruction algorithms often yield poor results with blurriness or wavy patterns when faced with inconsistent inputs, while adopting ISOMER notably improves the reconstruction results. We include a new evaluation experiment comparing Wonder3D with and without ISOMER to provide a clearer understanding of the benefits in Table 1.

Additionally, we added a new test with rotated objects in Table 2 to test robustness in non-front-facing views. The test results show that Unique3D still performs well in this case, and even the geometry prediction is more accurate (Because the shape of an object is better estimated, e.g. a book).

Comment 4: Insufficient Quantitative Evaluation

We chose 30 random objects following the standards set by previous work (CRM, SyncDreamer, Wonder3D). Given that SyncDreamer takes up to 50 hours to generate 30 objects and results from other papers show that SyncDreamer performs significantly worse than existing methods. We exclude SyncDreamer and conduct a thorough experiment on the full GSO dataset. During our experiments, we found that GRM’s online samples were unavailable and lacked open-source code, preventing further testing of GRM’s results. The results for all methods are provided in Table 1.

Dear Reviewer,

Thank you for your patience and thorough review. We have addressed the issues you raised with detailed responses. We welcome you to engage in further discussion with us. Your insights are invaluable, and we are eager to clarify any points or provide additional information as needed.

Looking forward to your feedback and any further questions you may have.

Best regards, Authors