NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction

Thank you for your insightful feedback. We hope the following response can address your concerns.

Design of Explicit Bias Correction

The bias correction looks like partially. It doesn't punish negative SDF before $t^*$, so it is not completed.

As you mentioned, although aligning $t^*$ with the SDF zero-crossing is intuitive, our experiments showed that this approach requires special design considerations. We tried simply constraining $f(t^\*)$ to approach zero but found that visible surface defects persisted. We also attempted to constrain the SDF values both before and after $t^\*$, which led to very strange optimized surfaces, likely due to overly strong constraints on the SDF. We have shown the visual result in Figure 3 from our PDF file related to the global response.

The key to our method's design lies in correcting large relative bias with explicit loss functions, while smaller biases are corrected by gradually scheduling the lower bound of the scale factor. We empirically found that visible surface errors are always caused by $t^\*$ is being before SDF zero-crossing points $t^0$. Therefore, we penalize the SDF value just after $t^\*$ to trend towards negative values, which prevents $t^*$ from being before $t^0$. For the case where $t^\*$ is after $t^0$, we can directly address it by gradually scheduling the lower bound of the scale factor, as the bias in this situation is relatively small.

We provided a mathematical explanation under the assumption of a sufficiently small local surface. In the VolSDF framework we used, In this case, if the angle between the ray and the plane is θ, and the distance from the ray’s origin to the plane is $d$, then the distribution of the SDF along the ray can be expressed as $f(r(t)) = −t\sin θ + d$. By setting the partial derivative of the rendering weight with respect to $t$ to zero, we can directly obtain a closed-form solution for $t^\*$:

$$t^*=\frac{s\ln (k) + d}{\sin\theta},$$

where $k=2\sin\theta$ if $\sin \theta \leq 0.5$, and $k=(2+\sin\theta-\sqrt{\sin^2\theta+4\sin\theta})^{-1}$ if $\sin \theta > 0.5$.

In this case, the closed-form solution for $t^0$ can be directly obtained as $t^0 = d/\sin\theta$. Then the distance between $t^*$ and $t^0$ is

$$t^{\Delta} = t^* - t^0 = \frac{\ln k }{\sin\theta}s.$$

When $t^\Delta < 0$, $t^\*$ is before $t^0$, and when $t^\Delta > 0$, $t^\*$ is after $t^0$. In our PDF file related to the global response, we visualized the values of $t^\Delta$ under different $\theta$ in Figure 2. We found that when $t^\*$ is before $t^0$ ($\sin\theta$ less than 0.5), the relative bias is significantly greater than when $t^\*$ is after $t^0$. This aligns with our experimental findings, where visible erroneous surfaces are primarily caused by $t^*$ preceding $t^0$. Therefore, we penalize the SDF value just after $t^\*$ to prevent $t^\*$ being before $t^0$.

We will include the complete derivation in the appendix.

Comparison with TUVR

TUVR is not compared, which also aims to reduce the bias.

Since TUVR is not open-source and only reports metrics on the DTU dataset, we reproduced its unbiased SDF-to-density technique and combined it with a hash grid to create the TUVR-Grid method for comparison. Additionally, we compared our results on the DTU dataset with the reported results in the paper that did not use MVS priors (TUVR-MLP). The results are shown in the table below.

Results on Tanks and Temples dataset:

Results on DTU dataset:

The performance of TUVR is not ideal on all datasets because its unbiased nature is not fully guaranteed, and it is also somewhat affected by over-regularization.

Evaluation of $t^*$

how to evaluate $t^*$?

In lines 197-198 of the paper, we mentioned that we directly use the sampled point with the highest weight as the point where $t^\*$ is located. This is because we perform importance sampling along the ray before rendering, and the sampled points are already concentrated in areas with higher weights. Therefore, there is no need for additional iterations to find $t^*$. Experimentally, this approach does not significantly impact the results.

Other Evaluation Metrics

Why only use F-score for validation? What about other scores, such as Chamfer distance.

On the Tanks and Temples benchmark, previous work only reported the F-score as a surface quality metric. We compared our F-score results with the reported F-scores from other methods, although we also evaluated additional metrics using our method. However, these additional metrics cannot be directly compared with other methods.

Additionally, we compared the PSNR metric for image reconstruction on the Tanks and Temples dataset and the Chamfer distance metric on the DTU dataset. The results are shown in the table below:

Impact of Explicit Bias Correction on Training Time

Does it become slower due to the bias correction?

After incorporating the explicit unbias correction design, the time per iteration increased from approximately 59ms to 63ms. After 300,000 iteration, this results in an additional 20 minutes of training time. This is primarily because we need additional network inference to compute the SDF slightly beyond $t^*$.

Thank you for your insightful feedback. The issues we mentioned regarding SDF-based rendering, such as the bias problem, might indeed be addressed through improved SDF-to-density modeling. While it is possible to tackle these issues directly through mathematical modeling, we believe that our detailed design can be very easily implemented within the existing framework and effectively and significantly improve surface quality. This also provides considerable value.

Impact of Additional Signal Supervision

Is it possible to avoid the biased situation shown in Figure 3 by introducing depth or other supervision information?

We tested monocular depth and normal supervision on an indoor scene from ScanNet++ to examine the density bias. We found that the density bias no longer caused significant surface errors. This aligns with intuition, as the impact of density bias depends on the convergence of the surface. Any SDF-to-density design ensures that when the scale factor approaches zero, the weights along the ray can approximate a Dirac distribution at the SDF zero-crossing points.

We believe that the bias issue is caused by photometric ambiguity, meaning that the surface cannot be accurately determined based on color alone. This is consistent with the outlier distribution in MVS methods. From the MVS perspective, the scattered distribution in low-texture areas of MVS results is similar to the weight distribution along the ray and is considered outliers and removed. However, our goal is dense surface reconstruction, and we need to uniquely determine the surface within this scattered distribution. Therefore, we select the part with the highest weight as the most reliable surface.

When we have more reliable supervision signals, such as monocular depth and normals, these outliers decrease because the model can use these additional signals to accurately determine the surface.

Image Reconstruction Comparision

There is a lack of comparison of image reconstruction results. Since color is used in the training process and the baseline method in the paper includes image reconstruction comparisons, why does this paper not provide qualitative or quantitative results of image reconstruction?

While our design is primarily intended for surface reconstruction tasks, we have also compared the results of NeuS and Neuralangelo using the image quality evaluation methods from Neuralangelo, across different parameter scales.

As shown, with fewer parameters, our method (Ours-19) achieves image reconstruction quality close to that of Neuralangelo. With a larger number of parameters, our method (Ours-22) surpasses Neuralangelo in terms of image reconstruction quality.

Addressing Typographical Errors and Omissions

In the experimental section, Table 2 does not explain the measurement indicator of the data. Combined with Appendix 5, it can be inferred that the indicator is the percentage F-Score.

Thank you for your valuable feedback and for pointing out the typographical errors and missing elements in our manuscript. We sincerely apologize for these mistakes. We will carefully revise the manuscript to correct these issues and ensure that it meets the highest standards of clarity and accuracy.

Thanks for your insightful feedback. For more comprehensive ablation study, comparison with TUVR, results on DTU and more explanation of stochastic gradients, please refer to our global response. Apologies for the brevity due to word constraints.

Motivation of Local Scale Factor

Why an adaptive scale is important.

We further illustrate the importance of the local scale in Figure 3 from our PDF file related to the global response, using a simple single plane scenario for explanation. This plane has a low-texture region on the left and a rich-texture region on the right.

Without special constraints, the rendering weight should converge to a Dirac delta function at the surface in the richly textured region and form a scattered distribution in the weakly textured region.

However, under the assumption of a global scale factor, all areas on the plane follow the same distribution which will be derived in the following sections. This means that both richly textured and weakly textured regions share the same density bias, preventing the surface from converging correctly to the richly textured surface with higher certainty.

By introducing the local scale factor, the most significant difference is that the distribution of rendering weights along the ray is no longer uniform. The network can adaptively converge in the richly textured regions, and the density bias in these areas is no longer affected by the low-texture regions.

In Adaptive shells, their focus is primarily on rendering quality and they did not evaluate surface reconstruction metrics. A straightforward application to surface reconstruction may lead to issues such as increased density bias. We tackle this by implementing special designs to ensure effectiveness, such as gradually scheduling the lower bound of the scale factor to correct relatively small biases and the explicit bias correction.

Design of Explicit Bias Correction

In Equation (8), why does it only penalize a single point instead of a region or multiple points (both SDF > 0 and SDF < 0)?

Although aligning $t^*$ with the SDF zero-crossing is intuitive, our experiments showed that this approach requires special design considerations. We attempted to constrain the SDF values both before and after $t^\*$, which led to very strange optimized surfaces, likely due to overly strong constraints on the SDF. We have shown the visual result in Figure 3 from our PDF file related to the global response.

The key to our method's design lies in correcting large relative bias with explicit loss functions, while smaller biases are corrected by gradually scheduling the lower bound of the scale factor. We empirically found that visible surface errors are always caused by $t^\*$ is being before SDF zero-crossing points $t^0$. Therefore, we penalize the SDF value just after $t^\*$ to trend towards negative values, which prevents $t^*$ from being before $t^0$. For the case where $t^\*$ is after $t^0$, we can directly address it by gradually scheduling the lower bound of the scale factor, as the bias in this situation is relatively small.

We provided a mathematical explanation under the assumption of a sufficiently small local surface. In the VolSDF framework we used, In this case, if the angle between the ray and the plane is θ, and the distance from the ray’s origin to the plane is $d$, then the distribution of the SDF along the ray can be expressed as $f(r(t)) = −t\sin θ + d$. By setting the partial derivative of the rendering weight with respect to $t$ to zero, we can directly obtain a closed-form solution for $t^\*$:

$$t^*=\frac{s\ln (k) + d}{\sin\theta},$$

where $k=2\sin\theta$ if $\sin \theta \leq 0.5$, and $k=(2+\sin\theta-\sqrt{\sin^2\theta+4\sin\theta})^{-1}$ if $\sin \theta > 0.5$.

In this case, the closed-form solution for $t^0$ can be directly obtained as $t^0 = d/\sin\theta$. Then the distance between $t^*$ and $t^0$ is

$$t^{\Delta} = t^* - t^0 = \frac{\ln k }{\sin\theta}s.$$

When $t^\Delta < 0$, $t^\*$ is before $t^0$, and when $t^\Delta > 0$, $t^\*$ is after $t^0$. In our PDF file related to the global response, we visualized the values of $t^\Delta$ under different $\theta$ in Figure 2. We found that when$t^\*$ is before $t^0$ ($\sin\theta$ less than 0.5), the relative bias is significantly greater than when $t^\*$ is after $t^0$. This aligns with our experimental findings, where visible erroneous surfaces are primarily caused by $t^*$ preceding $t^0$. Therefore, we penalize the SDF value just after $t^\*$ to prevent $t^\*$ being before $t^0$.

We will include the complete derivation in the appendix.

Impact of Color Conditioning on Normal

Why would optimizing the color conditioned on the normal restrict topological change?

In Figure 6 from our PDF file related to the global response, we experimentally observed that in indoor scenes, when color conditioning on normals is applied, the optimization process becomes very slow and affects the optimization results. However, this issue is nearly absent in outdoor scenes. We believe this is primarily due to the convergence behavior of the scale factor. Indoor scenes often have many low-texture areas, leading to larger scale factors and a greater scatter distribution. Additionally, color conditioned on normals, which is intended for geometry disentanglement as mentioned in IDR and only resonable for surface points, results in many details being optimized to incorrect positions before convergence is achieved.

However, our use of stochastic normals helps mitigate these erroneous surfaces. For detailed regions, the estimated normals have greater variance, which alleviates the impact of incorrect surfaces. This is also demonstrated in our ablation experiments, as shown in the table below:

Thank you for your insightful feedback. Below, we address the concerns raised in the review.

Image Reconstruction Comparision

The authors only provide F-score for the evaluation metrics except for some single scenes in appendix. It will be good to provide more such as PSNR, SSIM, etc.

While our design is primarily intended for surface reconstruction tasks, we have also compared the results of NeuS and Neuralangelo using the image quality evaluation methods from Neuralangelo, across different parameter scales.

As shown, with fewer parameters, our method (Ours-19) achieves image reconstruction quality close to that of Neuralangelo. With a larger number of parameters, our method (Ours-22) surpasses Neuralangelo in terms of image reconstruction quality.

Hash Grid with a Dictionary size of $2^{22}$

Is it possible to compute the proposed method under gird resolution of 2^22 just as nerualangelo did?

We also tested our method on the Tanks and Temples dataset with a hash dictionary size of 2^22 as shown in the table below.

We found that increasing the number of parameters did not significantly improve the results but instead dramatically increased the running time (from 18 hours to 48 hours). The redundant parameters (with a dictionary size of $2^22$ resulting in over a hundred million parameters) did not enhance surface accuracy but did help fit the training images better (PSNR from 26.90 to 27.67). Neuralangelo addresses some surface defects through over-parameterization. However, we achieve the same effect with significantly fewer parameters, thanks to our specialized design.

Furthermore, we test with a hash dictionary size of $2^{22}$ in a large-scale scenario from BlendedMVS dataset. By significantly increasing the number of parameters, the model is able to capture fine-grain details on images, thereby preserving more details in the reconstructed surface. In our PDF file related to the global response, Figure 1 displays the quantitative comparison.

Training time

It will be good to discuss the runtime compared with other baselines.

As mentioned in lines 504-506 of the original text, on the Tanks and Temples dataset, Neuralangelo requires nearly 48 GPU hours for optimization, whereas our method only requires approximately 18 GPU hours. This is due to our specialized design for addressing surface defects, which allows us to maintain performance without requiring as many parameters as Neuralangelo.