Shape as Line Segments: Accurate and Flexible Implicit Surface Representation

Thanks for your time and effort in reviewing our manuscript. In the following, we will address your concerns comprehensively.

Comment 1. How to handle cases where a line segment should intersect the surface multiple times? Such cases should be common when the line segment is larger than the local feature size.

Response.

• We would like to highlight that this is a prevalent issue encountered by both Marching Cubes and Dual Contouring (and their variants) based methods for extracting surfaces from implicit fields such as SDFs and UDFs, not exclusive to our approach. Both Marching Cubes and Dual Contouring assume that each edge of the cubes has at most one intersection point with the surface, which is similar to our proposed LSF. Under such an assumption, although the edge/line segment has more than one intersection with the surface when they are larger than the local feature size, only one intersection point is used when extracting the surface, thus introducing distortion.

• To solve this, one can increase the resolution of cubes when extracting the surfaces. This is also a commonly used solution in methods such as Marching Cubes and Dual Contouring. As shown in Fig. 15 of the Appendix, with the resolution increasing, the detailed structures are preserved after surface extraction.

Comment 2. I am not sure the proof for Theorem 1 is correct. Why are p and n not functions of u/v? What assumptions have to be made about the surface, to ensure this invariance to u/v?

Response.

• We have clarified this issue in Lines 175-195 of the revised manuscript. During surface extraction, methods like Marching Cubes, Dual Contouring, and their variants approximate the local geometry around each cube edge as a plane. Similarly, we can use the tangent plane at the query line segment to represent the local geometry around it, as shown in Fig. 3 of the manuscript.

• With the tangent plane utilized to approximate the local geometry around the line segment, we can select any point on the tangent plane, $\mathbf{p}_0$, and the normal vector, $\mathbf{n}$, to represent the tangent plane. In Theorem 1, we only need to replace $\mathbf{p}$ with $\mathbf{p}_0$, remaining others unchanged.

• To verify the correctness of Theorem 1, we utilize two different kinds of normal vectors in E-DC, random and zeros, and the visual comparison and error maps are shown in Fig. 14 of the revised Appendix. It can be seen that using the gradient of $s$ as the direction of normal vectors can achieve significantly better reconstruction accuracy, demonstrating the correctness of Theorem 1.

Comment 3. Eq(2), should there be a square of the dot product? Otherwise the minimization is not bounded from below.

Response. Thanks for pointing this out! We have corrected it in the revised manuscript.

Comment 4. Sec.3.3, it's said that surface normal is determined as the gradient of s. But the gradient is taken with respect to which variable? u or v? What if they give different gradients?

Response.

• Thanks for the valuable comment. We have clarified this issue in Lines 202 - 210 of the revised manuscript. For a line segment intersecting the surface, we use the gradient of the endpoint nearest to the intersection point to determine the normal vector's direction at that point. As shown in Fig. 2(b), if $s\leq0.5$, the normal vector is defined as $\mathbf{n}=\nabla_{\mathbf{u}}s/||\nabla_{\mathbf{u}}s||$; otherwise, it is defined as $\mathbf{n}=\nabla_{\mathbf{v}}s/||\nabla_{\mathbf{v}}s||$.

• In Eq. (2), the dot product is squared, so both $\mathbf{n}$ and $-\mathbf{n}$ produce the same result.

Comment 5. Related to the above question, the network producing the line-segment field should have certain invariance and equi-variance. That is, with respect to the switch of u/v, its output o (binary intersection) should be invariant, and its output s (ratio) should be equivariant. But the network design in its current form seems not preserve such properties.

Response.

• We confirm our method does not suffer from the mentioned issue. We have clarified this in Lines 233- 236 of the revised manuscript.

• As shown in Eq. (4) of the manuscript, the two endpoints of the query line segment are concatenated before being fed into the network. The intersection ratio $s$ is always defined as the ratio of the distance from the intersection point to the endpoint at the second concatenated position to the total length of the line segment. When preparing the data pairs for training, if the concatenation order of the two endpoints is exchanged, the associated ratio for supervision will also change accordingly, i.e., $s(\mathbf{u}||\mathbf{v})+s(\mathbf{v}||\mathbf{u})=1$, and the intersection flag will remain. Thus, during inference, the output ratio always indicates the ratio of the distance from the intersection point to the endpoint at the second concatenated position to the total length of the line segment and the intersection flag is independent of the concatenation order.

Comment 6. The results frequently show holes that are undesirable. Such holes would not appear for SDF based methods. What can be done to fix such issues?

Response.

• We employ a sigmoid function in the last layer of the network to constrain the output in the range of [0, 1]. For the intersection flag, we apply a threshold to binarize the output. If the threshold is set too low, some intersecting line segments may be incorrectly classified as non-intersecting, resulting in holes when extracting the surface using E-DC.

• The number of holes can be reduced by lowering the threshold for the intersection flag. In our experiments, the threshold was set to 0.1. We have included visual results of the extracted surfaces under different thresholds in Fig. 12 of the revised Appendix.

Comment 7. The ablation about normal vectors (Sec.4.3) is quite obscure, with only one visual example and minor differences. Can quantitative differences be observed with or without normal vector for dual contouring?

Response.

• We have added error maps of the reconstructed surface in Fig. 11 of the revised manuscript. The error maps show that for sharp structures, the reconstructed surface using E-DC with normals is significantly more accurate than E-DC without normals.

• The table below shows the quantitative comparisons, where in addition to the commonly used CD and F1 metrics, the suggested metrics by Reviewer SSdJ, namely ECD (Edge Chamfer Distance) and EF1 (Edge F-score), specially designed to evaluate the preservation of sharp edges [(Chen et al., SIGGRAPH 2022] are also used. Incorporating normals in our E-DC yields minor improvements in surface extraction metrics such as CD, NC, and F1, but leads to significant enhancements in ECD and EF1. This is because most regions of a shape are smooth, with only a small portion exhibiting sharp features.

Dear Reviewer eA2y

Thanks for your time and effort in reviewing our manuscript. In our previous response, we addressed your concerns directly and comprehensively. We very much look forward to your further feedback on our responses. Let us discuss.

Best regards,

The authors

Thanks for your time and effort in reviewing our manuscript, as well as your recognition. In the following, we will address your concerns comprehensively.

Comment 1. Quantitative comparison are only provided for a limited amount of shapes in ABC dataset.

Response.

• The ABC dataset used in our experiment is a widely recognized benchmark for reconstruction tasks. Additionally, we created a new dataset, the Non-Manifold ABC dataset, by randomly combining two models from the ABC dataset. The models in the newly constructed dataset consist of non-manifold structures, making it a more challenging benchmark. The numerical results on the ABC and non-manifold ABC datasets show the effectiveness of our method.

• Following your suggestion, we conducted additional comparisons on the DeepFashion3D dataset. As shown in the table below, it can be seen that our method still achieves better reconstruction performance.

Comment 2. The improvement in surface quality appears incremental. The line segment field uses query grids of $128^3$. What is the counterpart resolution of point queries for the compared methods?

Response.

The resolutions of used query grids in each baseline method were set as $128^3$, which is the same as our method. We have clarified this issue in Sec. 4 of the revised manuscript.

Comment 3. Given the method’s focus on sharper surface reconstruction, it is recommended that the authors also report metrics evaluated specifically around surface edges, similar to those used in ``Neural Dual Contouring (Chen et al., SIGGRAPH 2022)''.

Response.

Thanks for your valuable comments. We additionally utilize ECD and EF1 metrics to evaluate the reconstruction accuracy. The table below shows the numerical results of different methods, demonstrating the significant advantage of our method.

Comment 4. For the applications to Waymo data, are the data also rescaled to a unit cube?

Response.

We partition the point cloud from the Waymo dataset into multiple blocks using a side-window strategy, with each block reconstructed independently. The reconstructed surfaces from all blocks were then combined to form the complete scene.

Dear Reviewer SSdJ

Thanks for your time and effort in reviewing our manuscript. In our previous response, we addressed your concerns directly and comprehensively. We very much look forward to your further feedback on our responses. Let us discuss.

Best regards,

The authors

Thanks for your time and effort in reviewing our manuscript. In the following, we will address your concerns comprehensively.

Comment 1. Motivation and comparison with Neural Dual Contouring (NDC): The paper could better clarify its approach relative to NDC, particularly why optimizing vertex locations with QEF outperforms direct network predictions.

Response.
Our work is significantly different from NDC.

• Our work contains the following three contributions: 1) a novel line segment field (LSF)-based implicit surface representation (Sec. 3.1); 2) an optimization-based surface extraction method from LSFs (Sec. 3.2); and 3) a learning-based method to estimate the LSF of a given point cloud (Sec. 3.3).

• NDC focuses only on surface extraction from implicit representations. To be specific, Dual Contouring is integrated with neural networks to extract surfaces from traditional SDFs or UDFs.

• Moreover, regarding the surface extraction method, while both NDC and our E-DC are adaptations of Dual Contouring, they also differ significantly. NDC employs a neural network to predict the intersection point within each cube, making it a purely data-driven approach. However, this limits its generalizability, i.e., changes in the scale or resolution of SDF/UDF grids can result in significant distortions in the reconstructed surfaces. Differently, our E-DC utilizes Least Squares Method to calculate the intersection points in each cube, as shown in Sec. B of the Appendix. Additionally, NDC is slower than our E-DC when it comes to surface extraction.

Comment 2. Training and inference times: A table comparing performance, training time, and inference time would provide clarity. The method appears slower in inference than others; for instance, NDC achieves sub-second inference at a 128 resolution. Then, the question is how good is the trade-of between performance and inference time of the proposed method ?

Response.

• In the task of neural implicit representation (Sec. 3.3), the time cost consists of two parts, training/overfitting time and surface extraction time. We have recorded the runtime for these two parts, as shown in the table below. It is worth noting that the resolution of Marching Cubes, Dual Contouring, and our E-DC is 128. During training, our method takes slightly longer compared to neural BOF/SDF/UDF because it processes a 6D input vector and outputs a 2D vector, unlike the latter approaches which take a 3D input vector and produce a scalar output. NGLOD and UODF employ complex architectures, resulting in longer training times. In terms of surface extraction time, our method is slower than MC, MeshUDF, and DualMesh but significantly faster than DCUDF and NDC.

• In the task of surface reconstruction from point clouds (Sec. 3.4), the inference times of different methods are shown in the table below. Our method is comparable to POCO and GeoUDF but faster than GIFS. CAP-UDF is an optimization-based method, which requires more time.

Comment 3. Dependence on K-NN size for point cloud surface reconstruction: The paper lacks details on how the K-NN value is chosen for each dataset and point cloud density, which would help to understand its impact on performance. How the K-NN value is chosen for each dataset and point cloud density, which would help to understand its impact on performance?

Response.

• We clarify that we indeed studied the impact of the K-NN size on reconstruction accuracy in Table 4 of the manuscript and Fig. 19 of the Appendix (Fig.15 of the original Appendix), where the number of points is 40000.

• Following your suggestion, we further studied how the K-NN size affects reconstruction accuracy on point clouds with various densities (i.e., various numbers of points). As listed in the tables below, it can be seen that the variation trend of K-NN size is similar across point clouds with different densities, i.e., too large or small K-NN sizes can lead to a decrease in reconstruction accuracy, and the highest reconstruction accuracy is consistently achieved under various densities when the K-NN size is set to 20.

• We also test different K-NN sizes on the Non-manifold ABC dataset. As listed in the table below, obviously, the variation trend of K-NN size is similar across point clouds with different datasets, indicating that K=20 is an appropriate choice.

Comment 4. To solve the QEF optimization, the gradient the network w.r.t e one of the edge vertices should be used. Which one do you use? Also since they have different orientations how does this choice impact the stability of the optimization and the overall performance?

Response.
Thanks for the valuable comment. We have clarified this issue in the revised manuscript. For a line segment intersecting the surface, we use the gradient of the endpoint nearest to the intersection point to determine the normal vector's direction at that point. As shown in Fig. 2(b), if $s\leq0.5$, the normal vector is defined as $\mathbf{n}=\nabla_{\mathbf{u}}s/||\nabla_{\mathbf{u}}s||$; otherwise, it is defined as $\mathbf{n}=\nabla_{\mathbf{v}}s/||\nabla_{\mathbf{v}}s||$. We have added the corresponding illustration in Lines 202-210 at Sec. 3.2 of the revised manuscript.

Dear Reviewer hFqZ

Thanks for your time and effort in reviewing our manuscript. In our previous response, we addressed your concerns directly and comprehensively. We very much look forward to your further feedback on our responses. Let us discuss.

Best regards,

The authors

We appreciate the reviewer’s timely feedback and their acknowledgment that most of the questions were addressed in our initial response. Below, we address the remaining questions in detail.

Comment 1. I think a discussion on the difference between the proposed method w.r.t NDC and DualMesh-UDF should be included in the final version of the paper.

• We have included a discussion in Section 3.2 (Lines 224-230) of the revised manuscript highlighting the differences between the proposed E-DC method and NDC as well as DualMesh.

Comment 2. I have an additional question regarding the inference times. Can you clarify why your surface extraction time is lower than NDC [1]. I would expect your method to have a slight overhead due to the QEF optimization.

• The inference time reported in our previous response refers to the total surface extraction time, including the time spent calculating the SDFs/UDFs for the grids.

• We measured the runtime for calculating intersection points in each cube and performing triangle extraction, as shown in the table below. The results demonstrate that our E-DC is faster than NDC in terms of intersection point calculating in each cube and triangle extraction. This efficiency is due to our method not relying on a neural network to estimate the intersection points within each cube. The following results are added in Lines 972-977 of the revised appendix.

Comment 3. Also, if these numbers correspond to the ABC dataset why are they different from table 2 in [1] ? From this table one can also see that solving the QEFs (DC-est) is indeed slower that NDC.

• The scales of the shapes used in our method and NDC differ. As noted in Lines 291–292, we scaled all shapes to fit within a bounding cube with an edge length of 2, centered at the origin, spanning the range [-1,1]. In contrast, NDC rescales each shape to fit within a unit cube with an edge length of 1, also centered at the origin, spanning the range [−0.5,0.5]. Additionally, the evaluation metric configurations differ, where we use L1-CD, while [1] uses L2-CD. Furthermore, the F1-score threshold in our method is 0.005 and 0.01 (with a bounding box size of 2), whereas the threshold in [1] is 0.003 (with a bounding box size of 1).

• DC-est utilizes traditional Dual Contouring methods, where the QEFs are solved on CPU. Differently, NDC utilizes a trained neural network to predict intersection points in each cube, which is fast and with high efficiency. Thus, the inference of NDC is faster than DC-est.

I thank the authors for their detailed rebuttal. It addresses most of my questions. I think a discussion on the difference between the proposed method w.r.t NDC and DualMesh-UDF should be included in the final version of the paper.

I have an additional question regarding the inference times. Can you clarify why your surface extraction time is lower than NDC [1]. I would expect your method to have a slight overhead due to the QEF optimization. Also, if these numbers correspond to the ABC dataset why are they different from table 2 in [1] ? From this table one can also see that solving the QEFs (DC-est) is indeed slower that NDC.

[1] Neural Dual Contouring (Chen et al., SIGGRAPH 2022)

Dear Reviewer hFqZ

Thank you for your response to our previous response. We hope that our new response have addressed your remaining concerns and look forward to receiving your further feedback.

Best regards,

The authors

Thank you for your reponse and for the revision. Even if the proposed method can result in slightly slower extraction times compared to other methods (MC, MeshUDF, and DualMesh) , it introduces a representation that shows better performance. The extraction times may be also be improved using Octrees like in DualMesh. Overall, I think this is a good contribution and I'm willing to raise my rating.

Thanks for your time and effort in reviewing our manuscript, as well as your recognition. In the following, we will address your concerns comprehensively.

Comment 1. Some results exhibit only minor differences, as seen in Figure 8, where the outcomes of Ours and GeoUDF appear quite similar. Presenting additional results, such as error maps or normal maps, may provide clearer visual distinctions. I suggest add normal maps, which may highlight the differences more clearly in certain results, such as in Fig. 11.

Response.

We have included the error maps of the results in Figs. 13 and 17 of the revised Appendix.

Comment 2. Lack of analysis of limitations. It would be interesting to evaluate how the proposed method performs on thin structures, such as leaves, or particularly complex geometric structures, like a basket, to better understand its applicability to different types of shapes.

Response.

Unlike previous implicit representations that primarily focus on individual points in 3D space, our proposed SALS targets line segments in 3D space, which introduces additional complexity. Consequently, when the number of sampled line segments is limited, the network struggles to accurately model the LSF, leading to distortions in the reconstructed surfaces. As shown in the table below, the reconstructed models exhibit significant distortion when the number of sampled line segments is too low. We have added the discussion about the limitation in Sec. E of the revised Appendix.

Comment 3. Some related works need to be added. [1-3].

Response. Thanks for your valuable suggestion. We have discussed the mentioned papers in Sec. 2 of the revised manuscript.

Comment 4. In the network design, I’m interested in exploring the results of replacing max pooling with mean pooling, as it may provide more information of neighbors in feature aggregation.

Response.

We have replaced the max pooling with mean pooling in the network, and the comparison is shown in the table below. It can be seen that using mean pooling would cause an accuracy decrease. The reason is that mean pooling treats the features of K-NN points equally. More specifically, compared to max pooling, mean pooling introduces some unimportant information when processing less significant points, such as those on the boundary of the K-NN patch, resulting in reduced accuracy.

Dear Reviewer 1tcg

Thanks for your time and effort in reviewing our manuscript. In our previous response, we addressed your concerns directly and comprehensively. We very much look forward to your further feedback on our responses. Let us discuss.

Best regards,

The authors