MIND: Material Interface Generation from UDFs for Non-Manifold Surface Reconstruction

In the qualitative results of Figure 5, I would also suggest presenting the original images, which helps to determine which results are closer to the ground truth.

Thanks for your suggestion. We will provide the original images in the revision.

What is the N.A. in Figure 5?

In this case, N.A. indicates that DMUDF fails to extract any mesh facets from the given UDF.

Specifically, DMUDF uses an octree-based structure to improve computational efficiency, but it relies on the UDF being accurate across the entire spatial domain. However, DEUDF, as a UDF learning method, focuses on accuracy near the surface and often produces unstable or noisy UDF values farther away. This inaccuracy leads the DMUDF octree to misidentify valid leaf nodes, resulting in complete reconstruction failure in large regions. As a result, no mesh can be generated for this case, and we mark it as N.A. in Figure 5 to reflect this failure.

In addition to chamfer distance, are there any other metrics to demonstrate the superiority of MIND?

The main advantage of MIND over previous methods lies in its ability to support non-manifold surface reconstruction. However, there is currently no standardized benchmark for quantitatively evaluating this capability. To partially address this, we incorporate geodesic distance comparisons in our experiments to help assess the quality and correctness of the extracted non-manifold structures.

The presented results are obtained on simple scenarios. Can the non-manifold be applied to more realistic and complex scenarios, such as medical image scenarios?

Yes, MIND can be applied to more realistic settings. In fact, we have tested our method using UDFs learned by NUDF on the Left Atrial Appendage (LAA) segmentation dataset, which contains anatomically complex structures and potential non-manifold features. We will provide the visual results in the revision.

Thanks for the suggestion from Reviewer Rx5y, we present the results of the standard deviations and medians on the Shapenet-Car dataset and Deepfashion3d dataset. And the following results will be added to the revision.

There are no results on variants / ablations of the approach, which would reveal the benefit to the reader of certain design decisions. For example, at L182, small enhancements to M3C are described, which "enhance accuracy" and "further refine", yet there is no experimental validation of the extent to which these help; similarly for the refinement in the paragraph at L185.

Thank you for pointing this out. We agree that ablation studies are valuable for understanding the contribution of each component.

In the revision, we will include additional mesh extraction results that compare our method to the original M3C and our pipeline without DCUDF optimization.

However, due to this year's NeurIPS rebuttal guidelines that prohibit uploading additional figures, we are unable to include these visual results in the current response. We will ensure that these ablations are clearly presented and discussed in the revised submission.

The bolded claim at L44 seems inconsistent with e.g. DMUDF and NDC – while these do not always work perfectly, they do address the relevant task/setting and can in principle achieve non-manifold surface reconstructions.

Thank you for the observation. We agree that both DMUDF and NDC are capable of generating non-manifold surfaces in principle. We will revise the statement to clarify our intended meaning: while these methods may produce non-manifold structures, they often do so without explicitly controlling for manifold corrrectness. As a result, they may introduce unintended non-manifold artifacts even in regions that should be manifold.

In contrast, our method is specifically designed to capture non-manifold features where they are required, while preserving the integrity of manifold regions.

We will revise the statement and clarify this distinction in the revision.

Is it possible to perform the qualitative evaluation of similarity of geodesic distance fields instead quantitatively on the shapenet / deepfashion experiments?

Thank you for the suggestion. ShapeNet could indeed be a suitable candidate for evaluating the effectiveness of MIND. However, in practice, many ShapeNet models are constructed by stacking individually modeled manifold parts, which are not truly non-manifold in a topological sense, and would require extensive data preprocessing and remeshing to convert them into a clean dataset with genuine non-manifold topology.

Once such preprocessing is completed, it would be possible to perform quantitative evaluations, such as comparing geodesic distance fields computed on the extracted surfaces against ground-truth mesh surfaces.

However, because of the significant preprocessing effort required, we consider this to be a direction for future work.

The subject area is only borderline appropriate for NeurIPS – it is pure computer graphics / geometry processing, with no machine learning involved in the proposed approach (it can be applied to neural UDFs, but that is not in any way intrinsic to the method).

Our method is designed to address a critical bottleneck in UDF-based 3D learning pipelines. As an effective mesh extraction algorithm for UDFs, our method directly supports and enhances neural approaches for 3D reconstruction and generation.

UDFs have recently gained significant attention in the AI and NeurIPS communities as a flexible representation for 3D shapes, especially in tasks involving neural implicit functions. Prior works (e.g., NDF, CAP-UDF, NeUDF, NU-NeRF, DreamUDF) demonstrate the advantage of UDFs in representing open surfaces, but their full potential, specifically for representing non-manifold structures, remains underutilized due to the lack of suitable mesh extraction algorithms.

Although some methods such as DMUDF and NDC attempt to extract non-manifold geometry, they often introduce uncontrolled and excessive non-manifold artifacts, even in regions that are intrinsically manifold. In contrast, our method explicitly preserves manifold regions while correctly capturing non-manifold features, unlocking a unique capability of UDFs that was previously inaccessible.

We believe that our method has direct implications for learning-based 3D reconstruction. For example, NU-NeRF reconstructs nested transparent objects by combining multiple SDFs but suffers from discontinuities at non-manifold intersections due to the limitations of SDFs. Similarly, in 3D generation, methods such as TripoSF aim to produce models with rich internal structures, but the use of SDFs prevents mesh reconstruction at non-manifold junctions. Our method may inspire 3D AIGC work on generating non-manifold structures, either by extracting meshes from neural UDFs or, in the future, by attempting to integrate material interface (MI) prediction into neural pipelines.

In summary, we believe our work makes a foundational contribution to UDF-based shape modeling, which is highly relevant to current and emerging research directions in 3D learning. We hope that by presenting this work at NeurIPS, we can stimulate further research in this area and provide a missing tool for the community working on UDF-based 3D reconstruction and generation.

Voxel resolution dependence causes missing out on finer geometric features

We agree that resolution imposes fundamental limits on geometric accuracy. However, this limitation is not unique to our method, as it is shared by all voxel-based surface extraction techniques, including MC, DC and their many variants.

These methods generate discrete mesh representations, whose level of detail is inherently constrained by the grid resolution. While increasing resolution can recover finer details, the discrete and resolution-dependent nature is a general characteristic of the entire class of methods.

The performance of the method seems to be bottle-necked by the accuracy of the underlying UDF prediction method.

This is in fact a common challenge for all mesh extraction algorithms. In general, no surface extraction method can recover geometric details that are not already encoded in the input distance field. However, different methods vary in how well they handle noise or imperfections in the UDF.

Our method demonstrates improved robustness to UDF noise due to the erosion operation, which stabilizes region labeling and removes spurious fragments. Furthermore, the DCUDF-based optimization further refines the extracted geometry, improving accuracy up to the intrinsic limit imposed by the input UDF quality.

Moreover, our method can further improve accuracy without increasing resolution by directly subdividing the mesh extracted from M3C and then performing the optimization process. For example, we trained a UDF on the Armadillo model using DEUDF, achieving a Chamfer Distance (CD) of 0.003484 at a resolution of 256. By applying one subdivision step (splitting each triangle into four), the CD decreases to 0.003335.

Laplacian smoothing strategy makes it harder to judge the impact of the global labeling aspect of MIND

Thanks for raising this question. It is important to note that the global multi-labeled field in MIND is responsible for determining the topological structure of the reconstructed surface, specifically, how different regions are partitioned and connected.

In contrast, Laplacian smoothing and subsequent DCUDF optimization refine only the geometry of the surface without altering its topology.

The impact of MIND’s global labeling can therefore be assessed through the topology-related results presented in the paper, such as geodesic distance comparisons and highlighted non-manifold edges. These reflect the correctness of the extracted connectivity structure, regardless of geometric smoothing.

Lack of quantitative metrics like resolution sensitivity, parameter sensitivity (r1, r2)

Thank you for the suggestion. The parameters $r_1$ and $r_2$ depend on the grid size and can be flexibly adjusted as long as the conditions provided in the experimental section are met. We will provide quantitative results at different resolutions.

How are your input UDF's obtained?

Our work focuses on extracting zero level sets from learned UDFs. Several prior works have developed methods to learn UDFs from point clouds or multi-view images. In our experiments, we use UDFs generated by a variety of state-of-the-art learning-based approaches, including CAPUDF, LevelSetUDF, DEUDF, NU-NeRF, and NeUDF. We use them as input to our method to evaluate the effectiveness of MIND in extracting non-manifold surfaces.

Given that you are using voxel-based downsampling of point clouds, can you provide an ablation study of voxel resolution vs accuracy of capturing fine details?

Thank you for your suggestion. We use PCA to calculate normals of point clouds. Generally, point cloud orientation algorithms require a dense point cloud to obtain sufficiently reliable normal vectors. Since calculating normals from point clouds and obtaining the local side field is not time-consuming, we do not recommend reducing the resolution of the point cloud downsampling. An excessively low point cloud density will decrease the reliability of normals, thereby generating unnecessary topological structures. We will supplement the relevant experimental results in the revision.

Can the authors explain how is this different from Neural Dual Contouring's approach (which uses the corner signs on a grid)?

NDC uses a neural network to directly predict the corner signs of grid cells, which are then used to extract surfaces. However, this approach is sensitive to prediction errors due to inherent instability and lack of spatial consistency. For example, NDC may assign inconsistent signs to adjacent cubes where no surface is present, leading to redundant faces in the reconstructed mesh.

In contrast, our method computes the initial local side field using optimization based on geometric cues, rather than learned predictions. This provides enhanced stability and consistency, especially in regions with complex or ambiguous geometry. As a result, our method significantly reduces the types of artifacts commonly observed in NDC.

Laplacian smoothing tends to smooth/blur sharp features. Have the authors considered using a quadratic error function optimization to preserve the sharp features?

Thank you for your insightful suggestion. We agree that using the quadratic error function to compute vertex positions is effective for preserving sharp features. However, in this work, we focus on the non-manifold topology identification and non-manifold mesh extraction. Hence, we would like to leave this as a future work to improve the mesh accuracy.

Have the authors considered comparing against MeshCAP, MeshUDF?

Both MeshUDF and MeshCAP extend Marching Cubes to support surface extraction from UDFs. However, Marching Cubes is inherently limited to generating manifold structures, and these methods cannot preserve non-manifold structures by design. Since our method specifically targets non-manifold surface reconstruction, we believe it would be inappropriate to perform a direct comparison against approaches that are fundamentally restricted to manifold outputs.

That said, MeshUDF and MeshCAP remain useful baselines for evaluating performance on manifold regions, and we include them where relevant in our experiments. In particular, the original NeUDF employs MeshUDF for mesh extraction, and we reported the NeUDF results in both the main paper and the supplementary material. We will make this connection more explicit in the revised version.

I appreciate the authors' effort to write the rebuttal and I acknowledge the responses.

I have increased my rating.

Reviewer TBUd

Why in Eq.1 and 2 are the indicator functions normalized by a cubic term?

The sign of a query point in our local side field is determined based on the normal vectors of nearby surface points. In manifold regions, these normals are typically consistent, leading to a stable sign determination. However, near non-manifold structures, multiple surface patches with differing normals may intersect. This can make the sign assignment ambiguous or unstable.

To address this, we adopt a formulation inspired by the generalized winding number, where the contribution of each neighboring point is inversely weighted by the cube of its distance to the query point. This cubic normalization gives more influence to points that are closer, effectively biasing the sign toward the dominant local geometry. In particular, it encourages alignment with the normal vector of the nearest surface patch, helping to suppress fluctuations caused by distant conflicting normals.

This weighting strategy improves robustness and locality in sign estimation, especially near regions with complex topology or non-manifold junctions.

It would be really good to have a limitations section in the paper. The MI insight seems cool, but is it always applicable. Further insights into where it works and where it fails would be invaluable.

Thanks for your suggestions. Our method can reliably generate non-manifold features where needed, in contrast to dual contouring variants that may introduce spurious non-manifold elements even in manifold regions. However, some limitations remain.

First, our approach requires an erosion operation, which may inadvertently remove small geometric structures and lead to missing facets. We generally advise against using excessively low resolutions during erosion. If a lower resolution is necessary elsewhere (e.g., for sampling the unsigned distance field), we recommend applying erosion after upsampling the data.

Second, due to the optimization step involved in our pipeline, the method is more computationally expensive than non-optimization-based approaches. Nevertheless, this additional cost is justified by the significant improvement in reconstruction quality—particularly in faithfully preserving non-manifold structures, which existing methods are unable to achieve.

Please also see the discussion on limitations in the supplementary material.