U-CAN: Unsupervised Point Cloud Denoising with Consistency-Aware Noise2Noise Matching

We sincerely appreciate Reviewer hTqa for the acknowledgment of our work and constructive feedback. We respond to each question below.

Q1: Alternative backbone networks.

We thank reviewer hTqa for the insightful suggestion regarding the integration of the proposed framework with alternative backbone networks. Following this advice, we replaced the EdgeConv-based denoising network with PointNet++ and PointTransformer to evaluate the generality of U-CAN across different architectures. The results are reported in Table F below.

Table F: Effectiveness of U-CAN with different backbones.

As demonstrated, the U-CAN framework consistently achieves robust denoising performance across different backbone networks. PointNet++ is a relatively older backbone that provides stable but suboptimal results, while PointTransformer, a more modern architecture, achieves performance comparable to our default denoising model. Notably, we directly integrate these backbones into U-CAN without the need for manual tuning of settings or hyperparameters specific to each architecture. These results highlight the strong generality, adaptability, and robustness of the proposed framework.

Q2: Evaluation on indoor point cloud datasets like ScanNet.

We sincerely thank the reviewer hTqa for the valuable advice and fully recognize the importance of evaluating U-CAN under real-world indoor scene scans. To provide a more comprehensive evaluation of U-CAN's performance, we conducted an experiment on the ScanNet dataset, a real-world indoor dataset consisting of noisy point clouds obtained from scanning sensors. These point clouds contain complex structures (e.g., furniture, partitions) and exhibit varying noise distributions (e.g., sensor reflections and occlusions). We apply U-CAN to predict denoised point clouds from these scans and report the numerical metrics comparing the denoised predictions with the ground truth clean meshes provided in the ScanNet dataset. The results, along with comparisons to state-of-the-art unsupervised point denoising methods, are presented in Table B below, where U-CAN achieves the best performance.

Table B. Comparisions under indoor ScanNet dataset.

U-CAN achieves superior performance on both the indoor and outdoor scene point clouds scanned in the real world, demonstrating its remarkable capability in real-world downstream applications in robotics, autonomous driving, AR/VR, and autonomous interior navigation.

We deeply appreciate the reviewer y6U3 for the thoughtful feedback and time invested in evaluating our work. We respond to each question below.

Q1: The difference on motivation and core framework with NoiseMap.

We would like to thank the reviewer y6U3 for pointing out the similarity in motivation between U-CAN and NoiseMap [1]. U-CAN did share a similar motivation with NoiseMap in transferring the success of the Noise2Noise scheme from the 2D image domain to 3D point cloud denoising. However, the core framework and methodology of U-CAN differ significantly from those of NoiseMap in four key aspects:

1. Learning scheme and framework. NoiseMap is an overfitting approach which requires more than 10 minutes to converge for each noisy observation and cannot generalize to new observations. In contrast, U-CAN is a standard learning-based point cloud denoising framework that learns data-driven patterns during training. The resulting model can directly generalize to unseen noisy observations with a fast inference time (10 seconds). As a result, U-CAN is suitable for downstream tasks, while NoiseMap’s reliance on per-instance overfitting hinders its use in real-world scenarios.

2. Difference in core contributions. (a) A key innovation of U-CAN is the Denoising Consistency Constraint which aims to learn consistent denoising patterns by minimizing geometric differences between the denoised outputs of different noisy observations. We are the first to identify and address an overlooked issue in prior unsupervised denoising methods for 2D and 3D domains: the Noise2Noise scheme often fails to produce consistent predictions from different noisy inputs, leading to ambiguity in optimizing fine geometric details.

(b) Beyond 3D vision, another key contribution of U-CAN is its extension to unsupervised denoising in other modalities. The proposed Denoising Consistency Constraint is a general term applicable to both 3D point cloud and 2D image denoising. We introduce a modified version of SOTA unsupervised image denosing method ZS-N2N, achieving significant performance gains. We believe this general constraint can benefit unsupervised denoising across a broader range of modalities, including but not limited to 2D and 3D vision—for example, speech denoising.

3. Representation difference. NoiseMap adopts Signed Distance Functions (SDFs) as its 3D shape representation, which are inherently limited to closed surfaces. In contrast, U-CAN predicts denoising paths directly on point clouds, enabling it to represent 3D shapes with arbitrary topologies. To evaluate the performance of both NoiseMap and U-CAN on non-closed 3D shapes, we conducted additional experiments on the DeepFashion3D dataset, which consists of 3D garments with open surfaces. Quantitative comparisons are presented in Table C below.

Table C: Comparisons with NoiseMap under DeepFashion3D.

4. Model architecture. The architectures of NoiseMap and U-CAN differ significantly. NoiseMap is an overfitting method that uses a simple MLP as its neural network backbone. In contrast, U-CAN is designed as a robust learning-based point cloud denoising model with efficient inference, adopting a multi-step architecture built upon dynamic EdgeConv modules.

Q2: Lack of comparison with NoiseMap.

We would like to clarify that we have conducted comprehensive comparison between U-CAN and NoiseMap in terms of both performance and efficiency. Section C.1 of the Appendix is dedicated to this comparison: Table 9 reports quantitative results on CD, P2M, and runtime, while Figure 9 provides visual comparisons. These results demonstrate that U-CAN achieves performance comparable to NoiseMap, both quantitatively and qualitatively.

For the reviewer’s convenience, we include Table 9 from the Appendix below.

Table 9 of the Appendix: Comparisons with overfitting method NoiseMap.

The reason we do not include NoiseMap in the comparison in the main paper is that it is an overfitting approach, requiring more than 10 minutes to converge for each noisy observation and failing to generalize to new observations. With redundant overfitting, NoiseMap achieves good performance but cannot be used in real world applications. Therefore, we excluded NoiseMap from Table 1, as comparing it with other learning-based baselines would not be fair. Nonetheless, U-CAN achieves comparable performance to NoiseMap, with 60x faster runtime, demonstrating its robustness and superiority.

Q3: Claim of multi-step denoising.

We fully acknowledge that the multi-step denoising design was first introduced in IterativePFN and should not be claimed as a contribution. We will revise the statement in L.54 to "We introduce U-CAN, a novel framework for unsupervised point cloud denoising by leveraging a neural network to infer a multi-step denoising path ..."

Q4: Loss function setting of multi-step denoising.

Indeed, your understanding is accurate: the multi-step denoising framework applies a loss function at each step for supervision. We observed the same phenomenon as previous studies, where applying loss functions at each step results in more robust point cloud denoising. The results of our ablation studies on the loss settings are reported in Table D below.

Table D: Ablation studies on the multi-step loss setting.

Q5: Performance under low noise levels.

We would like to clarify that the "worse" P2M performance at 1% noise is actually due to a version difference in the P2M implementation used in ScoreDenoise. For direct comparison with our most relevant baseline, ScoreDenoise, we directly report the numerical results from their paper. However, the P2M implementation in PyTorch3D used by ScoreDenoise was unstable at the time of their results, leading to abnormally good outcomes that cannot be reproduced with the latest version, particularly at lower noise levels (e.g., 1%). For more details, please refer to Issue 12 in the ScoreDenoise GitHub repository. Recent supervised methods, such as IterativePFN (CVPR 2023), DP-LTS (CVPR 2024), and 3DMambaIPF (AAAI 2025), have confirmed the issue and follow the stable PyTorch3D P2M function. With this stable function, P2M values are significantly higher. Please refer to Table 1 in IterativePFN for details.

We justify that we use the same stable PyTorch3D implementation as IterativePFN/DP-LTS/3DMambaIPF, which also face the issue of larger P2M values. To ensure a fair comparison, we will update Table 1 in U-CAN with results from IterativePFN. We update the P2M results for ScoreDenoise-TTD using the stable implementation and report them in Table E below. Note that all P2M results in U-CAN, except for Table 1, are evaluated with the consistent stable PyTorch3D version and do not suffer from this issue.

Table E: Updated P2M comparison with ScoreDenoise-TTD.

Furthermore, we would like to clarify that U-CAN consistently outperforms previous unsupervised methods across all noise levels, including those lower than 1% and higher than 3%. We have conducted comprehensive comparisons in Section C of the Appendix, where Table 7 and Table 8 demonstrate that U-CAN significantly outperforms the unsupervised SOTA methods GLR, DMR-TTD, and ScoreDenoise-TTD at noise levels of 0.2%, 0.5%, 4%, and 5%.

Q6: Missing related works.

We greatly thank reviewer y6U3 for the advice on improving the related work section. We follow the suggestions to include more discussions of latest full-supervised denoising methods. We will add the statements in Sec.2.2, after L.91:

"StraightPCF advances IterativePFN by learning to move noisy points to the clean surfaces along the shortest path. PD-LTS designs an invertible network for achieving noise disentanglement in the latent space. P2PBridge and SuperPC incorporate diffusion models into point cloud denoising. 3DMambaIPF leverages the powerful Mamba model for more efficient point denoising."

The corresponding references will be added.

Q7: Over-smooth results on flat surfaces.

We acknowledge that over-smoothing on flat surfaces (e.g., walls) is a common issue in unsupervised methods. We thank reviewer y6U3 for the insightful comment. A practical solution is to reduce the number of denoising steps, especially in scenes with rich structural details. In our observations, a small number of steps (e.g., 1 to 2) is sufficient to remove strong noise like artifacts near windows or road signs, while also preserving flat surface details. We will include visualizations on the Paris-rue-Madame dataset using the 1 to 2 step version of U-CAN in the revised submission, since figures cannot be uploaded during the rebuttal phase of NeurIPS.

Q8: Typos and clarifications.

The symbol $P$ in Equation (1) is actually the noisy point cloud $P_a$ for the initial step. We will correct the symbol and other typographical mistakes in our revision.

Dear Reviewer y6U3,

We sincerely appreciate your willingness to raise the score and your constructive feedback, which has substantially improved our work. It appears the final rating has not yet been updated in the system. At your convenience, would you kindly submit it?

Thank you again for positive assessment and thoughtful engagement with our work.

Best regards,

The authors

We deeply appreciate the reviewer 25Zo for the thoughtful feedback and time invested in evaluating our work. We respond to each question below.

Q1: The noise distribution of different noisy observations.

The noisy observations $P_a$ and $P_b$ are corrupted with noise sampled from the same distribution. We clarify that the training samples can be obtained in two ways: a) multiple noisy observations of the same object. This is done by adding noise drawn from the same distribution multiple times to the same object. The introduced noises are just different samples from the same noise distribution. This setting is realistic and supported by modern LiDAR systems, which can capture 10–30 noisy observations per second with a consistent noise distribution. b) several subsets of a dense noisy observation. For example, we can achieve 5 training instances of 10k points from a noisy observation of 50k points. Since all subsets originate from the same noisy observation, they share the same noise distribution.

In conclusion, both approaches produce training data with a consistent noise distribution. An ablation study comparing these two input strategies is presented in Table 12 of the Appendix, where results demonstrate the effectiveness of both.

For the reviewer’s convenience, we include Table 12 from the Appendix below.

Table 12 of the Appendix: Ablation studies on input patterns.

Q2: Can the network learn from complementary regions of $P_a$ and $P_b$?

We would like to clarify that all regions of the training noisy observations $P_a$ and $P_b$ are corrupted with noise from the same distribution and at the same level. There are no "complementary regions" that could result in data leakage. Actually, we add the noises to the entire global shape, where each single point at every location of the global shape is perturbed by noise sampled from the same distribution.

Q3: The performance of U-CAN under point clouds contain noise in the real world.

We fully recognize the importance of evaluating U-CAN on real-world scans. In fact, we have conducted experiments and demonstrated the performance of U-CAN on the Paris-rue-Madame dataset, which is an outdoor real-world dataset acquired using laser scanners and contains complex scanning noise. Please refer to Sec.4.2 of the main paper for more details on the experimental settings and visualizations. A comprehensive comparison between U-CAN and both supervised and unsupervised baselines, including PCN, DMR-TTD, and ScoreDenoise-TTD, is presented in Figure 5 of the main paper.

Paris-rue-Madame is a real-world outdoor scanned dataset. For a more comprehensive evaluation on the performance of U-CAN, we conduct another experiment on a real-world indoor scanned dataset ScanNet. The input data from ScanNet consists of noisy point clouds captured by scanning sensors. We apply U-CAN to denoise these scans and evaluate the results by comparing the predictions with the ground-truth clean meshes provided in the dataset. Quantitative results and comparisons with state-of-the-art unsupervised denoising methods are shown in Table B below, where U-CAN achieves the best performance.

Table B. Comparisions under indoor ScanNet dataset .

U-CAN achieves superior performance on both the indoor and outdoor scene point clouds scanned in the real world, demonstrating its remarkable capability in real-world downstream applications in robotics, autonomous driving, AR/VR, and indoor navigation.

Q4: The model weights in two branches of U-CAN, model complexity and inference time.

The two branches in U-CAN are actually the same network with shared weights, and therefore do not introduce any additional model complexity. U-CAN contains ~ 789K parameters, which is slightly more than ScoreDenoise (~ 187K) and DMR (~ 225K). However, we argue that U-CAN remains a lightweight network, as most recent (supervised) point cloud denoising methods contain over 1M parameters.

We also clarify that U-CAN achieves comparable runtime to other unsupervised point cloud denoising methods despite having more parameters, thanks to its efficient EdgeConv-based architecture. Table A below presents the inference time comparison between U-CAN, state-of-the-art unsupervised baselines (ScoreDenoise-TTD, DMR-TTD), the supervised baseline PCN, and the traditional algorithm GLR.

Table A. Runtime comparisons under PU-Net dataset.

Q5: The introduction of Noise2Noise.

We thank reviewer 25Zo for the valuable suggestion to include more descriptions of Noise2Noise in the introduction. In fact, we have devote a full paragraph (L.139-L.144) on Sec.3.2 to introducing the original work and core techniques of Noise2Noise. To provide a more comprehensive background, we will add more content of Noise2Noise to the introduction. Specifically, we will add the sentense before L.34:

"Noise2Noise [19] is a pioneering framework for unsupervised image denoising, which learns to produce clean images by encouraging the denoised output to resemble other noisy observations of the same image at pixel-level. Inspired by Noise2Noise [19], we introduce U-CAN ..."

Q6: The compared algorithms.

We would like to clarify that most recent works focus on supervised point cloud denoising. In the area of learning-based unsupervised point cloud denoising, the compared methods DMR-TTD and ScoreDenoise-TTD are the most recent and relevant to U-CAN. Additionally, we have included a comparison with another recent overfitting-based approach, NoiseMap, published at ICML 2023, in Table 9 and Figure 9 of the Appendix. U-CAN demonstrates comparable performance to NoiseMap, both quantitatively and qualitatively.

For the reviewer’s convenience, we include Table 9 from the Appendix below.

Table 9 of the Appendix: Comparisons with over-fitting method NoiseMap.

The reason we do not include NoiseMap in the comparison in the main paper is that it is an overfitting approach, requiring more than 10 minutes to converge for each noisy observation and failing to generalize to new observations. With redundant overfitting, NoiseMap achieves good performance but cannot be used in real world applications. Therefore, we excluded NoiseMap from Table 1, as comparing it with other learning-based baselines would not be fair. Nonetheless, U-CAN achieves comparable performance to NoiseMap, with 60x faster runtime, demonstrating its robustness and superiority.

Dear Reviewer 25Zo,

As the reviewer-author discussion period is nearing its end, we are sincerely looking forward to your feedback on our rebuttal. Please let us know if our responses address your concerns. We would be glad to make any further explanation and clarification.

Thanks,

The authors

Dear Reviewer 25Zo,

We sincerely appreciate the time and effort you have dedicated to reviewing our work. As the discussion period approaches its conclusion, we would like to kindly follow up to confirm whether our responses have adequately addressed your concerns. If there is any aspect requiring further clarification, we would be more than happy to provide additional explanations and continue the discussion.

We greatly appreciate your valuable feedback and look forward to hearing from you.

Best regards,

The authors

Dear Reviewer 25Zo,

Thank you again for the time and thought you have devoted to reviewing our work. With less than a day remaining in the discussion period, we would like to kindly check whether our responses have sufficiently addressed your concerns. If any points would benefit from further clarification, we would be happy to provide additional explanations promptly.

We sincerely appreciate your feedback and look forward to your reply.

Best regards,

The authors

We deeply appreciate the reviewer gzhj for the thoughtful feedback and time invested in evaluating our work. We respond to each question below.

Q1: The computational cost and training time of U-CAN.

As the reviewer correctly pointed out, the computational cost of EMD increases substantially with the number of points. We address this problem by learning point denoising at the patch level, where EMD is computed only between local point cloud patches containing a relatively small number of points (around 1,000). This strategy significantly reduces training cost and leads to more robust and efficient denoising. We conducted an ablation study to investigate the impact of varying patch sizes at Sec.D and Table 11 of the Appendix, where the results indicate that a patch size of 1,000 points is optimal for U-CAN. This configuration supports efficient training with EMD, where the convergence of U-CAN takes only about 25 hours on a single NVIDIA 3090 GPU. For reference, the training of ScoreD-TTD takes about 40 hours on a single NVIDIA 3090 GPU.

For the reviewer’s convenience, we include Table 11 from the Appendix below.

Table 11 of the Appendix: Ablation studies on point cloud patch sizes.

Moreover, we clarify that EMD is used only during training and does not affect the inference efficiency of U-CAN. Table A below presents a comparison of inference times between U-CAN, state-of-the-art unsupervised baselines (ScoreDenoise-TTD, DMR-TTD), the supervised baseline PCN, and the traditional algorithm GLR.

Table A. Runtime comparisons under PU-Net dataset.

Q2: How are different noisy observations generated.

The noisy observations $P_a$ and $P_b$ can be achieve by: a) multiple noisy observations of the same object. The manner is supported by modern LiDAR systems which can capture multiple (10-30) noisy observations per second. b) several subsets of a dense noisy observation. For example, we can achieve 5 training instances of 10k points from a noisy observation of 50k points. An ablation study of training U-CAN with these two input patterns can be found on Table 12 of the Appendix, where the results demonstrate the effectiveness of both kinds of input.

For the reviewer’s convenience, we include Table 12 from the Appendix below.

Table 12 of the Appendix: Ablation studies on input patterns.

Q3: The 1-1 matching pattern between noisy and denoised point clouds.

Yes, your understanding is correct: clusters of noisy points $P_a$ are mapped to their "closest" points in the denoised point cloud of another noisy observation $P_b$, based on Earth Mover's Distance (EMD). It is important to note that "closest" here is defined by EMD, which seeks to minimize the total one-to-one transport cost from $P_a$ to the denoised $P_b$, rather than the point-wise Euclidean distance. Mapping based on Euclidean distance is consistent with the implementation of Chamfer Distance, which has been shown to fail to converge under the noise-to-noise matching scheme in unsupervised point cloud denoising, leading to a severe degradation of CD/P2M from 2.497/1.105 to 15.14/11.36, as demonstrated in Table 4 of the ablation study.

The core insight of the noise-to-noise matching scheme is that, by aligning one noisy observation with the denoised prediction of another observation of the same shape in a one-to-one manner, the underlying clean structure can be statistically inferred. However, we also observe that this strategy may result in inconsistencies across different denoising results. To address this, we propose a consistency-aware constraint that minimizes geometric differences between pairs of denoised predictions, leading to more robust denoising performance.

Dear Reviewer gzhj,

Many thanks for all the invaluable feedback and positive assessment! We really appreciate your expertise and time invested in evaluating our work.

Best regards,

The authors