Response to Reviewer dthJ

We appreciate your recognition of our work. Your insightful suggestions help make our sensitivity analyses more comprehensive and convincing. In addition to the response to your valuable feedback, we conducted further explorations on rendering quality and input resolution (Q1 and Q2 responses to Reviewer 1QVN).

Q1: Insights into the rendering angles and different numbers of views

PointAD interprets point clouds from their 2D renderings, where rendering angles and numbers collectively determine the amount of information derived. They have different emphases. For rendering angles, the importance lies in the discrepancy between adjacent angles, as this affects the information granularity that PointAD obtains from adjacent views. When the angle discrepancy is fixed, the number of renderings determines the coverage of 3D information in the resulting 2D renderings. To capture all point cloud information, especially abnormal points, it is crucial to ensure comprehensive coverage. Therefore, our approach in selecting rendering angles and the number of renderings is to guarantee that all points in point clouds are adequately represented.

Based on this principle, we conducted experiments to assess the impact of the number of renderings on PointAD's detection performance, circularly rendering point clouds to ensure even coverage of all points. As shown in Table 5 of the manuscript, increasing the number of views allows PointAD to gather more detailed information from the 2D renderings, benefiting from smaller angle discrepancies, which improves detection and localization results. However, when the number of views increased from 9 to 11, we observed a performance decline in PointAD, with the I-AUROC for global multimodal detection dropping from 87.4% to 86.4%. This suggests that incorporating too many views could introduce redundant information, resulting in 2D renderings with extensive overlap and excessive local detail. This overemphasis on local information can impede global recognition. Hence, the appropriate number of views benefits point understanding from informative views while mitigating the adverse effects of redundant local information.

To further explore the impact of the absolute angle, we shift the rendering angles while keeping the angle discrepancy unchanged. The original adjacent angle discrepancy in our paper is $\frac{1}{5}\pi$. We divide this discrepancy into four parts and perform angle shifts of $\frac{1}{20}\pi$, $\frac{2}{20}\pi$, and $\frac{3}{20}\pi$ to test the impact of varying rendering angles. Table 6 shows that PointAD maintains consistent performance across different rendering angles, demonstrating its robustness to variations in angles different from those used during training.

Q2: The robustness under varying lighting and occlusions.

Further exploring the robustness of PointAD under different conditions could enhance its generalized detection performance. We conducted ablation studies to test its sensitivity under different lighting conditions and with occluded point clouds below.

To evaluate the impact of rendering lighting, we adjusted the lighting conditions to render point clouds, generating variant datasets with different lighting. We used both stronger and weaker lighting to render point clouds compared to the original dataset, covering a broad lighting range. We denote stronger and the strongest lighting as "+" and "++", and weaker and the weakest lighting as "-" and "--". Visualizations of the resulting samples are presented in Figure 5. The experiments were conducted on MVTec3D-AD, where we tested PointAD, trained on the original dataset, on these lighting variant datasets. Table 7 shows that PointAD can still detect anomalies even with significant discrepancies in rendering lighting, suggesting that PointAD is not sensitive to variations in rendering lighting.

Next, we evaluated detection performance with occluded point clouds. We occluded point clouds by removing those invisible from a specific rendering angle and then used the same rendering parameters to project the remaining points. During this process, we observed that abnormal regions might be occluded totally. Unlike class classification, where class semantics remain unchanged by point occlusions, removing these anomaly semantics would transform anomaly samples into normal points. Therefore, we selected rendering angles that allow visibility of part or all anomalous regions when the point cloud is an abnormal instance. The occluded point clouds are shown in Figure 6. We then used PointAD, trained on the original dataset, to test the resulting occluded dataset. Table 9 shows that PointAD suffers from performance degradation when the point clouds are occluded. We attribute this to two aspects:

1. Despite this strategy, the occluded point clouds could still lose part of the anomaly semantics;

2. Occluded point clouds could create unexpected sinkholes on the surface, which may cause PointAD to identify these areas as hole anomalies incorrectly.

Q3: Experiments on additional datasets

please refer to the bottom of Response to Reviewer GgEU for our response to Q3 due to character limitation.

Q4: Discussion of potential limitations and future work.

We suggest potential improvements for PointAD as follows:

1. The current rendering process in PointAD does not involve optimization. Using fixed rendering angles for diverse objects might limit detection performance. Adapting rendering angles conditioned on specific objects could make 2D renderings more informative.
2. PointAD currently treats the 2D renderings as independent instances. Modeling the correlations between multiple 2D renderings based on view angles could enhance both 2D and 3D representations.

If our responses address your concerns, we would appreciate it if you could consider raising your score to acknowledge our efforts in addressing your questions.

Your support is greatly appreciated. We're more than happy to take any further questions if otherwise.

Response to Reviewer GgEU

Thanks for acknowledging our work. Your insightful comments help highlight our technological contributions in comparison to concurrent works and further promote comprehensive evaluations.

Due to the character limit, please refer to the bottom of General Response for our reply to Q1.

Q2: Please compare with part of them on experiments.

We have included these simplified illustrations in the revised version. For experimental comparisons, since the source codes for PromptAD, CLIP3D-AD, IMRNet, and M3DM-NR are not available, we compared PointAD's performance with WinCLIP and VAND (AnomalyCLIP has been incorporated in the current submission). Note that VAND and CLIP-AD are similar methods, and we selected VAND for comparison due to time constraints. Table 4 shows that PointAD significantly outperforms these baselines across all datasets in the one-vs-rest setting. The consistent superiority across diverse baselines demonstrates the effectiveness of PointAD. Due to the page limitation, we will provide results for the cross-dataset setting in the revised version.

Q3: Does PointAD still hold the merit of computational efficiency when compared to other methods such as AnomalyCLIP and Cheraghian?

Thanks for pointing out your concern. We have supplemented the computation overhead of WinCLIP, VAND, and AnomalyCLIP. In addition, as you recommended in Weakness Section 2, we report the computation consumption of unsupervised methods such as CPFM and 3DSR. From Table 5, we can observe that PointAD consumes medium computation overhead to achieve the best results. A lightweight version of PointAD is a worthwhile direction to explore for future work.

Q4: This separation can disrupt the reader's flow and understanding.

Thanks for your detailed feedback. We agree that the incomplete figure captions could confuse readers. We will supplement the necessary captions to all figures in the manuscript to enhance clarity.

Q5: Could you elaborate on the multi-view rendering process of point clouds (as described from line 134 to line 140), particularly how the color of the rendered images is determined ?

Given the point clouds and their corresponding point-level labels, we first resize both the point clouds and labels to 336 × 336. To represent the 3D point clouds in multiple 2D renderings, we use the Open3D library to render the point clouds from various views in a circular manner to preserve 3D information. We use gray ([0.7, 0.7, 0.7]) as the rendering color for all objects in our experiments for two main reasons:

1. Prominent colors might introduce a correlation between anomaly semantics and color information, potentially reducing PointAD's generalization capacity. Using gray helps mitigate this color-induced bias in detecting 3D anomalies.

2. Gray effectively represents 3D anomalies in 2D renderings, facilitating PointAD’s ability to capture the corresponding anomaly semantics.

PointAD requires corresponding 2D labels to explicitly align 3D and 2D anomalies. Therefore, we color the 3D point clouds according to the point labels, with normal regions marked as 0 (black, including the background) and abnormal regions as 1 (white). We then render these from the same angles to obtain the corresponding 2D pixel labels.

Q6: Given the description at line 564, which refers to 2D RGB information, how is this 2D data integrated with the rendered multi-view images? Are there any strategies which are employed to bridge the gap between the rendered training data and the RGB information used in testing?

Thank you for raising your concern. Since 2D RGB data includes the RGB information of all points, we directly feed this 2D data into PointAD to obtain both the RGB anomaly scores (global) and the anomaly score maps (local). As shown in Figure 4, the RGB information can be considered as the additional 2D rendering that includes RGB information of all points. Unlike 2D RGB information, 2D renderings from different views represent only partial information. We need to aggregate these renderings to generate the point anomaly scores and point anomaly score maps. For fusion, we combine the RGB point anomaly score and the anomaly score maps using a weighted average.

We do not use any additional strategies for fusing RGB and point clouds. The effective fusion is due to the explicit alignment of 3D and 2D spaces, which allows PointAD to learn from both point and pixel anomalies collaboratively. This approach captures generic abnormal patterns that are independent of object class semantics and colors, enabling PointAD to generalize well to RGB information during testing. It is worthwhile to explore a fusion mechanism to further bridge this gap. One potential approach could be to introduce 2D rendering information as a condition for representation learning, which might improve PointAD’s generalization to unseen class semantics.

We appreciate your acknowledgment of our work. If our responses satisfy you, we would be grateful if you could consider improving your scores.

Due to character limitation, we continue the response to Q3 for Reviewer dthJ here.

Q3: Experiments on additional datasets

Since 3D anomaly detection is an emerging field, publicly available 3D datasets are limited. Fortunately, we have found a newly available industrial dataset, Anomaly-ShapeNet. However, we fail to find a point cloud dataset related to medical diagnostics and plan to incorporate such data when it becomes available.

Anomaly-ShapeNet is a challenging dataset, comprising 1600 point clouds from 40 classes with 6 anomaly types, without RGB information. We conducted a one-vs-rest setting, using ashtray0, bottle, and cup0 separately to train PointAD. The averaged results on the remaining classes are reported in Table 9. The results indicate that PointAD consistently achieves superior performance by a significant margin compared to other baselines.

Q2: Compare PointAD with those new methods on MVTec-3D AD

We find SOTA unsupervised methods (i.e., M3DM [1], BTF [2], and SDM [3]), which provide the codes. We are pleased to compare these unsupervised methods as the upper boundary of performance. Therefore, we follow their original settings, training category-specific models to report their results. Additionally, we also include them in the comparison of computation overhead. These results will be incorporated into the revised version.

From Table 1, we can observe that PointAD even outperforms the unsupervised method on certain metrics. Specifically, PointAD outperforms BTF in global point detection, achieving 82.0 I-AUROC and 92.4 AP, compared to BTF's 76.3 I-AUROC and 91.8 AP. Besides, PointAD obtains higher P-AUROC for local point detection compared to M3DM, with scores of 95.5 vs. 92.8. It also surpasses SDM with a P-AUROC of 97.2 vs. 93.3 in the local multimodal detection. Although there is still room for improvement compared to these unsupervised methods, these results demonstrate the superiority of PointAD.

Table 2 demonstrates that PointAD strikes a balance between computational overhead and performance. However, further reducing memory consumption remains a potential area for improvement.

If you have any further concerns or questions, please let us know. We are more than happy to address them.

References:

[1] Wang, Y., Peng, J., Zhang, J., Yi, R., Wang, Y., & Wang, C. (2023). Multimodal industrial anomaly detection via hybrid fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8032-8041).

[2] Horwitz, E., & Hoshen, Y. (2023). Back to the feature: classical 3d features are (almost) all you need for 3d anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2968-2977).

[3] Chu, Y. M., Liu, C., Hsieh, T. I., Chen, H. T., & Liu, T. L. (2023, July). Shape-guided dual-memory learning for 3D anomaly detection. In Proceedings of the 40th International Conference on Machine Learning (pp. 6185-6194).

Response to Reviewer 1QVN

Thank you very much for your acknowledgment and insightful comments. Your comments inspire us to explore the robustness of PointAD from the rendering process and input under different conditions.

Q1: If the renderings are of poor quality, could the model's performance degrade?

Thank you for pointing out your concerns. PointAD interprets point clouds through their corresponding 2D renderings, and the quality of these renderings impacts the information that PointAD can extract from the original point clouds. In our manuscript, we used the Open3D Library to render the point clouds, but it does not provide an API for controlling rendering quality. To simulate varying rendering quality, we applied Gaussian blur with different extents ($\sigma$) to the 2D renderings. Sample visualizations are included in Figure 1. Specifically, we conducted experiments on MVTec3D-AD using different blur sigma values (i.e., $\{1, 5, 9\}$). Table 1 shows that the detection performance of PointAD diminishes as rendering quality decreases (with increasing sigma). However, the degradation is acceptable even when the renderings are heavily blurred (sigma equals 9). In such cases, PointAD still outperforms baselines that use high-quality renderings.

Q2: How effective is this method in detecting anomalies in low-resolution, sparse scans?

We appreciate your detailed feedback, which has been invaluable in refining our work. To create low-resolution point clouds, we downsample the entire high-resolution point clouds using Farthest Point Sampling (FPS) with various sampling ratios. This strategy allows us to generate corresponding low-resolution datasets for training and evaluating PointAD. Visualizations of these datasets are provided in Figure 2. We train PointAD using the resulting low-resolution samples and test PointAD on the same resolution.

Table 2 demonstrates that PointAD maintains a strong detection capacity for low-resolution point clouds when the downsampling ratios are 20%, 30%, 50%, and 70%. Even at 20% resolution, PointAD still achieves state-of-the-art performance. This indicates that PointAD is generally applicable to point clouds with various resolutions.

Q3: How significantly does the noise affect the recognition on the point score map?

Thanks for your insightful comments. We agree that failure detection could arise from unusual point density and distribution. To demonstrate the effect of point density, we randomly select normal regions of point clouds and subsequently increase or decrease the density of these regions through upsampling and downsampling. We provide a qualitative analysis in Figure 3. The visualization shows that PointAD effectively resists noise at reasonable levels. However, when noise levels are extremely low or high, the corresponding regions become excessively sparse or dense. This causes normal regions to appear similar to hole anomalies or squeezed anomalies, leading PointAD to classify these noisy areas as anomalies.

Q4: When anomalies are located in areas where point density or distribution are inconsistent or significantly different from the training data, can this model still generalize well to new, unseen anomalies?

To investigate the impact of point density differences between the training and test datasets, we train PointAD using high-density point clouds (original dataset) and then test it on the low-density versions of the datasets, downsampled as described in Q2. Table 3 shows that PointAD can still detect anomalies even when retaining 50% of the points from the original point clouds. However, when more points are removed (30% and 20% sample ratio), PointAD experiences an obvious performance degradation. We attribute the misdetection to the overly sparse point clouds forming holes. Nevertheless, PointAD can still detect anomalies even with a significant gap in point density between the training and test domains (e.g., 100% vs. 20%). This demonstrates PointAD's ability to generalize across different point densities.

In summary, the experimental results demonstrate PointAD's robustness to diverse rendering conditions. We attribute this robustness and generalization to PointAD's efficient hybrid representation learning and CLIP's powerful feature extraction. If you are satisfied with our responses, we would greatly appreciate it if you could consider raising your score. Your support is very important to us.

Dear reviewers,

As the reviewer-author discussion period is about to end by 8.13, please take a look at other reviewers' reviews and authors' rebuttal at your earliest convenience. It would be great if you could ask authors for more clarification or explanation if some of your concerns are not addressed by the rebuttal.

Thanks,

AC