Target-Guided Adversarial Point Cloud Transformer Towards Recognition Against Real-world Corruptions

We sincerely appreciate your detailed and insightful reviews. We hope our response can address your concerns.

Q1.1: The term "local pattern" is ambiguous and lacks a formal definition. Does it refer to a graph constructed from the Point Cloud set tokens or the importance ranking of the tokens? The paper should provide a clear and formal definition of "local pattern.".

Sorry for the confusion about the definition. Local pattern refers to the geometric structure of a small region within the point cloud, which is captured by a subset of tokens. The clear and formal definition will be included in the final manuscript.

Q1.2: Regarding Figure 1, the confusion matrices in parts (a) and (b) appear indistinguishable to the reviewer. Clarification on their differences would be beneficial.

Sorry for the confusion. We have provided a visualization in the authors' response document that highlights the differences more clearly. The dominant red along the diagonal underscores our approach's superior performance compared to the standard transformer. We will incorporate it in the final manuscript to ensure the distinctions are clear.

Q1.3: The abstract is difficult to understand, particularly in describing the two modules. It requires significant improvement for readability. The description in lines 42-52 is much clearer than the abstract and could serve as a model for revision

Sorry for the confusion about the abstract. As you suggested, we have carefully revised the abstract for readability. The revised abstract will be included in the final manuscript.

Q2: The paper employs the mini-pointnet method to generate Point Cloud tokens. The paper should evaluate the adversarial mechanism using different tokenization methods to validate its robustness.

Thanks for your advice! As you suggested, we compare our method with two alternative tokenization methods: mini-DGCNN and mini-PCT. The results are shown in the table below.

As we can see from the results, the performance of our adversarial mechanism is relatively stable across different tokenization methods. This indicates that our method is not sensitive to the specific tokenization method used and can effectively improve the robustness of point cloud models. Additionally, we would like to mention that the choice of tokenization method may affect the performance of the model on different tasks and datasets. Therefore, we recommend exploring different tokenization methods and choosing the most suitable one for specific applications. We will add the results in the final version.

Q3: The proposed framework appears general and potentially applicable to various methods, which is a positive aspect. Can this method be applied to other state-of-the-art (SOTA) methods by incorporating the "digging sub-optimal patterns" mechanism?

Thank you for your suggestion! As you suggested, we have extended the 'digging sub-optimal patterns' mechanism to two state-of-the-art (SOTA) methods, PointM2AE and PointGPT, on the ModelNet-C dataset. The results are promising and demonstrate the general applicability of our approach.

As shown in the table below, incorporating our 'digging sub-optimal patterns' mechanism into PointM2AE and PointGPT resulted in significant reduction in mCE scores. These results suggest that our approach can effectively enhance the robustness of various point cloud recognition models. By encouraging the model to explore and utilize a broader range of patterns, our method enables the models to better generalize to corrupted data.

Q4.1: The notations used in the focal tokens identification section are unclear. The paper uses m = 1, ..., D when introducing the F_topk, which is a R^{k \times C} matrix. Is D equivalent to C? The reviewer did not find a definition for D.

Sorry for the confusion. This is a typo. The variable D in F_topk should indeed be equivalent to C, which represents the number of feature channels in the tokens. We will revise the paper to correct this error and ensure consistency in our notation.

Q4.2: Additionally, M in equation (2) is introduced as a vector rather than a matrix.

You are correct that M in equation (2) is a vector rather than a matrix. We will revise the equation and the corresponding text to reflect this correction.

Q4.3: The process for selecting the top-k tokens is not adequately described. Detailed explanation on how these tokens are chosen should be included.

Sorry for the confusion. We will provide a more detailed explanation of how the focal tokens are selected.

1. Feature Response Calculation: For each token, we compute the feature response for each channel, with the help of auxiliary supervisory process. This involves assessing how strongly each token responds in each of the C channels.

2. Sorting Tokens by Channel Responses: Once the feature responses are calculated, we sort the tokens based on their response values within each channel. This step ensures that tokens with higher responses are ranked higher.

3. Selecting Focal Tokens: After sorting, we select the top k tokens for each channel. This selection is done by choosing the k highest-ranked tokens based on their feature responses in each channel.

By following these steps, we ensure that only the most significant tokens, in terms of their feature responses, are retained for further processing.

For further clarity, we kindly refer you to the A.6 part of the Supplementary Material, where we provide pseudo-code of focal tokens identification process.

We sincerely appreciate your detailed and insightful reviews. We hope our response can address your concerns.

Q1: Augmentation methods like PointMixUp and PointCutMix can improve the robustness, the experiments should be performed.

Thanks for your advice! As you suggested, we further evaluate the performance of our approach with various data augmentation methods on ModelNet-C dataset, the results are as follows. Beside the discussed PointMixup [1] and PointCutMix [2], we additionally incorporate experiments with the data augmentation techniques PointWOLF [3], RSMix [4], and WOLFMix [5].

Among these, PointMixup, PointCutMix and RSMix fall under the category of mixing augmentation, where they mix several point clouds following pre-defined regulations. PointWOLF pertains to deformation techniques that non-rigidly deforms local parts of an object. WOLFMix combines both mixing and deformation augmentations, which first deforms the object, and subsequently rigidly mixes the deformed objects together.

As shown in the table, data augmentation methods further improve the robustness of our method against point cloud corruptions. Employing mixing or deformation data augmentation techniques independently can enhance the robustness of the model, e.g. the results of our model with PointWOLF (67.0% mCE) and with PointMixup (66.2% mCE). When these two techniques are combined, as in WOLFMix, the robustness of the model is further augmented (64.7% mCE). Additionally, these experiments demonstrate the compatibility of our method with various data augmentation techniques, further underscoring its potential in addressing data corruption. We will add the results in the final version.

Reference:

[1] Pointmixup: Augmentation for point clouds. ECCV 2020.

[2] Pointcutmix: Regularization strategy for point cloud classification. Neurocomputing 2022.

[3] Point Cloud Augmentation With Weighted Local Transformations. ICCV 2021.

[4] Regularization Strategy for Point Cloud via Rigidly Mixed Sample. CVPR 2021.

[5] Benchmarking and Analyzing Point Cloud Classification under Corruptions. ICML 2022.

Q2: Similar previous works like PointDP are not compared.

Thanks for your advice! The primary objective of our approach is to enhance the model's robustness against real-world corruptions. Consequently, most of our experiments are centered around this goal. The experimental results presented in the paper demonstrate the effectiveness of our method in achieving this objective.

Improving the model's defense against point cloud attacks is a secondary goal. We made significant efforts to include comparisons with relevant methods such as PointDP and IF-Defense. However, we were unfortunately unsuccessful in these attempts.

1. As PointDP is not an open-source model, we were unable to obtain its implementation to evaluate its performance in comparison to our baseline method. Therefore, we could not include a direct comparison within the constraints of this submission.

2. Additionally, due to time limitations, we were unable to conduct experiments on other related method such as IF-Defense in time. However, we are committed to addressing this in the final manuscript and will make every effort to include these comparisons.

However, during rebuttal, we have conducted some extending experiments on ModelNet40-C dataset. We kindly refer you to our response to Q2 of Reviewer K9Vn for details.

We appreciate your understanding and will strive to improve our manuscript based on your valuable feedback.

Q3: Some typo errors in equation 3.

Sorry for the typo errors. We have carefully revised the paper to fix all typos.

Q4: How about the performance on SOTA MVImageNet dataset?

Thanks for your advice! It is very necessary to evaluate APCT in more challenging MVImageNet [6] dataset.

In our paper, we have experimented on five datasets of different tasks, i.e., ModelNet-C and ScanObjectNN-C (classification against corruption), ShapeNet-C (part segmentation against corruption), ScanObectNN (classification), ModelNet (attack defense). On different benchmarks with various domains, our APCT can attain competitive performance to existing specialist models.

As you suggested, we further evaluate the performance of our approach on one additional dataset Mvimgnet [6]. It is a challenging benchmark for real-world point cloud classification, which contains 64,000 training and 16,000 testing samples. As shown in the table, our approach achieves 86.6 OA and still exhibits good generalization capacity in real-world scenarios.

Reference:

[6] Mvimgnet: A large-scale dataset of multi-view images. CVPR 2023.

We sincerely appreciate your detailed and insightful reviews. We hope our response can address your concerns.

Q1: Disscussion about the impact of the proposed method on computational overhead could be important for practical implementations.

Thanks for your advice! It is very necessary to discuss the computational overhead of the proposed method. As you suggested, we discuss the impact of the proposed method on computational overhead as follows, including memory, training and inference speed. Experiments are conducted on one GeForce RTX 3090.

As seen in the table, our method incurs no additional memory overhead. When compared with the baseline technique, our method brings a slight decrease in training speed (~ 6% delay) and inference speed (~ 6% delay), while delivers a significant 4.0% mCE reduction.

Q2: Results on the mentioned ModelNet40-C dataset.

Thanks for your advice! As you suggested, we further evaluate the performance of our approach on ModelNet40-C [1] dataset. It is a comprehensive benchmark on 3D point cloud corruption robustness, consisting of 15 common and realistic corruptions.

As shown in the table, our approach exhibits remarkable robustness on the ModelNet40-C dataset, it beats the superior PCT by 1.4 and achieves the ER_cor of 24.1. The results on ModelNet40-C demonstrate that our APCT has excellent robustness to various point cloud corruptions. We will add the results in the final version.

Reference:

[1] Benchmarking and Improving Robustness of 3D Point Cloud Recognition against Common Corruptions. arXiv 2022.