Towards Learning Group-Equivariant Features for Domain Adaptive 3D Detection

We thank the reviewer for the constructive assessment of our work. In the following, we address the concerns point by point. Please feel free to use the discussion period if you have any additional questions.

1. Lack of variety? We address the multi-class adaptation in the main author rebuttal. In addition, as you suggested, we uploaded additional visualizations and performance with other base detectors in Figure A2 and Table A3.

2. What are the voxel size, model size, and running speed? The voxel size is set following the comparing methods ST3D and DTS, which depend on the adaptation task. Please refer to Table A4 in the rebuttal pdf for the model size and running speed.

We would like to add more discussion on a few more details, as official comments are available for discussion during the whole author-reviewer discussion phase. Here are a few more details that we wish to clarify based on the reviewer's questions and concerns.

3) Have you tried different 3D detectors like CenterPoint and recent SOTA detectors? Did the proposed model go through apple to apple check?
In the main paper, we have reported the result with mainly Second-IoU [37] and PointPillars [9] as the base detectors for the fair comparisons with the existing works, where all the configurations, such as voxel size, learning rate, number of filters, thresholds, etc, for the adaptation stay the same, as mentioned in Sec. 4.2 of the main paper. As one of the other famous base detectors, we have included additional results with CenterPoint, as the reviewer suggested, and DSVT[32] in Table A3 for the NuScenes to KITTI adaptation task. We selected DSVT as one of the most recent base detectors due to the fact that it is designed for LiDAR only, which is compatible with our purpose and is one of the top-performing methods. Specifically, Table A3 shows the apple-to-apple comparison of DTS and our proposed method given the same base detectors, CenterPoint and DSVT backbone, followed by CenterPoint Head. Interestingly, using CenterPoint Head leads to worse results compared to anchor-based methods, such as Second-IoU, when we directly use the model only trained on the source domain to target without self-training. We think this is because the anchor-based methods inherently encourage the spatial voxel features (F_{bev}) to learn the bigger surrounding context with supervision to predict IoU between boxes, rather than center position-based supervision from CenterPoint. Nevertheless, our proposed method constantly outperforms the SOTA method for domain adaptive detection, DTS, under the same setting as the base detectors. Also, changing the base detectors to the more recent method, such as DSVT, leads to more improvement for adaptation with around 1.89 gain in AP_{3D}, proving the applicability of our method.

4) Performance Gain by model size?
Table A4 shows the runtime analysis for inference speed and model size on an NVIDIA A10 GPU. Here, on KITTI validation set, we measured the inference speed of three models, namely ST3D, DTS, and ours. As can be seen, the current SOTA DTS requires around 43% more memory while underperforming than GroupExp-DA, suggesting that the improvement made by our proposed method is not simply based on the increment of model size. The inference speed of our method is among the slowest. However, we would like to note that the speed has comparably more room for improvement than the precision without sacrificing model size by using optimized software such as onnxruntime, tensorRT, etc.

5) More discussions on multi-class adaptation settings.
We have conducted comprehensive experiments designed for other categories of objects, such as pedestrians and cyclists. This can be found in the main author rebuttal for all reviewers. As you kindly suggested, we added more visualizations focusing on cyclists and pedestrians in KITTI validation set, which can be found in Figure A2.

6) Code is not provided.
We will release the code and checkpoints as soon as the paper gets accepted.

7) Missing authors of [39].
Thank you for letting us know about the reference issue. We will fix it in the final version.

Again, we thank the reviewer for constructive and valuable comments on our paper. We will make sure to update the main paper based on the additional results acquired based on your comments. Please let us know if you have any further comments or questions.

Dear Reviewer mXQC,

We sincerely appreciate your efforts in providing constructive and professional reviews. During the rebuttal period, we have tried our best to provide feedback and rebuttal to address your concerns. All of them will be involved in our final revised version.

If you have any further concerns, we are more than happy to discuss more before the author-reviewer interactive discussion period ends. Thank you again for your insightful and meaningful comments.

Best regards, Authors of Paper 9774

We thank the reviewer for the constructive assessment of our work.

We address the concern you raise in the main author rebuttal. In particular, Table A2 shows the experimental results for the multi-class adaptation setting.

Please feel free to use the discussion period if you have any additional questions.

We would like to add more discussion on a few more details, as official comments are available for discussion during the whole author-reviewer discussion phase. Here are a few more details that we wish to further clarify based on the reviewer's questions and concerns.

1) Multi-class adaptation experiment and ReDB.
As mentioned in the main rebuttal for all reviewers, Table A2 shows performances of ST3D, ReDA, GroupExp-DA without ReDA, and GroupExp-DA with ReDA for NuScenes to KITTI adaptation. For a fair comparison, we brought the result reported from the ReDB paper. For ours with ReDA implementation, we simply add cross-domain examination, diversity sampling, and class-balanced self-training available from the GitHub repository of ReDB, which are proposed for multi-class adaptation settings. Other configurations also follow ReDB's configuration using Second-IoU as the base detector. Interestingly, for ReDB + GroupExp-DA, we observe that the performance improvement from the ReDB model is more significant for 'cyclist', which is 8% and 5.67% for $AP_{BEV}$ and $AP_{3D}$, compared to 1.1% and 1.7% for 'car', suggesting a potential synergy of the two methods for more challenging class, cyclist. The straightforward explanation would be that the focuses of ReDB and GroupExp-DA are different in making improvements, which leads to additional performance boosts when they are both used together for improving different aspects. We are further investigating this to get more intuition.

2) New Table for multi-class adaptation setting.
We agree with the reviewer's point of view that ReDB needs to be compared differently from other baselines that focus on the single-class adaptation setting. We will separate ReDB from the current Table 1. Specifically, we will create another table focusing on the multi-class adaptation setting with proper performances of other baseline methods from the ReDB paper as a more challenging yet closer to real-world scenarios. Table 1 will remain for performance comparison with the single-class adaptation-based methods without ReDB.

We appreciate the reviewer for the constructive and valuable comments on our paper. We will make sure to update the main paper based on the additional results acquired from your comments. Please let us know if you have any further comments or questions.

We thank the reviewer for the constructive comment on our work.

We would like to let you know that the concern that you raised is addressed in the author rebuttal, which is visible to all reviewers. In particular, Tables A1 and A2 in the uploaded rebuttal pdf show the experimental results for other categories for NuScenes $\rightarrow$ KITTI adaptation tasks. Please feel free to use the discussion period if you have any additional questions.

We would like to add more discussion on a few more details, as official comments are available for discussion during the whole author-reviewer discussion phase. Here are a few more details that we wish to further clarify based on the reviewer's questions and concerns.

1) Generalization ability to other categories.
As mentioned in the main rebuttal, we have additionally conducted comprehensive experiments for other categories, such as pedestrians and cyclists, following the reviewer's comments, which are available in the uploaded PDF. Initially, we focused on only 'Car' category, as it is still challenging for domain adaptative detection that most existing works on single-class adaptation focus on. As can be seen in both Tables A1 and A2, the proposed GroupExp-DA shows consistent improvements from the baselines, demonstrating its ability to generalize to other categories. Although the APs are still the lowest among all categories, we observe an interesting performance improvement from the current SOTAs for 'Cyclist' category, which are 4.58/%/7.06% for the single class adaptation and 8% and 5.67% for the multi-class adaptation settings. We think this might be because each group in GroupExp-DA focuses on objects with different features, which are beneficial for generalization when the number of available labels in the source domain is significantly smaller. For example, in NuScenes, the number of labels for 'cylist' class, which are the sum of 'bicycle' and 'motorist,' is 11,538, while 'car' and 'pedestrian' have 27,449 and 22,447 labels, respectively. The imbalance is more significant in KITTI where the number of labels available for 'Cyclist' is 734 while 'Car' and 'Pedestrian' are 14,357 and 2,207, respectively.

We appreciate the reviewer for the constructive and valuable comments on our paper that provided us with new insights. We will make sure to update the main paper based on the additional results acquired from your comments. Please let us know if you have any further comments or questions.

Thanks for the strong rebuttal. My concerns have been addressed, and I will raise my rating to weak accept. Please include the content in rebuttal to the next revision.

We thank the reviewer for the constructive assessment of our work. In the following, we address the concerns point by point. Please feel free to use the discussion period if you have any additional questions.

(1) How does the proposed method correct the bias in self-training? We would like to let you know that we address the concern in the author rebuttal. The grouping and explorative group update encourage objects with similar features to be close to each other in the feature space. This leads each group to focus on objects with different features, whereas previous works are prone to be biased by objects with globally dominant features, as they are similar to having one group. As the self-training is designed to collect pseudo-label using highly confident detections, many groups with different focuses are able to find larger variation of objects with different features, alleviating the bias problem. Figure A1 demonstrates the concept on NuScenes. Here, we extract each object feature using ground truth object boxes with RoI pooling from $F_{bev}$ for DTS and $F_{cbev}$ for ours. The left figure shows that for all methods, the recall improves for the object with features similar to those of the group center. The figures in the middle and right show that multiple groups are able to discover more objects with variety.

(2) Hyperparameter selection is addressed in the main author rebuttal.

(3) Training pipeline is added in the rebuttal pdf that we uploaded.

We would like to add more discussion on a few more details, as official comments are available for discussion during the discussion phase. Here are a few more details that we wish to further clarify based on the reviewer's questions and concerns.

1) Clarification on overcoming the bias in self-training and Figure A1.
As mentioned in the rebuttal, our key insight is that each group distributes the focus of RPN to objects with different features, side-stepping the bias caused by globally dominant features of objects, as can be seen in Figure 1 of the main paper. In order to further demonstrate this bias problem, we attempt to show that objects in the target domain with similar features as a dominant feature in the source domain are likely to be detected more and used as pseudo labels, causing bias in self-training as follows. To find the dominant feature and compare the similarity between the dominant feature and each object, we first acquire the object feature using the spatial feature, which is $F_{bev}$ for DTS and $F_{cbev}$ for ours, with RoI Pooling using ground-truth boxes in the source domain. The dominant feature is then acquired by averaging all the object features, assuming that outnumbering dominant features will naturally be reflected in the mean. Figure A1 (left) shows recall with respect to the distance between the dominant feature from the source domain and each object feature in the target domain (NuScenes) after self-training. As can be seen, it is clear that all methods exhibit higher recall for the objects that are closer to the dominant features, the centers acquired by averaging, and the recall decreases for objects that have less similar features to the dominant features. Here, similar to the source domain, we acquired the object feature in the target domain by utilizing the ground-truth boxes and spatial features for the analysis. Figure A1 (middle) shows the number of objects depending on the similarity between the center and object feature in the target domain. Based on the observation from Figure A1 (left), we can conclude that having more number of objects close to the center in Figure A1 (middle), such as R1 and R2 in x-axes, means the detector is less biased. As can be seen, our grouping strategy leads to more samples in R1 and R2, suggesting that the proposed GroupExp-DA trains the base detector with fewer biases compared to the current SOTA, DTS, for domain adaptive detection. Also, the base detector trained with both the grouping and explorative update consistently shows better performance (green bar) than the base detector trained only with the grouping with an update using all available labels (orange bar), highlighting the contribution of the explorative group update for self-training. Figure A1 (right) shows how each group contributed to the formation of the green bar (Ours+Group.+Exp.Up.) in Figure A1 (middle).

2) Is $\mu$ directly updated by gradient back-propagation?
$\mu$ is indirectly updated by gradient back-propagation. In a batch, it is the mean of object descriptors (MLP + Global max pooling in Sec. 3.2) that belong to the same group. This batch-wise mean $\hat{\mu}$ is linearly combined (moving average) to existing $\mu$ to preserve the gradient flow from the object descriptor extraction module in Sec 3.2 for learning. Therefore, since the group equivariant spatial feature uses $\mu$ for Regional Proposal Network (RPN), the object descriptor extraction module is updated using the gradient calculated from $L_{det}$ with additional group contrastive loss $L_{att}$ and $L_{repel}$ to produce better object descriptors, which are used to update \mu. This process is now more clearly summarized in the Algorithm table in the rebuttal pdf, as you kindly suggested. In addition, the contribution of each loss can also be found in Table 2 of the main paper.

3) What is the closed gap in Table 1?
Closed gap is the metric proposed in ST3D to show how much performance gap is closed between Source Only model, which is trained only on the source domain and tested in the target domain, and Oracle model, which is trained on the target domain with the ground truth. It is calculated as follows: Closed Gap = (AP_{model} - AP_{source only}) / (AP_{oracle} - AP_{source only}). Here, AP_{model} is the Average Precision (AP) calculated using the tested model's prediction. Similarly, AP_{source only} and AP_{oracle} are AP predicted using the source-only and oracle models. We will clarify this in the main paper. We appreciate your comment.

4) Algorithm.
The training pipeline is now summarized into an Algorithm table in the uploaded rebuttal pdf, as you kindly suggested. For the inference, only lines 5,6 and 7 in the table are used in the pipeline.

We are grateful for the reviewer's constructive comments on our paper. We will update the main paper based on the results from your comments. Please let us know if you have any further comments or questions.

Dear Reviewer mHQJ,

We sincerely appreciate your efforts in providing constructive and professional reviews. During the rebuttal period, we have tried our best to provide feedback and rebuttal to address your concerns. All of them will be involved in our final revised version.

If you have any further concerns, we are more than happy to discuss more before the author-reviewer interactive discussion period ends. Thank you again for your insightful and meaningful comments.

Best regards, Authors