Progressive Exploration-Conformal Learning for Sparsely Annotated Object Detection in Aerial Images

Thank you for your constructive suggestions. Here is our detailed response.

(1) Our proposed conformal pseudo-label explorer and multi-clue selection evaluator are meticulously encapsulated classes that can be directly invoked. In practical implementation, they exhibit excellent user-friendliness, with no challenges to overcome.

(2) Compared with baselines, during the training process, our PECL has a higher time cost due to an iterative detector updating procedure and an extra conformal pseudo-label exploration procedure; however, in the testing process, our method takes the same time as baselines with the same detection network.

(3) Compared with other semi-supervised/sparse-annotated methods, our PECL demonstrates a degree of superiority in the learning process. Specifically, the training time of our PECL, Unbiased Teacher[1], Co-mining[2] methods are 13.35h, 16.74h, 15.42h on the DOTA dataset at 5% label rate. We conduct all experiments on two NVIDIA 2080Ti gpus.

(4) Additionally, we further consider accelerating the convergence speed of the explorer and evaluator by reducing the complexity of the exploratory state and action spaces, as well as the frequency of updating the target network, thereby enhancing overall computational efficiency.

(5) For small and occluded objects in aerial images, our proposed conformal pseudo-label explorer designs exploratory characteristics based on the non-conformity score, which considers the imbalanced probabilities between small and large objects. In future work, we will incorporate the size of objects into the characteristic design to further enhance the performance and robustness of our algorithm.

(6) In order to intuitively reflect that the experimental performance is one of our main contributions, we will revise Lines 71-73 to:``......demonstrate the effectiveness of our PECL, which outperforms the baselines and state-of-the-art methods by at least 5.63%, 1.35%, respectively......’’.

(7) According to our investigation, there are only two semi/sparse-supervised methods for object detection in aerial images in 2024, Pseudo-Siamese Teacher[3](TGRS 2024), S2O-Det[4](TII 2024), and neither of them is open source. We will add them to related work, and track the comparative experiments in the future.

(8) For Section The Proposed Method, it would arouse the readers’ interest to add illustration chart of the proposed SCIR framework.

(9) According to your suggestion, we have presented a diagram of our proposed PECL framework in Figure 4 of overall author rebuttal PDF file.

(10) For minor writing/formatting errors, we will polish this paper in the revision.

Reference

[1] Unbiased teacher for semi-supervised object detection.

[2] Co-mining: Self-supervised learning for sparsely annotated object detection.

[3] Pseudo-Siamese Teacher for Semi-Supervised Oriented Object Detection.

[4] S2O-Det: A Semisupervised Oriented Object Detection Network for Remote Sensing Images.

Thank you for your constructive suggestions. Here is our detailed response.

(1) H2RBox[1] learns object center localization from horizontal box annotations in the weak supervision branch, utilizes scale and spatial constraints to learn object width, height, and rotation angle information in the self-supervision branch. H2RBox-v2[2] utilizes the principle of symmetry to learn flip-rotation consistency, thereby predicting the rotation angle of object. PointOBB[3] and Point2RBox[4] use Multiple Instance Learning and Knowledge Combination, respectively, to learn rotated box regression from single point supervision. We will add more weakly-/semi-supervised aerial object detection methods in the related work of the revision, which are all aimed at alleviating the time-consuming and labor-consuming of rotated annotations.

(2) To prove the generality of our PECL, we try to conduct experiments on general detection dataset, COCO dataset. Following the setting of BRL[7], four sparsely annotated training sets are generated. In Table 1 of overall author rebuttal PDF file, we compare the different methods with our method trained under three annotation sets. It can be seen that under all sparse conditions, our PECL outperforms the Co-mining[8] by 0.2%, 0.6%, 0.8%, respectively, indicating that our proposed method can mine supervised signals effectively.

(3) Thank you for the reminder; we will add more OBB/HBB, full-/weakly-/semi-supervised aerial object detection methods in the related work of the revision.

(4) The visualization of the changes of pseudo-labels during the training process has been presented in Figure 3 of overall author rebuttal PDF file. The results indicate a significant improvement in the quality of pseudo-labels after introducing reinforcement learning.

(5) We encountered some difficulties when applying our PECL to DETR-based detector, e.g. ARS-DETR[9]. Our PECL is effective in pseudo-label exploration on dense object detectors, because a sufficient number of proposals provide enough samples for statistics-based conformal prediction.However, for DETR-based detector, which has sparse proposals, our PECL cannot perform well. This idea is worth further exploration in future work.

(6) For minor writing/formatting errors, we will polish this paper in the revision.

(7) Sparsely Annotated Weakly-Supervised Oriented Object Detection is indeed an interesting and valuable research setting. However, we encountered difficulties when integrating our PECL with H2RBox-v2. Specifically, H2RBox-v2 is a weakly supervised method based on FCOS[5] detector, whose label assignment strategy requires the participation of all points on the feature map to classify each candidate point as a positive or negative sample. Therefore, this strategy requires ground truth boxes to have very high quality. We need to modify the FCOS label assignment strategy to allow for the assignment of ignored samples (similar to Faster R-CNN[6]), in order to reduce dependence on the quality of ground truth boxes. This involves a lot of work, and we have just proposed a preliminary idea that has not yet been implemented. We will implement this in future research work.

(8) Compared with baselines, during the training process, our PECL has a higher time cost due to an iterative detector updating procedure and an extra conformal pseudo-label exploration procedure; however, in the testing process, our method takes the same time as baselines with the same detection network.

Reference

[1] H2RBox: Horizonal Box Annotation is All You Need for Oriented Object Detection.

[2] H2RBox-v2: Incorporating Symmetry for Boosting Horizontal Box Supervised Oriented Object Detection.

[3] PointOBB: Learning Oriented Object Detection via Single Point Supervision.

[4] P2RBox: A Single Point is All You Need for Oriented Object Detection.

[5] FCOS: Fully Convolutional One-Stage Object Detection.

[6] Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.

[7] Solving missing-annotation object detection with background recalibration loss.

[8] Co-mining: Self-supervised learning for sparsely annotated object detection.

[9] ARS-DETR: Aspect Ratio-Sensitive Detection Transformer for Aerial Oriented Object Detection.

Thank you for your constructive suggestions. Here is our detailed response.

(1) Our proposed PECL has improved performance by at least 5.63% compared to supervised baselines. Compared to other state-of-the-art methods, we have also achieved at least 1.35% growth. Given the limited improvement of other methods, this achievement is already quite challenging in the current research field, and it also indicates that our method still has a lot of room for improvement.

(2) Compared to SAOD task in general images, SAOD task in aerial images faces greater challenges.

• Firstly, objects in aerial images have characteristics such as arbitrary orientation, dense distribution, and large aspect ratios. Figure 1 of overall author rebuttal PDF file intuitively counts the number of objects in each image in the DOTA and COCO datasets, which reflects the relatively high density of objects in aerial images. Motivated by this, our proposed PECL utilizes rich contextual information between objects to mine high-quality pseudo-labels.
• Secondly, aerial datasets often exhibit serious long tail problems, resulting in significantly imbalanced prediction results for different classes. Figure 2 of overall author rebuttal PDF file shows the statistical analysis on the selected candidates of each class. As in Figure 2, when the threshold is with 0.9, less positive/negative candidates are selected; when reducing it to 0.5, positive/negative candidates are all increased. This confirms that it is difficult to select pseudo-labels with a certainty threshold. In contrast, our PECL employs reinforcement learning to learn an adaptive policy to decide the confidence of selecting candidates. Therefore, SAOD in aerial images is necessary and challenging.
• The diversity of rotation angles remains another challenge in aerial object detection. The introduction of rotation angles is not merely about adding a parameter to learn. The periodicity of angles leads to the boundary discontinuity and square-like problems[1], causing issues in the regression of rotated boxes and affecting the localization quality of rotated bound boxes. This is also an issue worth investigating in the SAOD task and can serve as the motivation for future work.

(3) For minor writing/formatting errors, we will polish this paper in the revision.

Reference

[1] Detecting Rotated Objects as Gaussian Distributions and Its 3-D Generalization