OD³: Optimization-free Dataset Distillation for Object Detection

Thank you for your valuable suggestions and for giving us the opportunity to address the points you raised!

Q1: The experiment comparison is unfair for the baseline methods (random, uniform, k-center, herding, and DCOD).

We appreciate this concern. Below are our own runs of the core-set selection methods using our framework with 1% IPD:

DCOD [1]: Since their codebase is not publicly available, we contacted the authors to share the codebase or the datasets to facilitate better comparison with their method and exact reproduction of their results. However, they have declined to share them. Also, YOLOv3-SPP (their baseline) outperforms Faster R-CNN 101 (our observer baseline) with reported performance of 44.3% vs. 39.8%, respectively. We believe that the comparison remains fair, as our approach is evaluated under a more challenging setting (lower-performing baseline).

Q2: For the PASCAL VOC dataset, why do the authors only present the results of the proposed method and not include the baselines?

For the PASCAL VOC dataset, we are unable to directly compare against DCOD since they used COCO metrics instead of the standard PASCAL metrics, which does not facilitate a direct comparison with training on the uncompressed dataset. However, we will include the results for the baseline core-set selection methods as follows (1.0% IPD):

Q3: In Table 2, the results are very surprising. Usually, the mAP metric is much smaller than mAP50.

We appreciate the reviewer's keen observation. Upon further inspection, the reported metrics are redundant, and we will keep the mAP@50 column, which reflects the default VOC metric using the area method.

Q4: Lack of ablation study on soft-label generation.

As suggested, we will add the below results of the soft-label ablation:

Q5: How is the storage of soft labels handled? In many dataset distillation works for classification, researchers found that soft labels require significant storage.

Following the approach of RDED [2], we avoid explicitly storing soft labels by using a knowledge distillation framework with a teacher model.

Q6: The algorithm needs clarification. How can a bounding box b add ℓ? Also, $\hat{x}$ is not defined in the algorithm.

We understand the confusion and will re-define the bounding box modification as follows:

$b' = (x - \ell_x, y - \ell_y, w + 2\ell_x, h + 2\ell_y)$

As for $\hat{x}$, we will define it as the final reconstructed image after adding the objects to the canvas.

Q7: In Eq. (2), how does the proposed method represent information density?

The object confidence score normalized by area effectively captures the information density of the canvas. Based on this formulation, we select objects with higher confidence scores. A higher value indicates that the canvas contains more object-related information, while the object size ensures that confidence is measured per unit area.

Q8: The function 'a' in Eq. (2) and Eq. (3) conflicts.

Originally, we used a different symbol in Eq. (3), so to resolve the conflict, we will change 'a' in Eq. (2) to uppercase or adopt an alternative notation for clarity.

Q9: In Eq. (4), the Information Diversity metric counts distinct objects on the canvas. How are distinct objects defined?

As objects are added to the canvas, the method tracks each object {or} separately, even if it intersects with other objects. This ensures that distinct objects are accounted for.

We appreciate your thoughtful feedback and hope these clarifications address your concerns.

References:

[1] Qi, Ding, Jian Li, Jinlong Peng, Bo Zhao, Shuguang Dou, Jialin Li, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Cairong Zhao. "Fetch and forge: Efficient dataset condensation for object detection." Advances in Neural Information Processing Systems 37 (2024): 119283-119300.

[2] Sun, Peng, Bei Shi, Daiwei Yu, and Tao Lin. "On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9390-9399. 2024.

Thank you for your constructive feedback and for giving us the opportunity to address your concerns!

Q1: The AP performance for different object sizes is omitted.

The AP performance for different object sizes is reported in Table 3 and Table 5 of the main paper, as well as Table 7 and Table 8 of the appendix. Since these metrics were not reported for the SoTA method DCOD [1], we did not include them in Table 1 for direct comparison.

Q2: How do detection transformers and more recent detectors perform?

We appreciate this suggestion and agree that evaluating transformer-based models is valuable. To further highlight the generalizability of our approach, we have conducted additional experiments and included the results accordingly.

We hope this clarifies your concerns, and we appreciate your thoughtful review!

References:

[1] Qi, Ding, Jian Li, Jinlong Peng, Bo Zhao, Shuguang Dou, Jialin Li, Jiangning Zhang, Yabiao Wang, Chengjie Wang, and Cairong Zhao. "Fetch and forge: Efficient dataset condensation for object detection." Advances in Neural Information Processing Systems 37 (2024): 119283-119300.

We sincerely appreciate your thorough review and for recognizing our contributions!

Q1: How does this method perform on the latest transformer-based detectors?

We agree that evaluating performance on transformer-based detectors is important. To further demonstrate the generalizability of our approach, we have conducted additional experiments and included the results accordingly.

Q2: Additional ablation experiments regarding the confidence threshold are missing.

Thank you for your suggestion. The ablation study on the confidence threshold is indeed provided in Table 8 of the appendix.

Q3: The dataset distillation technique claims to speed up training while maintaining model performance. Could the authors provide a performance comparison with models trained on uncompressed datasets?

Certainly! The following is the reported performance by mmdetection of the used models on the full versions of MS COCO:

Q4: The performance drop is somewhat significant between distilled training and full-scale training.

The performance drop is 8.9%, from 39% (full-scale with 100% of the original dataset) to 30.1% (distilled with only 5% IPD). With these results, we have successfully bridged around 10% of the gap between the previous SOTA and the theoretical upper bound at a 1.0% compression rate. Similarly, a recent work from the image classification task, which has been more thoroughly explored, is RDED [1]. This method achieved 33.9% accuracy at IPC=10 on ImageNet-100 with ResNet-101 vs training full-scale on the entire dataset achieving 78.25% (gap of 44.35%). Thus, we believe $OD^3$ exhibits a huge step forward toward completely bridging the gap between distilled and full-scale training.

We appreciate your valuable feedback and hope this addresses your concerns.

References:

[1] Sun, Peng, Bei Shi, Daiwei Yu, and Tao Lin. "On the diversity and realism of distilled dataset: An efficient dataset distillation paradigm." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9390-9399. 2024.

Thank you for your thoughtful review and valuable suggestions!

Q1: The evaluation is limited to a single backbone architecture (ResNet-50) and two detectors (RetinaNet & Faster R-CNN), which severely limits its generality. The paper should additionally evaluate the method on more modern backbones (such as ViTs) and detectors (such as DETR).

We appreciate this insight and agree that evaluating performance on transformer-based models is important. To further demonstrate the generalizability of our approach, we have conducted additional experiments and included the results accordingly.

Q2: Eq. 6 has formatting issues.

We will redefine the equation as follows:

$\quad \mathbf{z}_i = f^\textrm{fpn}(f^\textrm{backbone}(\mathbf{x}_i))$

With the updated format:

$\mathcal{L}\textrm{mse} = \mathbb{E}{(\mathbf{x}_i,\mathbf{y}^\textrm{feat}_i)} \Big\Vert \mathbf{y}^\textrm{feat}_i - \frac{\mathbf{z}_i - \textrm{mean}(\mathbf{z}_i)} {\textrm{std}(\mathbf{z}_i) + \epsilon} \Big\Vert_2^2.$

Q3: $x_{i+1}$ is missing superscripts in Eq. 10 & Eq. 12.

The superscripts $^{a}$ and $^{ar}$ will be added to $x_{i+1}$ in the revised version.

Q4: The mmrazor reference renders as "(Contributors, 2021)".

As suggested, we will correct this reference in the revised version.

We appreciate your valuable feedback and hope these clarifications address your concerns.