Zero-shot Quantization for Object Detection

We would like to thank Reviewer R2o8 for the insightful feedback. Your suggestions for clarification and improvement are constructive. Please find the responses below:

Q1: In Section 3.2 and ‘Relabel’ paragraph, The supervision mentioned in formula 5 is done by GT. In my understanding, the role of GT is replaced by the generation of the "teacher" model. In the initial stage, all information such as image and position are initialized as Gaussian noise. How does the teacher model obtain information to complete this optimization?

A1: Thank you for your question. The teacher model is fixed throughout the process and does not obtain any information. Instead, we optimize the initialized Gaussian noise image and GT to obtain the object information from the teacher model, including the object category, the box coordinates, and the category distribution. In Formula 5, x represents the trainable input image, and y is the ground truth label information, which includes label type, size, and location details. ϕ(x) denotes the neural network’s predicted label information for the image.

In the "Relabel" paragraph of Section 3.2, we start by independently sampling a label set 𝑌 for each image based on the sampling method outlined in Table 1, initializing the input image 𝑋 as Gaussian noise. In this context, 𝑋 in Formula 5 represents Gaussian noise, while 𝑌 denotes the initially sampled single label set. We then calculate the gradient of 𝑋 and update it to minimize the loss defined in Formula 5, keeping the label 𝑌 fixed. The terms $L_{BNS}$ and $L_{prior}$ guide 𝑋 toward characteristics of real data used in model training, while $L_{detect}$ drives ϕ(x) to predict labels closer to 𝑌.

After several rounds of updates, 𝑋 starts to acquire basic image features and capture information from 𝑌. At this point, we use a pretrained network to recognize 𝑋 and incorporate high-confidence regions into 𝑌 while removing low-confidence labels, thereby forming an updated Y. This iterative process continuously refines both the image 𝑋 and the label set 𝑌 until convergence. The resulting synthetic images effectively capture real image characteristics from the pre-trained teacher model, providing high-quality synthetic data for QAT training.

Q2: Is it the model itself that is being quantized or is it a model with other requirements?

A2: Thank you for the question. The teacher and student models represent the models before and after quantization, respectively. Thus, the teacher model refers to the quantized model in its full-precision form. A visual representation can be found in Figure 2, where the "Full-precision network" corresponds to the full-precision teacher model, and the "Quantized network" corresponds to the quantized student model. Both models share the exact same network architecture, with the only difference being that the teacher model’s parameters have higher numerical precision.

Q3: It is recommended that the author add a detailed explanation of f_l after formula 7. What specific layers does the author distill?

A3: Thank you for the suggestion. Our key models consist of the CNN-based single-stage object detection model YOLO and the CNN-based two-stage model Mask-RCNN. In these models, the CNN layers are primarily responsible for feature extraction and combination, with parameters that capture substantial information from the features of training images. Similarly, the Batch Normalization (BN) layers retain scaling (γ) and shifting (β) coefficients derived from training data. Therefore, in our experiments, we selected fl to include all CNN and BN layers, allowing for a more comprehensive extraction of information about the real training data from the pre-trained model. Additionally, we observed that choosing fl as only the CNN layers still yielded comparable results, likely because the CNN layers contain significantly more parameters than the BN layers. This suggests that sufficient information can be distilled from the CNN layers alone to enhance the model's performance on real-world images.

Q4: What is the time consumption of generating samples in the first stage?

A4: Thanks for the question. In the initial stage, utilizing 8 RTX 4090 GPUs for image generation allows us to produce 256 images every 20 minutes, resulting in a total of 160 minutes to generate 2,000 images. Additionally, the calibration set we generate effectively captures the overall characteristics of the training set and can be reused multiple times throughout the model's quantization-aware training process. As the number of training iterations increases, our method's acceleration rate can ultimately achieve a 16x speedup during the training phase. We also include the time consumption in Appendix C of the revised version of manuscript.

Thank you for your detailed suggestions and comments. Please find the corresponding responses below:

Q1: In L78 and L79, the paper claims category imbalance in the current detection dataset, can you provide some examples illustrating that the previous method is struggling with it? Also, can you provide visualization/analysis your method is capable of handling the imbalance problems?

A1: Thank you for the suggestion. First, we want to clarify that the current zero-shot quantization studies can only generate balanced datasets, as seen in studies like [2],[3], [4], [5]. However, in real-world scenarios, class distributions in object detection tasks are often highly imbalanced. For example, as illustrated in Figure 1(b), labels such as “person” and “car” are very common, whereas labels like “hair dryer” and “toaster” appear much less frequently. Our method can generate a label distribution that closely matches the original label distribution, as visualized in Figure 1(b). To quantify the difference between label distributions sampled by different methods and the original label distribution, we use KL divergence as a metric (normalizing all distributions so that the sum of label probabilities equals 1).

Table A: Comparison of KL Divergence Across Various Label Distributions

$KL(P_{ours}||P_{data})$ : 0.177

$KL(P_{fp}||P_{data})$ : 0.241

$KL(P_{ms}||P_{data})$ : 0.614

$KL(P_{tile}||P_{data})$ : 0.630

Here, $P_{data}$ refers to the label distribution of the original COCO training set, while$P_{ours}$, $P_{fp}$, $P_{ms}$, and $P_{tile}$ represent the label distributions obtained by sampling 2048 data points using our adaptive sampling method, as well as the false positive sampling, random multi-label sampling, and tile sampling methods from [1], respectively. As shown, our adaptive sampling method achieves the smallest KL divergence compared to all other methods, indicating that it most effectively preserves the characteristics of the original data.

Q2: In L80 and L81, the paper claims "fine-tuning strategy for zero-shot quantized detection network with synthetic calibration data has not been studied". Can the author clarify more about this claim, since [1] also fine-tuning from synthetic data? What is the difference between your fine-tuning approach compared to [1] and how you can improve from [1]?

A2: We appreciate your insightful question regarding the differences in the fine-tuning phase between our work and [1].

This claim highlights that no prior work in the field of object detection has explored network quantization in a data-free setting. Although [1] also uses synthetic data for fine-tuning, there are two key differences in its approach compared to ours.

First, in the framework of [1], the student network remains a full-precision network, with weights updated using full-precision gradient information. In contrast, we train a quantized model through knowledge distillation using Straight-Through Estimator (STE), which is a more challenging approach. Second, we employ more fine-grained distillation information: while [1] relies solely on final logits distillation, we combine feature maps from intermediate layers with the final layer logits for distillation.

In our quantization-aware training setting, using only the fine-tuning method from [1] leads to a notable performance drop, as shown in Table B.

Table B Comparison with Logits Distillation using the same synthetic data (YOLOv5-s on MS-COCO)

We sincerely appreciate Reviewer f6ut for the insightful feedback. Your suggestions and insights are very helpful for further enhancing the submission quality. Please find the responses below:

Q1: The framework combines synthetic data generation, adaptive sampling, and feature-level distillation within quantization-aware training (QAT), creating a complex pipeline. This complexity may hinder practical adoption and implementation in real-world settings.

A1: Thank you for the suggestion. Our experimental setting focuses on quantization-aware training (QAT) of object detection models in a zero-shot scenario. In object detection tasks, relevant information includes label, size, and location distributions. As demonstrated in the ablation study in Table 5, the absence of any distribution information (Gaussian distribution) or the use of only label distribution (as in in-distribution MultiSample and Tile) results in a significant decline in QAT performance. In contrast, our proposed adaptive sampling method effectively extracts all three types of distribution information from the pretrained model. This emphasizes the critical role of our adaptive sampling within the entire pipeline.

Additionally, data generation and knowledge distillation are also standard paradigms in zero-shot tasks, as seen in [1], [2], [3], [4]. We incorporate more fine-grained, feature-level distillation in the knowledge distillation phase, as QAT requires more precise control than full-precision fine-tuning to achieve optimal performance.

Q2: The framework relies heavily on the quality of synthetic data for calibration, which may introduce variability in performance, especially in challenging real-world environments. Ensuring consistency in synthetic data quality is critical for stable results.

A2: Thank you for the suggestion. To further demonstrate the robustness and superior performance of our method, we conducted additional experiments. Starting with Gaussian noise, we utilized our adaptive sampling approach to generate 5,000 synthetic images and randomly selected 2,000 images for each quantization-aware training session. Using YOLO v5-s as the base model, we evaluated performance under various bit-width configurations, repeating each experiment five times. The performance of each run, along with the standard deviation across results, is presented in Table A.

As illustrated in Table A, despite variations in the synthetic datasets used for each run, the results consistently demonstrate stable and reliable performance, underscoring the robustness of our method when applied to diverse synthetic data inputs.

Table A Method Stability: Comparison Using Different Synthetic Data for QAT Training (YOLOv5-s on MS-COCO)

We collected a dataset of 5,000 synthetic samples and randomly sampled 2,000 samples each time to perform QAT training. The results, including standard deviations, are reported under different bit-width configurations. Here, WXAX indicates that both weights and activations are quantized to X bits during the QAT training process.

Reference:

[1] Zhong, Yunshan, et al. "Intraq: Learning synthetic images with intra-class heterogeneity for zero-shot network quantization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

[2] Liu, Yuang, Wei Zhang, and Jun Wang. "Zero-shot adversarial quantization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

[3] Li, Huantong, et al. "Hard sample matters a lot in zero-shot quantization." Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. 2023.

[4] Zhong, Yunshan, et al. "Fine-grained data distribution alignment for post-training quantization." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.