SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning

We sincerely appreciate your insightful and constructive review. We have carefully taken your insights into account and addressed each point. Please kindly refer to the revised manuscript, where the specific updated parts are discussed in detail within the following answers. We marked added or modified parts as blue for reviewers’ convenience.

[W1] Header of Table 1.

[A1] We thank the reviewer for carefully checking the manuscript and finding the error in the header of Table 1. We modified the first row to identify both the model and dataset of each experiment, and clarified the setup.

[W2] Advantage of low-pass filters.

[A2] As noted by the reviewer, a low-pass filter directly modifies the synthetic image, without any consideration of visual features. We exploit this low-pass filter to reduce the noise in the synthetic dataset on the fly by applying traditional techniques. The success of this technique clearly shows that mitigating the noise in the synthetic dataset is a key issue that should be addressed in the ZSQ domain. We appreciate the reviewer’s suggestion to incorporate visual features and to explore adaptive filtering, which will be part of our future work.

[W3] Observation (Figure 2) under limited conditions.

[A3] We further analyze the CAM pattern discrepancies across different settings in Appendix C.4. Table 5 demonstrates that 1) the saliency map varies notably for the quantized models trained on synthetic datasets, and 2) SynQ effectively addresses this issue through CAM alignment (Idea 2).

[W4] Computational cost comparison with the SOTA methods.

[A4] To analyze the impact of the additional computational overhead introduced by SynQ, we added a runtime analysis in Appendix C.2. From Figure 8, we observe that the runtime overhead caused by SynQ is minimal, contributing just 82.19% to the overall fine-tuning time on average. Overall, SynQ achieves a significant accuracy improvement with only a slight increase in quantization time.

Dear Reviewer 7rcv,

We are grateful for your constructive feedback and are delighted to hear that our efforts have resulted in an overall improvement. Regarding the official implementation of SynQ, it is available online through both the supplementary materials and via https://anonymous.4open.science/r/SynQ_off, as mentioned in line 107 of the manuscript. For the brief experiment discussed above, our implementation relies on the official codes of RepQ-ViT [1] and RandAugment [2]. Overall, we are deeply thankful for your feedback and are looking forward to further perfecting our research.

With our thanks,

Authors of Submission 6292

References.

[1] Li et al., “Repq-vit: Scale reparameterization for post-training quantization of vision transformers”, ICCV 2023

[2] Cubuk et al., “RandAugment: Practical Automated Data Augmentation with a Reduced Search Space”, NeurIPS 2020

We sincerely appreciate your high-quality review with constructive comments. We have carefully taken your insights into account and addressed each point. Please kindly refer to the revised manuscript, where the specific updated parts are discussed in detail within the following answers. We marked added or modified parts as blue for reviewers’ convenience.

[W1] Repetition of three major limitations.

[A1] We understand the reviewer's point regarding the repeated references to similar content across the paper. That said, we consider these challenges to be a key aspect of the paper and feel that reiterating their importance is necessary to convey their central role in this paper. We aim to use a top-down structure so that even readers skimming the paper could easily grasp the key messages, recognizing that this naturally results in some repetition, though thorough readers like the responsible reviewer may find it less necessary. In this context, we intentionally unified the titles of challenges and their corresponding observations to avoid potential confusion, while providing varying perspectives within the content of each. We hope this clarifies our intentions and aligns with the reviewer's valuable observations.

[W2] Novelty of ‘Soft Labels for Difficult Samples’ (Idea 3).

[A2] As the reviewer pointed out, our proposed idea of using ‘Soft labels for difficult samples’ presents the novelty of dynamically adjusting the usage of CE loss depending on sample difficulty. To review the previous works, one set of papers such as Qimera [1] and AIT [2], support the use of soft labels rather than hard labels. Another set of papers, such as HAST [3], emphasizes the importance of the difficulty distribution of samples within the synthetic dataset. Inspired by the definition of difficulty in Equation 3 and the observation in Figure 3, we combined the insights from both sets of research and proved that the decision to use hard labels based on difficulty is both intuitive and experimentally supported. While the aforementioned methods all determine the use of hard and soft labels uniformly across samples, loss splitting based on sample difficulty distinguishes our work from existing approaches and makes a novel contribution.

[W3(a)] Comparison with Genie [4].

[A3] The key distinction between Genie [4] and our proposed SynQ is in the quantization approach; QAT methods including SynQ adopt min-max quantization, while Genie adapts advanced techniques such as LSQ [5], QDrop [6], and BRECQ [7]. We provide a detailed comparison on the settings of Genie and SynQ to explain why Genie is neglected from our competitors in the main experiments (Table 1) in Appendix C.7. Due to the differences in quantization strategies and experimental conditions, evaluating Genie alongside zero-shot QAT methods is challenging. Then, we integrate SynQ with Genie and evaluate its evaluation performance to compare the performance with Genie and highlight the broad applicability of our proposed method. The results in Table 8 demonstrate the superiority of SynQ, showcasing its compatibility with diverse quantization techniques such as zero-shot PTQ.

[W3(b)] Additional experiments on ViT baselines.

[A4] We agree on the necessity of experimenting with SOTA models in paper writing. Regarding the Zero-Shot Quantization (ZSQ) of Vision Transformers (ViTs), three papers have been published to date: PSAQ-ViT [8], PSAQ-ViT V2 [9], and CLAMP-ViT [10]. However, we have only been able to conduct experiments by applying SynQ to PSAQ-ViT, as outlined in Section 5.3. Despite our intention to experiment with SynQ on both PSAQ-ViT V2 and CLAMP-ViT, we encountered two issues that prevented us from doing so: 1) neither paper has an official code implementation available in their public repositories, and 2) we were unable to reproduce the results reported in those papers. For PSAQ-ViT V2, while it was claimed to be released along with the PSAQ-ViT code, we found only the PSAQ-ViT code when we checked the link (https://github.com/zkkli/PSAQ-ViT), and observed that there have been no updates since the last commit two years ago. For CLAMP-ViT, the official code repository (https://github.com/georgia-tech-synergy-lab/CLAMP-ViT) includes only the mixed-precision model weights and evaluation code, with no code for the synthetic dataset generation or quantization process. The failure to reproduce the experiments is due to the lack of detailed information regarding certain hyperparameters and other experimental settings, which made it impossible to replicate the reported results. Given that SynQ is a synthesis-aware fine-tuning technique, it is applicable to all methods that generate synthetic datasets. We further aim to continue following relevant studies on ZSQ of ViTs and explore the impact of SynQ on other methods, beyond PSAQ-ViT.

[W4] Illustration of the synthetic dataset.

[A5] We appreciate the reviewer’s gentle recommendation on the visualization of synthesized data to help understand the superiority of SynQ. In Appendix C.10, we compare the images from three synthetic datasets before and after the low-pass filter. From Figure 12, we have two observations. First, the visualized images show distinct patterns and differences across various classes. Second, the low-pass filter removes noise effectively while preserving essential features in the generated images, especially noticeable in lower resolution samples.

[Q1] Investigating “off-target prediction” when training with real images.

[A6] Although intuitive and persuasive, Figure 2 has a limitation that we observe under limited conditions. In Appendix C.4, we extend this to entire synthetic datasets generated from various methods by evaluating the CAM discrepancy, to validate that the challenge of “off-target prediction” is a common issue. Also, we investigate quantized models that are fine-tuned with real datasets to check if the challenge is induced by the usage of synthetic datasets. Table 5 highlights that training with synthetic datasets exacerbates CAM discrepancy, as the reviewer pointed out.

[Q2] Baseline of SynQ.

[A7] As discussed in Section 4.5, we adopt calibration center synthesis [11], difficult sample generation [3], and sample difficulty promotion [3] as baseline to produce the synthetic dataset for the best performance. This baseline method is utilized for the main results and observations including Tables 1, 3, 5, and Figure 7. To clarify the experimental settings and for better reproducibility, we provide the step-by-step implementation details of the baseline in Appendix D. As we combine only the synthetic dataset generation part of two papers HAST and TexQ for baseline, the performance of this baseline is not listed in Table 1.

[Q3(a)] Further analysis on the difficulty threshold $\tau$.

[A8] We conduct a hyperparameter analysis on difficulty threshold $\tau$ under different settings in Appendix C.11. Figure 13 depicts the ZSQ accuracy with different $\tau$ values of (a) a ResNet-20 model pre-trained on CIFAR-10 dataset, (b) a ResNet-20 model pre-trained on CIFAR-100 dataset, and (c) a MobileNet-V2 model pre-trained on ImageNet dataset. As shown in the figure, SynQ shows similar tendency across different settings, while (a) maximizes with the $\tau$ value of 0.7. This is because the error rate of pre-trained models in Figure 3 begins to increase at a higher difficulty level of approximately 0.65 for CIFAR-10, compared to 0.5 for the others. In summary, SynQ shows a common trend of its performance regarding $\tau$ across various settings, where the optimal $\tau$ should provide a nice trade-off between containing sufficient samples and not using wrong samples.

[Q3(b)] Magnitude comparison between CAM and cross-entropy losses.

[A9] We compare the magnitude of CE and CAM losses in the table below:

We report the 3bit quantization result of a ResNet-20 model pre-trained on the CIFAR-10 dataset, a ResNet-20 model pre-trained on the CIFAR-100 dataset, and a ResNet-18 model pre-trained on the ImageNet dataset. The ratio of CE loss to CAM loss is around 1K for all three cases. Thus, for the best-performing case, $\lambda_{CAM}$ is set much larger than $\lambda_{CE}$ to balance the scales of the two terms during computation.

[Q4] Application on different baselines.

[A10] We appreciate the suggestion of the reviewer to add extra experimental results applying SynQ on noise optimization baselines. In Appendix C.6, we evaluate the 3bit ZSQ accuracy of the ResNet-18 model pre-trained on the ImageNet dataset, comparing the performance with and without SynQ (see Table 7). Beyond the existing results on three generator-based baselines (GDFQ [12], Qimera [1], and AdaDFQ [13]), we add extra results on three noise optimization baselines: IntraQ [14], HAST [3], and TexQ [11]. SynQ shows consistent performance enhancement through all baselines and bitwidth, with up to 8.11%p for noise optimization baselines.

Additionally, the reviewer asked the authors’ opinion on the performance of generator-based models. Specifically, when SynQ is applied to these methods, their performance is lower than that reported for SynQ in Table 1. The lower performance observed would be attributed to the synthetic dataset's quality; SynQ yields better results when fine-tuned with a higher-quality dataset. The accuracy of the generator-based methods is lower than that of other noise optimization methods, even after applying SynQ. This suggests that the image quality of these methods is inferior to the baseline.

References.

[1] Choi et al., “Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples”, NeurIPS 2021

[2] Choi et al., “It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher”, CVPR 2022

[3] Li et al., “Hard Sample Matters a Lot in Zero-Shot Quantization”, CVPR 2023

[4] Jeon et al., “Genie: show me the data for quantization”, CVPR 2023

[5] Esser et al., “Learned Step Size Quantization”, ICLR 2020

[6] Wei et al., “QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization”, ICLR 2022

[7] Li et al., “BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction”, ICLR 2021

[8] Li et al., “Patch Similarity Aware Data-Free Quantization for Vision Transformers”, ECCV 2022

[9] Li et al., “PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers”, TNNLS 2023

[10] Ramachandran et al., “CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs”, ECCV 2024

[11] Chen et al., “TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration”, NeurIPS 2023

[12] Xu et al., “Generative Low-bitwidth Data Free Quantization”, ECCV 2020

[13] Qian et al., “Adaptive Data-Free Quantization”, CVPR 2023

[14] Zhong et al., “IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization”, CVPR 2022

[A1 Comment] : Thanks to the authors’ response, the reviewer is able to understand why the authors repeated those contents. However, the reviewer just thought that if the authors had only summarized the limitations, rather than explaining them in detail repeatedly, readers who skim the paper can go back and reread them and the additional space can be used with other experiments and analyses.

[A2 Comment] The authors showed well that using soft labels with hard samples can enhance performance of a quantized model. However, it will be helpful to describe where the idea got inspiration from and what has changed like below.

• Several previous work adopted soft labels, but applying soft labels to hard samples only performs better.
• Model outputs can be good proxies for measuring sample difficulty, with the proposed soft label loss.

[A3 / A4 Comment] The reviewer understands the difficulty that the authors mentioned. Thank the authors for additional experiments to validate the proposal.

[A5 Comment] The contour of generated images in SynQ is similar to that in other works, while it is conspicuous that the noise is removed with low-pass filter. This can be a ground of Table 3. Thank the authors for the visualization.

[A6 / A7 Comment] Thank the authors for clearly addressing the questions through the experiment and providing answers.

[A8 Comment] As the reviewer expected, optimal points of $\tau$ differ according to model and dataset. Nevertheless, it is a good point that the overall tendency of performance change caused by fine-tuning $\tau$ is consistent. The reviewer appreciates the authors demonstrating it through experiments.

[A9 Comment] The reviewer presumed that the magnitude of CAM loss is small because $\lambda_{CAM}$ is very large, and the authors show that with an experiment. Even though $\lambda_{CAM}$ is very small, reducing it helps improving performance. It seems to be a good point to address in the future that those results can be achieved only with optimizing CE loss and CAM loss simultaneously, or CE loss and CAM loss are orthogonal. (CAM loss is critical for generalization ability of a quantized model) Thank the authors for providing such valuable insight.

With the authors' sincere answers, the reviewer has decided to raise the score to 6.

We sincerely appreciate your high-quality review with constructive comments. We have carefully taken your insights into account and addressed each point. Please kindly refer to the revised manuscript, where the specific updated parts are discussed in detail within the following answers. We marked added or modified parts as blue for reviewers’ convenience.

[W1/W2] Observations under limited conditions

[A1] We understand the reviewer's concern that our observations might be applicable only under certain conditions, raising doubts about whether the phenomena are truly widespread in the domain. To address this concern, we present additional experiments across a range of baseline methods, models, and datasets in Appendices C.3 and C.4, showing that the phenomena are consistently observed.

First, we show that noise in the synthetic dataset remains a widespread issue in ZSQ by extending the Figure 5 experiment across different settings (Appendix C.3). Figures 9 and 10 highlight the pervasive nature of noise in synthetic datasets and the consistent impact of low-pass filtering in numerous settings.

Second, we analyze the CAM pattern discrepancies across different baselines trained with synthetic and real datasets (Appendix C.4). Table 5 demonstrates that 1) the saliency map varies notably for the quantized models trained on synthetic datasets, and 2) SynQ effectively addresses this issue through CAM alignment (Idea 2).

These results demonstrate that the challenges we identified are critical issues that should be explored crucially in the zero-shot quantization domain.

[Q1] Evaluation on low-bit conditions.

[A2] We investigate the performance of SynQ in low-bit conditions in Appendix C.7. While following the experimental setting of Genie [1], we compare the ZSQ accuracy of Genie with and without SynQ in Table 8. SynQ successfully enhances the performance of zero-shot PTQ methods, with higher improvements in lower-bit conditions such as W2A2 or W4A4.

[Q2] Performance regarding the size of synthetic dataset.

[A3] We appreciate the reviewer’s gentle recommendation to investigate the performance variation according to the size of the generated synthetic dataset. We analyze this in Appendix C.9, where Figure 11 shows the 3bit ZSQ accuracy of the ResNet-18 model pre-trained on ImageNet dataset. SynQ demonstrates steadily increasing ZSQ accuracy with larger training datasets, outperforming TexQ even with only half the images. This result underscores the scalability and efficiency of SynQ to achieve high accuracy even with constrained training data.

References.

[1] Jeon et al., “Genie: show me the data for quantization”, CVPR 2023

We sincerely appreciate your insightful and constructive review. We have carefully taken your insights into account and addressed each point. Please kindly refer to the revised manuscript, where the specific updated parts are discussed in detail within the following answers. We marked added or modified parts as blue for reviewers’ convenience.

[W1] Application to other tasks.

[A1] To begin with, we thank the reviewer for the great idea to extend SynQ to other target tasks, such as object detection or segmentation. Applying SynQ to tasks beyond image classification would clearly showcase the wide applicability and broad versatility of our proposed method. However, zero-shot quantization of vision models for tasks other than image classification is still an unexplored area, with only one recent work [1] submitted to ICLR 2025 addressing this. Unfortunately, we were unable to apply SynQ on the given setting as 1) the work is still under academic peer review, 2) it lacks official code implementation and project page, and 3) the rebuttal phase does not provide sufficient time for our own implementation and validation. We will actively pursue further studies on these areas and explore expanding SynQ to other target tasks in our future work.

[W2] Details on the computational overhead.

[A2] In our submitted manuscript, we discuss the computational complexity of SynQ in Theorem 1 and its proof in Appendix C.1. As the reviewer pointed out, we agree that our proof could have been more detailed. Accordingly, we provide a more detailed analysis of the computational complexity in the updated proof included in Appendix C.1 of the revised manuscript. As a result, the computational complexity of the low-pass filter and CAM alignment is $O(NZlogZ)$ and $O(NLT_{\theta})$, respectively, where $N$ is the size of the synthetic dataset, $Z \times Z$ is the input dimension, $L$ is the layer count, and $O(T_{\theta})$ indicates the inference complexity of given pre-trained model. Furthermore, to analyze the actual impact of the additional computational overhead introduced by SynQ, we added a runtime analysis in Appendix C.2. The runtime overhead caused by SynQ is minimal, contributing just 82.19% to the overall fine-tuning time on average. Overall, SynQ achieves a significant accuracy improvement with only a slight increase in quantization time.

[W3/Q1] Robustness towards different types of noise.

[A3] We first clarify our main challenge of “noise in the synthetic dataset” to mitigate possible misunderstandings, then investigate the robustness of SynQ and the impact of the low-pass filter with further experiments. The novelty of our first challenge lies in observing that synthetic datasets generated by existing ZSQ methods exhibit greater sharpness and noise levels compared to real datasets (see Figures 1, 5, 9, and 10). This inherent noise stems from excessive high-frequency components in the generated samples, disrupting the fine-tuning process. We exploit a low-pass filter (Idea 1) on the fly, utilizing traditional methods to effectively mitigate noise. The demonstrated success of this approach emphasizes that addressing noise in synthetic datasets is a crucial challenge in the ZSQ domain.

To validate the robustness of SynQ, we evaluate how ZSQ accuracy decreases when different types of noise are intentionally introduced into the synthetic dataset, as detailed in Appendix C.8. Table 9 demonstrates that the low-pass filter effectively minimizes accuracy degradation across various noise types, surpassing the baseline in capacity.

To further verify the impact of the low-pass filter, we present additional experiments in Appendices C.3 and C.10. Figure 10 illustrates the amplitude distribution of synthetic datasets, comparing the results before and after applying the low-pass filter. The results highlight the consistent impact of low-pass filtering in numerous settings. Figure 12 provides a visualization of the generated images with different baseline methods and datasets, showing how the low-pass filter behaves at the image level. We observe that the low-pass filter removes noise effectively while preserving essential features in the generated images, which are especially noticeable in lower-resolution samples. These results highlight the consistent impact of low-pass filtering in numerous settings.

That said, as the reviewer pointed out, the current approach lacks consideration for the specific types of noise in the synthetic dataset. We thank the reviewer for the valuable comment; we will reflect on this insight and focus on improving the noise filtering approach in our future work.

[Q2] Including more related papers.

[A4] Following the reviewer’s suggestion, we expanded our related work in Section 6 to include DAFL [2] and DFQ-AKD [3], along with other relevant works such as FDDA [4], QALoRA [5], Mr.BiQ [6], RepQ-ViT [7], Genie [8], QDrop [9], etc.

References.

[1] ICLR 2025 Conference Submission4776 Authors, “Zero-shot Quantization for Object Detection”, Submitted to ICLR 2025, https://openreview.net/forum?id=XNr6sexQGj

[2] Chen et al., “Data-free learning of student networks”, CVPR 2019

[3] Choi et al., “Data-free network quantization with adversarial knowledge distillation”, CVPRW 2020

[4] Zhong et al., “Fine-grained data distribution alignment for post-training quantization”, ECCV 2022

[5] Xu et al., “Qa-lora: Quantization-aware low-rank adaptation of large language models”, ICLR 2024

[6] Jeon et al., “Mr. biq: Post-training non-uniform quantization based on minimizing the reconstruction error”, CVPR 2022

[7] Li et al., “Repq-vit: Scale reparameterization for post-training quantization of vision transformers”, ICCV 2023

[8] Jeon et al., “Genie: show me the data for quantization”, CVPR 2023

[9] Wei et al., “Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization”, ICLR 2022