GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers

Dear Reviewer yZXJ,

Thank you very much for your detailed review and positive evaluation of our paper! We are delighted that you recognize the novelty and effectiveness of our "preserving the optimization basin" insight, as well as the clarity of our paper. The questions you raised are very specific and valuable, and they will help us to further improve our paper.

Regarding Weaknesses:

1. On "activations are more important than weights" being common knowledge: We completely agree with your point. This insight has become a consensus in the quantization community, especially in recent excellent works like AWQ, which you mentioned. We cited this viewpoint in our paper to build the logical starting point for our methodology. We want to emphasize that the novelty of GPLQ lies not in restating this consensus, but in how we construct a brand-new QAT framework based on it. Unlike PTQ methods such as AWQ, GPLQ innovatively applies this insight to QAT by designing the "activation-first" two-stage process. This allows us to enjoy the high accuracy of QAT while circumventing its drawbacks of training instability and poor generalization.

2. On QAT and PTQ methods not being original: Yes, we adopted mature QAT and PTQ techniques as foundational components within the GPLQ framework. Our contribution is not in inventing new quantization operators, but in proposing a superior, decoupled quantization process. This is analogous to how ResNet's contribution was not inventing the convolution, but proposing the "residual connection" framework that enabled convolutional networks to be deeper and more powerful. Our supplementary Table B.5(response to Reviewer Xx2u) also demonstrates the modular nature of the GPLQ framework, showing that its performance is not rigidly tied to any specific component.

3. On a broader QAT comparison: We fully agree that a more comprehensive comparison would be more convincing, and we candidly discussed this issue in the "Limitations" section. The main obstacle is indeed the scarcity of open-source code for SOTA QAT methods, especially implementations that can support large ViT models and multiple tasks. Under the current circumstances, we have done our best to compare against the SOTA methods that were accessible and reproducible. The overwhelming advantages in efficiency and memory shown in Figure 1 and the newly added Table B.7(response to Reviewer FJP6) are, in themselves, a core contribution of our method.

Regarding Questions:

• Q1: Regarding $\mu_t$ in Eq (2): Thank you for your careful observation! The mean term $\mu_t$ here cannot be eliminated. In PCA projection, the data must first be centered (i.e., by subtracting the mean) before being projected onto the principal components $V_{sel}$. The means of the teacher features ($f_t$) and the student features ($f_s$) are different. Therefore, both must be centered using the teacher's feature mean, $\mu_t$, to ensure they are being compared in the same coordinate system.

• Q2: Regarding the training loss of traditional QAT in Figure 4(a): This is an excellent question that reveals a key point we wanted to make. The loss of traditional QAT rises sharply at the beginning of training ("escaped basin") because it is searching for a new optimization basin that is adapted to the quantization operations. While this new basin might achieve a lower loss value on the ImageNet training set (i.e., overfitting), it is often "sharper," leading to poorer generalization. In contrast, GPLQ's loss is low and smooth from the start because it remains within the original FP32 model's broad and highly generalizable "good" basin. Therefore, while the final loss value of QAT on the training set might be lower, its "path" and "destination" are detrimental to generalization, whereas GPLQ's are beneficial.

• Q3: How to better identify the "same basin": You've touched upon a core conceptual issue. We admit that "basin" is a figurative description. We support this argument with three dimensions of evidence: (1) Training loss curve (Figure 4(a)): GPLQ's smooth loss curve indicates its optimization process is stable and does not drastically jump out of the initial state. (2) "Internal FP32" weight performance (Table 5 of our response to Reviewer Xx2u): This is the most compelling evidence. The performance of GPLQ's trained "latent FP32" weights (81.1%) is nearly identical to the original FP32 model (81.2%), whereas traditional QAT's drops significantly (68.9%). (3) Weight space visualization: We plan to include a visualization of the weight space in the final version, using methods like t-SNE/PCA to intuitively show that GPLQ's weights are spatially closer to the original weights.

• Q4 & Q5 & Spelling/Grammar Errors: Thank you so much for pointing out these typographical and writing issues with such care! We apologize for these oversights and promise to correct the casing of w/o PCA and w/PCA (as in Table 6 of our response to Reviewer Xx2u) and all other formatting and textual issues you have identified in the final version.

Thank you again for your valuable review comments; your suggestions are of immense help in improving the quality of our paper.

Dear Reviewer FJP6,

We sincerely thank you for the time and effort you have dedicated to reviewing our paper and for raising several profound and challenging questions. Your rigorous scrutiny has prompted us to think more deeply about and further validate the core contributions and motivations of our work. Although we are disheartened by your current "Reject" rating, we are confident that the following clarifications and the substantial new experimental evidence we have added will fully address your concerns. We hope to persuade you to reconsider your evaluation of our work.

On missing theoretical analysis and/or using existing technical components:

You may not be familiar with the particular state-of-the-art of QAT. Indeed, there are papers on QAT, but they are far from being practically usable in real-world applications yet. As we briefly discussed in the introduction, a QAT-trained network or backbone cannot be generalized/used in a different scenario, the training took 37.5GB memory during training Swin-T (a tiny model!), and its training is very long (e.g., 600 epochs). We believe we need a framework that can train QAT models with a small GPU memory footprint and very fast, and the trained model should be generalizable.

Only after such a framework is available does it become feasible to perform theoretical analysis. And, it does not hurt to use existing components in such a framework—the contribution or breakthrough is a framework that makes QAT practically possible (e.g., reducing GPU memory from 37.5GB to 6.6GB, training from 600 epochs to 1 epoch, and turning a non-generalizable model into a generalizable one), not any specific component. In fact, as we will explain below, all such components can be replaced freely with other components, yet the GPLQ framework still achieves our goals: generalizable, practical, and lightning-fast.

Regarding Weaknesses and Questions:

1. On the method not being ViT-exclusive and the motivation lacking deep theoretical analysis (Weaknesses 1-3 & Question 1):

   • Generality as a Strength, Not a Weakness: Your observation that GPLQ does not rely on ViT-specific features is very accurate, and we believe this is precisely one of GPLQ's core strengths. A truly valuable quantization method should possess good generality. To strongly demonstrate this, we have added experiments with GPLQ on classic CNN architectures (ResNet). As shown in Table B.3 of our response to Reviewer QuB9, GPLQ performs exceptionally well on ResNets, significantly surpassing SOTA PTQ methods designed for ViTs. This indicates that the principles we propose—"activation-first" and "preserving the optimization basin"—are cross-architectural.
   • A Deeper Analysis of Why "Activation-First" Works: You asked why this sequence improves quantization quality. The core mechanism lies in decoupling and stabilization. In traditional QAT, the quantizers for weights and activations are optimized simultaneously, creating a coupled system where they influence each other and can easily lead to oscillations. In particular, errors from sensitive activation quantization can mislead weight updates, forcing the weights to deviate from the original, superior optimization basin. GPLQ's first stage optimizes only the activation quantizers (while weights remain learnable at FP32), allowing the model to first adapt to the information loss from activation quantization and find a weight state that is most robust to it. This process is more stable (as shown by the smooth loss curve in Figure 4(a)) and remains within the original basin (Table 5 of our response to Reviewer Xx2u). Once activations are stabilized, the second-stage weight PTQ faces a simpler, more deterministic problem (quantizing from W32A4 -> W4A4, instead of W32A32 -> W4A4), thus making PTQ methods (like QwT) more effective.
   • Reinforcing the Empirical Motivation: We acknowledge that GPLQ originated from empirical observations, but these observations are well-supported by data. In addition to the evidence in the original paper, we plan to add visualizations of the weight space to more intuitively show how GPLQ keeps the weights in a region close to the original FP32 model, whereas traditional QAT causes them to drift to a distant location. This will provide stronger visual evidence for our core motivation.

2. Regarding the lack of Wall-Clock Time Comparison (Question 2): Thank you for pointing this out. We actually provided this comparison in Figure 1 of the paper. To more clearly showcase GPLQ's huge efficiency advantage, we have organized the information from Figure 1 into a detailed table (see Table B.7 below). This table explicitly lists the detailed comparison between GPLQ and methods like Q-Var and OFQ in terms of ImageNet accuracy, average downstream task accuracy, training time (minutes), and peak memory (GB). The data shows that GPLQ's training time (35 minutes) is approximately 500 times faster than Q-Var (17,400 minutes) and OFQ (18,360 minutes), while also having lower memory consumption. These concrete numbers strongly support our claim of "lightning-fast" efficiency.

   Table B.7: Detailed comparison of performance and resource overhead between GPLQ and SOTA methods.

   Model	Method	ImageNet Acc (%)	Avg. Downstream Acc (%)	Training Time (min)	Peak Memory (GB)
   Swin-T	Full-precision	81.2	70.79	3078	7.2
   	Q-Var	82.4	68.12	17400	25.9
   	OFQ	81.9	60.73	18360	37.5
   	GPLQ	79.8	71.18	35	6.6
   	RepQViT	73.0	62.79	3	2.3

3. Regarding Dependence on Existing Technologies and Novelty (Weakness 4): We understand your concern about novelty. We want to emphasize that GPLQ's innovation lies not in inventing an isolated component, but in proposing and validating a completely new and efficient quantization framework. To prove that GPLQ is not simply reliant on specific technologies, our component replacement experiments (Table B.5 of our response to Reviewer Xx2u) show that its performance remains robust. The true value of GPLQ is that it provides the community with a plug-and-play, extensible new approach that can bring QAT from theory to practice.

We sincerely hope that the detailed responses above, along with the extensive new experiments we have conducted to address your comments (CNN generality, training duration ablation, detailed efficiency data, etc.), can fully demonstrate the depth, novelty, and importance of our work. We kindly ask that you re-evaluate our contribution based on this new information. Thank you again for your valuable feedback.

I agree with and understand your perspective regarding the practical limitations of QAT. However, what I expected in the response - regarding motivation - was a more detailed toy experiment or theoretical analysis illustrating the effect of the proposed methods. Specifically, how does decoupling contribute to stabilizing the quantization process? For instance does it guide the distributions toward a more quantization-friendly state? Or does it effectively reduce oscillation during activation quantization or weight quantization, as suggested in your explanation?

Given that GPLQ is presented as a lightweight yet effective framework, I had hoped to see a more in-depth analysis, especially since such experiments could be conducted efficiently. I believe a thorough analysis is particularly important since GPLQ is proposed as a general framework, which would benefit from clearer insight into why and how it works.

However, considering its impact and practicality—and with the concern regarding the motivations of one method being partially addressed through your plan to include loss landscape visualizations—I have raised my score to 3 for now. While I understand merit in the work, I remain somewhat reserved in fully advocating for acceptance, as there is still room for improvement in analysis.

In your review:

As acknowledged in the limitations, GPLQ relies heavily on existing techniques. Although these components are combined effectively, the lack of in-depth analysis regarding the motivations and their interactions raises concerns about the novelty of the work.

Per the NeurIPS 2025 reviewer guidelines at https://neurips.cc/Conferences/2025/ReviewerGuidelines :

For example, if you argue about the lack of novelty, please provide appropriate references and point to existing mechanisms within.

Sorry, but do you happen to have such a reference? Not sure I'm following about how novelty of a framework/method relates to the analysis of the same.

Thanks!

Thank you for addressing my concerns within a short period. As your new analyses help clarify how the two main components of the framework work, I have raised my score to 4.

Thank you for pointing this out.
I did not provide additional references, as the authors are already aware that their work builds upon or extends existing methodologies [1,2,3].
I believe that as a framework, the paper should include a more thorough analysis of why and how each component functions and interacts, similar to last responses from authors. Given that frameworks are presented as general and potentially adaptable, such analysis is essential. Without it, the novelty and contribution of the work risk being perceived as merely a combination of existing methods.
I hope this addresses your concerns regarding my review.

[1] Esser et al., "Learned step size quantization", ICLR 2020
[2] Fu et al., "Quantization without tears", CVPR 2025
[3] Zhou et al., "All you need in knowledge distillation is a tailored coordinate system", AAAI 2025

Dear Reviewer QuB9,

Thank you very much for your detailed and constructive review of our submission (#8001). We greatly appreciate your recognition of our work's clarity and methodological innovation. Your feedback is invaluable for improving the quality of our paper. We will address your questions and concerns one by one.

Regarding Weaknesses:

1. On the concern that the method is a "dispersed" combination of different techniques: We appreciate this observation. The core contribution of our work is to propose GPLQ as a flexible and efficient quantization framework, rather than a single, fixed algorithm. Its "activation-first, weights-later" core idea is modular. To demonstrate this, we have added a supplementary experiment (see Table B.1 below) showing that the GPLQ framework maintains high performance even when components in the PTQ stage are replaced (e.g., changing the initialization method). This indicates that the success of GPLQ is not due to an incidental combination of specific components, but rather stems from the superiority of its framework design.

   Table B.1: Verifying the interchangeability of different components within the GPLQ framework.

   Model	Act-QAT Initialization	ImageNet Top-1 Acc (%)
   Swin-T	Percentile (as in RepQ-ViT)	79.8
   	MSE-based	79.8
   	MinMax	79.6

2. On alternative options for the activation QAT stage and initialization methods: You've raised an excellent point. We chose LSQ because it is widely validated and effective in the QAT field. We acknowledge that exploring other options, such as the EMA-based method mentioned in [a], would also be valuable. While a comprehensive comparison is planned for future work, we will add a discussion and a placeholder for preliminary comparisons in the appendix to explore the performance of different activation quantization strategies. Furthermore, our initialization step is designed to provide LSQ with a superior starting point, enabling rapid convergence within just one epoch of training. This is a key part of our "lightning-fast" training philosophy.

3. On the lack of study on fine-tuning duration: This is a critical question. We chose a 1-epoch fine-tuning to strike an optimal balance between performance and efficiency. To validate this choice, we have added an ablation study on the Act-QAT training duration (see Table B.2 below). The results show that extending the training from 1 epoch to 2, 5, or even 10 epochs yields very limited performance gains (<0.3%) on both ImageNet and downstream tasks, while the training cost increases manifold. This strongly demonstrates that 1-epoch training is the "sweet spot" for the GPLQ framework in terms of efficiency and effectiveness. Of course, our GPLQ framework can be easily modified if one is willing to invest more time for further potential improvements.

   Table B.2: Impact of Act-QAT training duration on model performance.

   Model (Model)	Act-QAT Epochs	ImageNet Top-1 Acc (%)	Avg. Downstream Acc (%)
   Swin-T	1 (Ours)	79.8	71.2
   	2	79.8	71.5
   	5	79.9	71.3
   	10	79.9	71.1

4. On the comparison between feature-mimicking loss and lower-bit quantization: You made an insightful comparison of our work with methods like BitDistiller. We agree that these methods have achieved remarkable success in sub-4-bit quantization. In fact, we argue that GPLQ's 'activation-first' framework is even more critical at lower bit-widths, as the challenges of activation quantization become more severe. To verify this, we conducted more challenging W3A3 quantization experiments (see Table B.6 of our response to Reviewer Xx2u). The results are encouraging: GPLQ significantly outperforms SOTA PTQ methods (e.g., by +21.5% on Swin-T), demonstrating the strong potential of our approach. More importantly, GPLQ achieves this exceptional performance while adhering to its lightweight and efficient core philosophy: using only 1 epoch for Act-QAT and a simple PCA-based feature-mimicking loss, thus avoiding the complex self-distillation loops found in methods like BitDistiller. Therefore, we believe GPLQ is not only a practical solution for 4-bit quantization but also a powerful framework for the sub-4-bit domain that strikes a superior balance between performance, efficiency, and simplicity.

Regarding Questions:

• Q1: Regarding QAT methods with no open-source code: Thank you for the question. When we wrote the paper, some key comparison methods like OFQ and Q-Var were reproducible, so we included them. However, others such as Q-ViT and PackQViT had limited open-sourcing (not all details were published). Furthermore, it was difficult to find official or reliable third-party implementations for these methods, especially for large models (e.g., Swin-B/L) or various vision tasks (e.g., detection). This prevented us from conducting a fair, large-scale side-by-side comparison. In particular, regarding memory usage and training time, many methods resulted in out-of-memory (OOM) errors on our hardware for large models, as illustrated in Figure 1 of our paper.

• Q2: Is GPLQ limited to Vision Transformers? Not at all. The core insights of GPLQ—that activations are harder to quantize than weights and that preserving the optimization basin is crucial for generalization—are universally applicable. To demonstrate this, we have successfully applied GPLQ to classic CNN architectures (ResNet) and achieved excellent performance (see Table B.3). This shows that GPLQ is a general framework with broad application potential.

  Table B.3: Generalization performance of GPLQ on CNN architecture (ResNet-18).

  Model (Model)	Method (Method)	ImageNet Top-1 Acc (%)	Avg. Downstream Acc (%)	Training Epochs
  ResNet-18	FP32	69.10%	58.78	-
  	RepQViT	56.97%	56.7	-
  	GPLQ (W4A4)	66.16%	59.44	1
  ResNet-34	FP32	74.90%	59.57	-
  	RepQViT	64.76%	54.13	-
  	GPLQ (W4A4)	70.36%	60.76	1
  ResNet-50	FP32	78.70%	66.17	-
  	RepQViT	63.68%	52.09	-
  	GPLQ (W4A4)	74.13%	67.68	1
  ResNet-101	FP32	76.20%	63.6	-
  	RepQViT	56.97%	60.26	-
  	GPLQ (W4A4)	73.83%	64.21	1
  ResNet-18	DoReFa	68.10%	-	200
  	PACT	69.20%	-	200
  	LSQ	71.1%	-	90
  ResNet-50	DoReFa	71.40%	-	200
  	PACT	76.50%	-	200
  	LSQ	76.70%	-	90

• Q3: Why is the feature-mimicking loss only applied to the penultimate layer? We chose the penultimate layer based on two considerations: (1) High-level semantic information: This layer's features contain the highest level of semantic information, which is critical for the model's classification and generalization capabilities. (2) Efficiency trade-off: Constraining the features of a single layer allows for effective knowledge transfer from the teacher model while minimizing computational overhead, which aligns with our "lightning-fast" principle. Applying the loss to all layers would significantly increase computational complexity.

• Q4: What is the hyperparameter for the feature-mimicking loss? We apologize for not stating this clearly in the original manuscript. In our experiments, the weight ratio of the feature-mimicking loss ($L_{PCA}$) to the original classification/task loss is 1:1. This means we did not introduce an additional hyperparameter $\alpha$ (or rather, $\alpha=1.0$). In our supplementary ablation study (Table B.4), we also explored the impact of different loss weights, and the results show that the default setting ($\alpha=1.0$) achieves the best downstream task generalization performance.

  Table B.4: Ablation analysis of the Feature-Mimicking Loss.

  Model (Model)	Loss Application Layer	Loss Weight (α)	ImageNet Top-1 Acc (%)	Avg. Downstream Acc (%)
  Swin-T	Penultimate (Ours)	1.0 (Default)	80.4	71.2
  	None (w/o PCA Loss)	-	80.3	69.6
  	Penultimate	0.1	80.4	70.3
  	Penultimate	10	80.2	70.7

Thank you again for your valuable time and profound insights. We believe these supplementary experiments and clarifications can address your concerns and further demonstrate the value of our work.

This is all great, thank you for the detailed rebuttal. Thank you authors!

1. Description of it being a "framework" makes contribution clear. The authors will open-source code, making this a general contribution to the whole community.
2. CNN experiment in the rebuttal shows generalization to other architectures, suggesting greater potential impact
3. The choice to use the feature focusing loss on only one layer is consistent with the framework's overall focus on efficiency and is a clear part of the whole design. The rebuttal makes clear why the pentultimate layer was chosen.
4. It's a good sign that the method is not sensitive to the hyperparameter of different loss weights. Thanks for the clarification.

Dear Reviewer Xx2u,

Thank you very much for your recognition of our work and for your insightful questions. You have accurately summarized the core ideas and advantages of our GPLQ method—its efficiency, low resource consumption, and strong generalization ability. We greatly appreciate your interest in our "activation is the key to quantization" narrative and your valuable questions regarding its generality.

Regarding Weaknesses and Questions:

You have merged several core questions into the "Weaknesses" section, and we will address them together here.

1. Regarding Novelty: A "clever combination" or a "truly new algorithm"? We acknowledge that GPLQ skillfully integrates several existing technologies. However, we believe its core contribution and novelty are manifested in the following aspects:

   • Paradigm Innovation: We are the first to systematically propose and validate the "activation-first (Act-QAT), weights-later (Weight-PTQ)" two-stage decoupled quantization paradigm. This is fundamentally different from traditional PTQ (which handles W and A simultaneously) and traditional QAT (which trains W and A quantizers simultaneously).
   • Insight-Driven: This paradigm is based on our core insight that "preserving the optimization basin" is crucial for generalization. By fine-tuning only the activation quantizers for just 1 epoch, we successfully keep the model within the high-performance optimization region of the original FP32 model, which is validated in Figure 4(a) and Table 5 of our paper.
   • Framework Modularity: Our supplementary experiments (Table B.5) show that as a framework, GPLQ's internal components (such as the initialization method for PTQ) are replaceable. This demonstrates the robustness and generality of its design, rather than it being a fixed, rigid process.

   Table B.5: Verifying the interchangeability of different components within the GPLQ framework.

   Model	Act-QAT Initialization	ImageNet Top-1 Acc (%)
   Swin-T	Percentile (as in RepQ-ViT)	79.8
   	MSE-based	79.8
   	MinMax	79.6

2. Regarding Generality: Can it go beyond ViTs and 4-bit settings? This is a point we are most eager to clarify. The design principles of GPLQ are general. To prove this, we conducted two key supplementary experiments:

   • Extension to CNN Architectures: We tested GPLQ on the classic ResNet family and achieved performance significantly superior to SOTA PTQ methods (RepQ-ViT), and even comparable to some long-duration QAT methods (see Table B.3 of our response to Reviewer QuB9). This demonstrates that GPLQ's effectiveness is not limited to Transformer architectures.
   • Exploration of Lower Bit-widths: We conducted more challenging W3A3 quantization experiments (see Table B.6 below). Although a drop in accuracy is inevitable, GPLQ's two-stage strategy still provides a strong baseline for low-bit scenarios. We believe the characteristic that "activations are more sensitive" becomes even more pronounced at lower bit-widths, where GPLQ's advantages may be more significant.

Table B.6: Performance of GPLQ in a lower-bit (W3A3) setting.

3. Is activation quantization truly the main bottleneck? We firmly believe so, especially when pursuing high generalization capability. Figure 2 in our paper intuitively shows that in PTQ, quantizing activations alone leads to a much larger accuracy drop than quantizing weights alone. More importantly, the experiment in Table 5 reveals the deeper reason: traditional QAT methods (like OFQ), in their effort to fit both quantized activations and weights, cause their internal "latent FP32 weights" to severely deviate from the original model. This leads to a seemingly high ImageNet accuracy (81.9%) but far inferior generalization ability (60.73%) compared to the FP32 model (70.79%) and our GPLQ (71.18%). This indicates that improper handling of activations can force the weights into a "bad" optimization basin, thereby harming generalization. GPLQ successfully avoids this pitfall by prioritizing the stabilization of activations.

In summary, we believe that GPLQ is not just a clever combination, but a new quantization framework driven by deep insights and possessing broad generality. We are using further experiments to clearly define its scope of application and believe it has the potential to become a practical and efficient solution for low-bit quantization across various model architectures.

Thank you again for your valuable feedback!