Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation

First and foremost, we sincerely appreciate your thoughtful comments.

Concern 1 : Limited merging setting

We emphasize that our work deliberately targets the most challenging scenario for enabling domain adaptation on resource-constrained edge devices. Specifically, we design our setting to be demanding in the following four aspects:

1. Edge devices lack sufficient resources for fine-tuning.
2. State-of-the-art PTQ algorithms are also too costly for such devices.
3. To minimize communication costs, only quantized models can be deployed.
4. Merging must remain lightweight.

While exploring boundary-breaking methods is important, we also believe addressing practical and challenging real-world deployment scenarios is equally critical. From this view, complex, data-intensive merging methods fall outside our scope. Given the strict resource constraints and latency sensitivity of edge environments, merging must remain lightweight.

With this in mind, we adopt the interpolation-based merging strategy proposed in [1]. Notably, our baseline outperforms prior MTDA techniques, demonstrating its strength as a representative of state-of-the-art trends in model merging.

[1] Wenyi Li, Huan-ang Gao et al, “Training-Free Model Merging for Multi-target Domain Adaptation”, ECCV 2024

Concern 2 : Attribution of accuracy degradation

As shown in Table 1 (a) and (b), quantization degrades each model's quality and negatively impacts merged performance. However, when quantization is done with merging in mind, quality is better preserved. In Table 1 (a), HDRQ matches QDrop in sole quantization but significantly outperforms it in harmonic mean(1.69 mIoU / 4.14 mIoU, before/after correction). This highlights the importance of merging-aware quantization.

Concern 3 : Baseline performance

We acknowledge that HDRQ initially underperformed QDrop and have devoted significant effort to address this. We identified a minor bug in our implementation of HDRQ—the quantization step size was not updated during training—leading to negatively biased results.

Corrected results are shown below:

HDRQ generally outperforms QDrop at lower bit-widths of weight (W4A8, W4A4, W3A3). In 11 wins, HDRQ’s average margin is 0.95—nearly double the 0.41 margin across 9 losses. These results demonstrate HDRQ’s consistent superiority over QDrop in many scenarios. We will revise all results in the final paper. Thank you again for the feedback led to this correction.

Concern 4 : Comparison with SoTA PTQ method

While we recognize the importance of comparing with recent methods, we note that BRECQ and QDrop remain strong and reproducible baselines despite earlier publication. Still, we agree on the importance of comparison with recent methods and conducted further experiments.

Specifically, we included FlexRound as a recent baseline, but excluded PTMQ due to its multi-bit focus and weak single-bit performance. We also tried to reproduce Shang et al.’s method, but the lack of official code and limited time prevented inclusion.

We validated implementation using 4W4A quantization on ResNet-18 with the official PyTorch checkpoint (top-1: 69.76%). FlexRound achieved 67.63%, and QDrop 67.87%, consistent with original reports (~0.2%p difference).

Then we compared our method and FlexRound on semantic segmentation under 4W4A setting:

While FlexRound is slightly better standalone, HDRQ achieves a higher harmonic mean-highlighting the advantage of merging-aware quantization.

Concern 5 : Model merging strategy

As detailed in our motivation, our goal is real-time model merging on edge devices. While advanced methods could improve merging quality, edge devices often lack resources to handle complex techniques. Thus, we adopt interpolation-based merging as our default, given the limited research on quantization-merging interplay. We hope this work offers valuable insights in this direction.

We also acknowledge the missing key references and will include the suggested citations in the final version.

Thank you for your insightful review and constructive feedback on our submission.

Weakness 1 : Complicated data and tasks

We agree that experiments on more complicated tasks are truly helpful. We appreciate your suggestion and will consider additional experimental results.

Due to the limited resource and time, we are not able to produce full experimental results for domainnet, but we here provide partial merging results of 3W3A quantized DA models from Real → Clipart, Sketch, Painting:

In this moment, we hypothesize HDRQ to work well with larger domain shift, since domain adapted weights are expected to still remain in single basin, where our theoretical analysis are valid. We expect our method to either surpass baseline methods or show comparable performance in other settings, as observed in the Office-Home dataset. We will include the full experimental results in the camera-ready version. Thank you for your invaluable suggestion.

Thank you for your valuable feedback. We've covered your key points and clarified our work.

Weakness 1 : Comparison with QAT

Quantization-aware training (QAT) typically yields higher quantization quality compared to post-training quantization (PTQ), but it demands full access to training data and significantly more computational resources. In our setting, which centers on per-device domain adaptation, the available target data is often too limited to support end-to-end fine-tuning without risking overfitting. To ensure practical applicability, we focus on PTQ and compare our approach against baseline PTQ methods.

Weakness 2 : latency/runtime comparisons

In our setup, fine-tuning and PTQ are done on a server, while only model merging and inference run on the edge device. Since the quantized models retain the same bit configuration and granularity, their latency and runtime remain unchanged after merging.

As a result, the only remaining computational component on the edge device is the model merging process. To clarify its cost, we measured the time on a Raspberry Pi 5 using MobileNet-V2. The process took 35.53 seconds with 100 noise sampling iterations on an ARM Cortex-A76 CPU—about 3.60 seconds per 10 iterations—showing that merging is feasible even on low-power devices.

Weakness 3 : Highly heterogenous domains

Domain adaptation typically uses a lower learning rate and fewer epochs than training from scratch, so we assume that the resulting weight deviations are not large enough to escape a single basin—even under significant domain shifts.

However, we acknowledge that if such a case occurs, additional treatments may be required to maintain the quality of quantized networks.

We conducted a partial experiment on DomainNet (Real, Clipart, Sketch, Painting), a more complex dataset, and observed consistent improvements over baselines. Please refer to our response to Reviewer 5KmE’s Weakness 1.

Question 1 : Sharpness measure

As a measure of sharpness, we report the Hessian trace of the Real→Clipart domain-adapted model quantized to W4A8 on the Office-Home benchmark. The table below shows that HDRQ exhibits a lower Hessian trace, indicating better flatness:

Question 2 : Multiple runs

We conducted five additional experiments with different seeds and report the mean and standard deviation:

We would like to sincerely apologize for an error identified during the revision process. Specifically, we discovered that in our earlier experiments, the quantization scale in our method was not properly updated, which led to the reporting of degraded accuracy. In the updated results presented in this table, we have corrected this implementation bug. As a result, the overall accuracy has improved significantly. While we deeply regret this oversight, we also hope for your understanding that this issue introduced a negative bias in evaluating the performance of HDRQ.

Question 3 : Other architectures

Since our analysis is agnostic to model architecture, we expect it to generalize well to vision-oriented models. However, architectures like transformers—which often exhibit unique characteristics such as outlier channels—may require additional techniques to preserve quantization quality. That said, existing baseline methods are also likely to experience similar degradation in such cases. We plan to include results on ViTs and other architectures in the final version.

Question 4 : Runtime, memory comparison

Here is a table comparing the maximum memory consumption and quantization time of baseline methods and ours:

As shown, quantization time slightly increases over QDrop due to noise sampling at each iteration. Theoretically, memory usage may also rise slightly, as sampled noise must be stored for each gradient step. However, actual measurements via torch.cuda.max memory allocated show similar memory consumption due to PyTorch's caching mechanism. From the result, this allows us to assume that the memory overhead of HDRQ is minimal.

In summary, these increases in memory consumption and overall time are minimal and represent an affordable cost.

The inference speed of models quantized with each method remains the same, as the quantized models have the same bit configuration and quantization granularity.

Question 5 : Ablation of noise-sampling-based rounding technique

The ablation study is provided in Figure 3 of the main paper, where "Naive" denotes merging without advanced sampling. For the merging of ambiguous cases with multiple valid quantization levels, the Naive method assigns one at random. Advanced sampling filters out low-quality samples, improving average performance and reducing distribution tail size.

We sincerely thank you for thoughtful feedback and constructive comments.

Weakness 1 : Main motivation

To begin with, we would like to emphasize that our work deliberately addresses the most challenging scenario for enabling domain adaptation on resource-constrained edge devices. Specifically, we design our setting to be highly demanding in the following four aspects:

1. The edge device lacks sufficient resources to support fine-tuning operations.
2. State-of-the-art PTQ algorithms are also too computationally expensive for such devices.
3. To minimize communication costs, only quantized models can be deployed.
4. Model merging operations must remain lightweight.

From this perspective, as some reviewers pointed out, an alternative approach—merging two unquantized models first and then applying PTQ—could be a viable solution. However, this approach becomes less practical when progressive adaptation is considered. In the case of progressive adaptation, additional merging must be performed on top of the previously merged weights. In our proposed scenario, the target model is expected to accumulate updated weights locally over multiple rounds. Since the server discards the personalized model after each fine-tuning round, this approach imposes a strong constraint. Nonetheless, it provides significant security advantages, as personalized models are not stored centrally. By contrast, in the alternative approach, the server must maintain a separate customized model for each device, which leads to increased storage overhead and raises potential concerns regarding information leakage. Please note that our proposed scheme works really well on our challenging scenario, showing the outstanding performance of the proposed idea.

Weakness 2 : Results vary with changes in the hyperparameter λ

Here, we present the results of model merging with varying values of $\lambda$. As shown in the table below, increasing $\lambda$ initially improves performance due to the regularization effect. However, beyond a certain point, excessive regularization starts to harm the quality of reconstruction during the quantization process, leading to accuracy degradation. Based on empirical observations, we selected $\lambda = 5 \times 10^{-2}$ as a balanced choice that consistently performs well across different settings. We adopt this value uniformly in all experiments throughout our paper.

Weakness 3 : Discrepancy in highlighted number

Sorry for the confusion, and thank you for pointing that out. As you correctly noted—particularly in Table 1(a)—the performance of QDrop should be more clearly emphasized. Our main intention was to highlight the quality improvement achieved after model merging, as demonstrated in Table 1(b). To avoid misunderstanding, we will revise Table 1(a) to clearly indicate that QDrop is the best-performing method for standalone quantization.

Other comments : Comparison with AWQ, SmoothQuant, SpinQuant, or GPTQ

In our experiments, we apply per-tensor, round-to-nearest quantization with truncation. Notably, the truncation range is learnable, following the formulation used in QDrop.

AWQ, SmoothQuant, SpinQuant, and GPTQ are designed specifically for LLMs, and thus differ fundamentally from gradient-based methods like QDrop. These algorithms perform progressive, layer-wise quantization without iterative updates to reduce the computational burden—an essential property given the vast number of parameters in LLMs. However, in the case of vision models, QDrop often outperforms these LLM-oriented techniques. For this reason, our primary focus is on vision-oriented PTQ approaches.

That said, we agree it is also valuable to assess the effectiveness of our proposed method under more advanced PTQ schemes, even if they were originally developed for LLMs. To this end, we conducted 4W4A quantization experiments on semantic segmentation using SmoothQuant, and compared our method against relevant baselines:

As shown in the table, ours performs significantly better than SmoothQuant, especially in the low-bit W4A4 case. We will update the results in the camera-ready version.