We appreciate your insightful comments and give the response as follows.

Q1 Generalization on ImageNet: Thanks for your valuable suggestion. We try the generalization performance of the merged model on unseen tasks like Classification task (ImageNet1k). We follow the setting in our paper, i.e., merging eight expert models, and then verfiy its general capability on the test set of ImageNet1k. The backbone is ViT-B/32 and the baseline is Task Arithmetic. The results in the table below show that the baseline Task Arithmetic method achieves 27.6% top-1 test accuracy, while the Surgery and ProbSurgery methods improve it to 48.5% and 52.7%, respectively. Thus, we believe our proposal represents a significant advancement in enhancing the generalization performance of existing model merging techniques.

Q2 ProbsurgeryV2, tntegrating Probsurgery into each block: Thanks for this valuable suggestion. Our proposed ProbSurgery also can integrated into each block of the ViT-based model to surgery the layer (or block)-level representation bias, which is more refined and contributes to better performance. By adopting the design in Surgery V2, we conduct experiments on Task Arthmetric with ViT-B/32 and show the result in the following table.

Q3 Typos: Thanks for pointing out these typos. We will correct these typos in the next version of our manuscript.

Q4-1 Other merging methods. Our method is applicable to all model merging approaches, not just those based on task vectors. The table below reports the performance of three non-task-vector methods—Fisher Merging, RegMean, Task Arithmetic + DARE, and EMR-Merging — using ViT-B/32 as the backbone. We can see that applying ProbSurgery consistently improves average accuracy. In conclusion, we believe that ProbSurgery's effectiveness has been fully validated.

Q4-2 Improvement in Adamerging: The improvement observed with Adamerging is not significant, as it is a more advanced merging method that results in representation bias during merging (as shown in Fig 4, a smaller distance is achieved by Adamerging). Therefore, when integrated with ProbSurgery, the improvement is not considerable.

Q4-3 NLP tasks: Since different NLP tasks always enjoy the same parameters in almost all layers or blocks given a model like Bert, it exhibits fewer representation bias compared to vision tasks, which are highly sensitive to parameter changes. This difference leads to minimal performance degradation when merging multiple NLP tasks. Thus, integrating our proposal with NLP tasks yields a relatively smaller performance improvement than with vision tasks.

Q5 Merging fewer models: We believe that any model fusion process, regardless of the number of models involved, induces parameter changes that lead to representation bias. Consequently, our method can alleviate this bias and enhance performance. Due to the word limit, we merged three models and present the results in the following table. Notably, our method also demonstrates performance gains even when merging a small number of models.

We appreciate your insightful comments and give the response as follows.

Q1 ProbsurgeryV2, integrating Probsurgery into each block: In this paper, ProbSurgery is integrated into the last layer. However, it also can be integrated into each block of the ViT-based model to surgery the layer (or block)-level representation bias, which is more refined and contributes to better performance. By adopting the design in Surgery V2, we conduct experiments on Task Arthmetric with ViT-B/32 and show the result in the following table. When our method is applied after every block, it can more effectively reduce representation bias and achieve better performance.

Q2 Integrating with other methods: Due to page restriction, we keep the same setting with Surgery that only conducts experiments on four baselines based on task vectors. Our method ProbSurgery is completely orthogonal to existing model merging approaches and can be incorporated into any model merging method to solve its representation bias problem. In the following table, we integrate ProbSurgery with other three methods that are not based on task vectors, including Fisher Merging, Regmean, and EMR-Merging. By integrating our method, each baseline sees a substantial boost in performance, highlighting the effectiveness of our approach in mitigating representation bias.

Q3 The size of the validation set Thanks for this constructive suggestion. We made an experiment to verify how the size of the validation set impacts ProbSurgery's performance. In the following table, we can see that using a larger proportion of the validation set helps the ProbSurgery module better capture and mitigate inherent biases, leading to higher overall accuracy. Notably, even when only 10% of the unlabeled validation data is used, our method still exceeds Surgery’s performance of 80.9%, demonstrating its robustness under limited data conditions.

Q4 Merging tens of models: Thanks for this constructive suggestion. To verify the performance in more challenging settings, we additionally add three tasks (ImageNet100, CIFAR100, and real-world Homeoffice) to original eight tasks. The results in the following table show that even when merging a large number of models, our method can effectively correct the representation bias and achieve superior performance.

We appreciate your insightful comments and give the response as follows.

Q1 Computational Complexity: In practical implementation, we only sample once to generate the representation bias in the ProbSurgery module, which does not incur any additional cost compared to the deterministic method. Our results demonstrate that a single sample is sufficient to achieve state-of-the-art performance—outperforming the previous Surgery method. In the next version of our manuscript, we will provide further insights and discussions on how the number of samples impacts the algorithm's performance.

Besides, we agree with your viewpoint that sampling multiple times can probably improve the estimation accuracy of the representation bias, thereby enhancing overall model performance. Specially, we sample $\rho$ times and compute their average values during model training. However, even if we perform multiple samplings, it does not incur extra computational cost since this averaging operation only involves first-order derivatives during backpropagation. In the following table, we try different sampling numbers and compare their performance. We can observe that increasing the sampling number does not lead to a significant improvement in overall performance. Therefore, to ensure better scalability, we have chosen not to introduce this hyperparameter explicitly, and instead, we set it to 1 by default.

Q2 Sensitivity to Hyperparameters: Based on our empirical experience, we have set the trade-off coefficient for the KL term to $1 \times 10^{-3}$. To further explore the algorithm's sensitivity to this hyperparameter, we conducted experiments with Weight Average on ViT-B/32 and show the result in the following table, which shows that the algorithm's average accuracy remains nearly constant across various hyperparameter values, demonstrating that our method is insensitive to this parameter.

Q3 Gaussian Assumption for representation: In this work, we treat the representation bias as a Gaussian mainly for tractability and flexibility. The normal distribution provides a compact parameterization ($\mu$ and $\sigma$) that is easy to optimize, while also serves as a high-entropy “baseline” choice—minimizing assumptions about the true shape of the bias. Although the real distribution may deviate from exact Gaussianity, empirical tests show that this assumption reliably captures the dominant uncertainty in merged representations.

We appreciate your insightful comments and give the response as follows.

Q1 The observation: Here, $G(\cdot)$ quantifies the difference in test accuracy, which reflects the average discrepancy between the feature representations produced by these two models. For a more formal representation, we extend the function $G(\cdot)$ to: $E_{t\sim[T]} \Big[\text{Acc}(f_{\theta_t}, D_t) - \text{Acc}(f_{\theta_{unif}}, D_t)\Big] \propto E_{t\sim[T]} \xi_t,$ where $\text{Acc}(f, D)$ denotes the test accuracy of model $f$ on the test set $D$. Detailed explanations will be added to our manuscript in subsequent revisions.

Q2 Upper performance in Fig2: The upper performance in Figure 2 denotes the average performance of various expert models on corresponding merged tasks.

Q3 More baselines: Due to page restrictions, we keep the same setting with Surgery that makes experiments on four methods based on task vectors. (Prob)Surgery is completely orthogonal to existing model merging methods and can be incorporated into all of them to mitigate the representation bias. In the table below (more detailed results can be found in the response to Reviewer HxL4, Q4-1), it shows that integrating (Prob)Surgery into existing baselines significantly improves performance. This confirms that (Prob)Surgery effectively mitigates representation bias in model merging and can be seamlessly incorporated into any merging approach.

Q4 Format of Eq. (4): We follow the objective in Eq. (1) to maintain the core motivation of Surgery, namely to remove the representation bias introduced by the merged model. Concretely, $z_{i,t}^{\mathrm{unif}} - \xi_{i,t}$ represents the corrected (post‐calibrated) representation rather than the raw merged representation $z_{i,t}^{\mathrm{unif}}$. Extending from a deterministic correction term $\xi_{i,t}$ in Eq. (1) to a probabilistic treatment in Eq. (4) lets us more accurately capture the uncertainty arising from parameter interference during merging.

Q5 Statement in Lines 217-219: The “limited representational capacity” refers to the deterministic nature of the Surgery module. While deterministic modeling can be simpler and faster to train, it struggles to capture the inherent uncertainty of multiple merged tasks. As Figure 1 shows, even after applying a deterministic correction, the merged features remain partially misaligned with each individual model’s distribution, implying that a single deterministic module does not generalize well beyond its initially learned task. Besides, “other tasks” denote the remaining tasks in the merged set, which the single deterministic Surgery module fails to accommodate.

Q6 Theorem 4.1: In McAllester (2003), the statement closest to the paper’s Theorem 4.1 is Theorem 1, often referred to as the “PAC-Bayesian Theorem”. It provides a general PAC-Bayes bound on the expected loss of a posterior hypothesis in terms of its empirical loss plus a KL‐divergence term from the prior—precisely the structure adapted in ours.

Q7 Expectation in Eq. (9): Eq. (9) measures how a predicted distribution $Q_{\omega}$ fits the observed (true) distribution $P^{\mathrm{bias}}$. We take the expectation of that score with respect to the “true” distribution $P^{\mathrm{bias}}$: $S\bigl(Q_{\omega}, P^{\mathrm{bias}}\bigr) =\int S\bigl(Q_{\omega}, z^{\mathrm{bias}}\bigr)\mathrm{d}P^{\mathrm{bias}}(z^{\mathrm{bias}}).$ Finally, it is written as $\mathop{\arg \min}\limits_{\omega} \mathcal{S}(Q_\omega, P^{\rm bias}) := \mathop{\arg \min}\limits_{\omega} E_{z^{\rm bias} \sim P^{\rm bias}}[Q_\omega, z^{\rm bias}]$, as shown in Eq. (9). Due to only one observation $z^{\rm bias}$ (a fixed bias) in practice, this equation can omit the expectation form and be reformulated as $\mathop{\arg \min}\limits_{\omega} \mathcal{S}(Q_\omega, P^{\rm bias}) := \mathop{\arg \min}\limits_{\omega} \mathcal{S}(Q_\omega, z^{\rm bias})$. Thus, in Eq. (10), we only have one variable $z^{\rm bias}$ and estimate the distribution $Q_\omega$ via sampling.

Q8 Validation set: First, the post-training step in (Prob)Surgery relies on an extra validation set that does not require any labels. Acquiring such a small-scale set is feasible in the real world since no label annotations are required. Meanwhile, to ensure a fair comparison, we employ test-time adaptation (TTA, adopted in many model merging methods like Adamerging) to post-train a merged model with Task Arithmetic. The results in the table below clearly demonstrate that our method significantly outperforms the TTA approach.

Q9 Typos: Thanks for pointing out these typos. We will correct these in the next manuscript.