DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation

Q1) How well does DaWin generalize to tasks beyond classification benchmarks (e.g., NLP or reinforcement learning)?

Answer: Although the scope of this work is limited to classification scenarios as noted in Section 2 of the manuscript, to investigate the applicability of DaWin beyond the classification setup, we additionally conduct experiments on the NLP domain.

• By following LoRA Hub [4] setup that adapts FLAN-T5 [10] with LoRA [3], among 196 trained LoRA weights, we randomly sampled 20 LoRA weights and conducted the dynamic interpolation per downstream tasks (Big Bench Hard [5]).
• Different from the image classification setup, we use the negative log-likelihood (NLL) and perplexity (PPL) as model expertise measures that are tailored to the autoregressive language model evaluation.
• Then, we leveraged few-shot samples (five input-output pairs used in the LoRA Hub method) to compute NLL and PPL. The average performance (exact matching) on 25 tasks from Big Bench Hard is in the table below.

• We can see that DaWin using PPL and NLL both show better average performance compared to LoRA Hub which requires additional training per each downstream task.
• Due to the lightweight nature of LoRA, DaWin only consumes 13% more memory allocation compared with the zero-shot pre-trained model inference.

Q2) Can the reliance on entropy as an expertise proxy limit DaWin's performance in certain scenarios, such as multi-modal data?

Answer: As we mentioned in Section 2 of the manuscript, we confined the scope of this work to the multi-class classification problem and accordingly derived a reasonable proxy for the model expertise. If we consider multi-modal input-output data distribution with corresponding multi-modal models such as CoCa [11] and LLaVA [12], we may need to design new suitable expertise metrics tailored to the class of model rather than the output entropy which is suitable for the classification model. Investigating a new score tailored to those multi-modal models will be crucial for future work direction, and we provide the results on the NLP domain as an example for this extension, e.g., using negative log-likelihood or perplexity of language models' output as a proxy of model expertise in the above paragraph.

W3/Q3) Trade-offs not fully explored: The paper does not sufficiently address the performance trade-offs between per-sample interpolation and simpler methods like weight averaging. How does DaWin compare in terms of inference speed and energy consumption to AdaMerging and WiSE-FT?

Answer1: We already provided the trade-off discussion as a separate paragraph in Section 6 of the manuscript including the inference speed, and we additionally provide results for other metrics here.

• We observed that DaWin can achieve outstanding classification accuracy that is comparable to or even better than training-intensive methods (AdaMerging) or fully sample-wise interpolation methods, while slightly increasing the wall-clock runtime compared to the hyperparameter tuning-intensive static interpolation methods.
• Moreover, to further investigate the trade-offs between DaWin and a simpler method, we provide results on additional metrics (FLOPs and peak memory allocation) and variants of DaWin, in the table below.

• Here, DaWin (parallel) denotes the setting where we parallelize the evaluations of individual models during the expertise estimation phase before conducting the interpolation. DaWin shows three times FLOPs than WiSE-FT, but the actual runtime is less than twice that of WiSE-FT because DaWin does not require intensive hyperparameter tuning, unlike WiSE-FT. Besides, the computational complexity can be further reduced by adopting the parallel inference pipeline for the model expertise estimation phase of DaWin.

Answer2: Comparison for the energy consumption can be also discussed through the GPU hours (that would be multiplied with a GPU-specific power-consumption quantity), GPU memory allocation, and FLOPs (those are relevant to algorithmic efficiency in terms of required operation on CPUs and GPUs).

• DaWin could not be better than the simpler method WiSE-FT in terms of energy consumption given increased runtime, FLOPs, and memory requirement.
• However, not only for better energy efficiency, we believe that ensuring robustness under distribution shifts (e.g., good OOD accuracy which can be remarkably improved by DaWin) is another crucial value that we should pursue for reliable AI as well [20].

Dear Reviewer yGMX, The authors are delighted that you find some advantages of our methods, including training-free characteristics, novel use of entropy, state-of-the-art performance, and computational efficiency! Your constructive comments made authors further deliberate on the limitation of the proposed method and raised a bunch of new insightful analyses that we are presenting through the responses below.

---

W1) Incremental novelty: While the method provides refinements over static interpolation, it shares similarities with existing methods (e.g., AdaMerging), which reduces its overall originality.

Answer: DaWin has unique design motivation and advantages over existing methods. We would like to respectfully refute the novelty concern by providing three perspectives as below.

• First, DaWin aims to address dynamic interpolation scenarios in contrast to AdaMerging which conducts a single interpolation and uses the model for all test samples. Meanwhile, based on the findings of our pilot study, we are motivated to build a dynamic interpolation method to achieve par better performance over static interpolation.
• Second, unlike existing dynamic interpolation methods that require additional training and extra modules to compute per-sample interpolation coefficients, DaWin is a training-free method that can instantly be adopted given (pre-)trained models.
• Lastly, DaWin pursues efficiency by leveraging the Beta Mixture Modeling approach which significantly reduces the amount of inference-time interpolation operations, and this design philosophy also discerns our method from previous dynamic interpolation methods.

W2/Q4) Over-reliance on entropy: The assumption that entropy reliably measures model expertise across all scenarios is not thoroughly validated, and alternative metrics could be explored further. What scenarios, if any, might lead to simpler methods outperforming DaWin, and how can this be systematically explored?

Answer1: We already experimented with some alternatives (X-entropy loss based on different pseudo labels) in Figure 5 the experiment section of the manuscript. Those are representative replacements for entropy. The results showed that entropy achieves the best results overall.

• We further conducted some experiments on other metrics for model expertise estimation: 1) maximum logit ratio, 2) exponentiated confidence ratio, and 3) exponentiated top1 and top2 confidence difference. The results are in the table below.

• As shown in the table above, our entropy ratio-based design outperforms alternatvies in this setup.

Answer2: We investigate a scenario when a simpler method can be preferred rather than DaWin.

• As Reviewer yGMX noted, the entropy over individual test samples does not always reliably estimate each model's expertise if the uncertainty calibrations of individual models are bad.
• We can investigate a potential failure mode of DaWin by controlling the level of uncertainty calibration of individual models, as the interpolation coefficients are computed from the output probability distribution through the entropy function.
• In the table below, we analyze the effect of the individual model's calibration by dividing the pre-softmax outputs of each model with a scalar value.

• As we can see, bad uncertainty calibration (manually adjusted toward under-confident prediction) of individual models makes DaWin fail to reliably estimate the per-sample coefficients and a simple method can be preferred in terms of ID and OOD performance trade-off.

Q) Did they see any trends in where they obtained similar interpolation coefficients for samples? Maybe certain clusters of samples are always better explained by a given model or not. Are there conclusions one could draw about the test set from this ?

Answer: Yes. We observed that in the robust fine-tuning (ImageNet) setup, the estimated coefficients are diverse within a dataset, whereas in the multi-task learning (Specialized visual recognition) setup, the estimated coefficients are mostly similar across samples within a dataset.

• Explanation) If there exist some dominant expert models for a test set, and the train sets of those dominant experts are remarkably similar to the test set compared to non-dominant models' ones, the estimated interpolation coefficients will be similar across test samples. Meanwhile, the opposite case (coefficients are diverse across samples) implies that the degree distributional similarities between the test set and train set of each model are roughly equal.
• As an extreme example, let's assume the train datasets of individual models are highly specialized to some domains that have narrow distributions, and the test set is also one of highly specialized task-specific data. Then, DaWin will induce similar coefficients over entire test samples that are skewed toward a model whose train distribution is the most similar to the test distribution.
• To systematically investigate this, we simulate some scenarios in which models used to merge have different levels of relevancy to the target test dataset.

• The above table presents the standard deviation (STD) of DaWin's per-sample estimated coefficients for each scenario.
• We observed that, in the skewed relevancy setups where one of the models has remarkably better expertise compared to another one, the STDs are much smaller than that derived from balanced relevancy setups where two models are trained on similarly relevant datasets or similarly irrelevant ones regarding test distribution.
• We conclude that the similarity among obtained coefficients depends on the relation between the test dataset and the datasets where individual models are trained.

Dear Reviewer uT7T, The authors appreciate your productive feedback (clarity suggestions and a valuable question), which helped the authors significantly refine the paper overall, and we are thankful for acknowledging the strengths of this paper: clear presentation, including motivating example (GT scenario), solid experiments, and trade-off analysis. We provide the responses to your comments as below.

---

Clarity: There are a few parts of the paper that are lacking details or are confusing: 1a. in the intro a foundational model != ImageNet. People refer to foundational models primarily for large scale models trained on vast amounts of data (GPT,Gemini,Flamingo)

Answer: In the introduction section of our manuscript, we do not use the term 'foundation model' to mean the ImageNet pre-trained model. The ImageNet and ImageNet-A are mentioned to illustrate the standard setup of robust fine-tuning scenario [6,7,8] where we fine-tuned a foundation model (CLIP). The CLIP-like vision-language models have been regarded as a representative class of foundation models [9,10,11] besides with large language models.

Clarity: 1b. It is unclear how precisely H(-) is computed. I appreciate this is probably obvious to the authors but it would be helpful to briefly mention.

Answer: Does reviewer uT7T wonder about how precise the estimation of model entropy is to the entropy over true conditional data distribution $P(Y|X)$?

• Unfortunately, we can not compute the true entropy because we do not have the true data-generating process on the benchmark datasets, so can not measure the quality of entropy estimation.
• However, as we adopted entropy as a proxy of X-entropy, we can alternatively compute the correlation between entropy and X-entropy. The Pearson correlation coefficients between entropy and X-entropy are provided below.

• As we can see, each model's entropy largely shows moderate correlations [18] with the X-entropy computed with ground truth labels.

Clarity: 1c. I am unconvinced by their linear mode connectivity explanation. My understanding is the point of the linear mode connectivity is that along that line between model weights, the error is bounded to some height. But this is not what's being checked -- it's being done for a single interpolated example.

Answer: We derived a new hypothesis analogy to linear mode connectivity (LMC) which is adapted to entropy setup. By reflecting the Reviwer uT7T's concern, we conduct an experiment to empirically verify this hypothesis.

• The sample-wise entropy valley hypothesis claims that 1) there exists a global coefficient $\lambda$ that the average entropy of the interpolated model is smaller than the minimum of two models' average entropy, 2) there also exists a function $\lambda(x)$ that determine per-sample coefficients such that average entropy from the interpolated models becomes much smaller than that of a static interpolation.
• In Figure H of the Appendix in the revised manuscript, we present individual models' entropy, static interpolation models' entropy, and dynamic interpolation models' entropy. The results aligned with our two hypotheses and therefore empirically support our statements (Please refer to Figure H of Appendix).

Dear reviewer uT7T,

The authors want to express their appreciation for your productive feedback and for taking the time to review this work. We have presented our rebuttals with additional experiments to address your concerns. The authors are wondering whether these are addressing your concerns. We are sincerely looking forward to further discussing our work with you who has remarkably contributed to improving the quality of the manuscript so far.

Best regards,

The authors

Q3) Could the proposed method result in negative weight mixtures for some datasets? If so, why?

Answer: We use the softmax function to ensure that DaWin's coefficient vectors have elements ranging from 0 to 1. Although DaWin can generate zero-approaching values for some experts that produce higher entropy than other models, it cannot have negative values.

W5) In real applications, entropy can be misleading, resulting in overconfident but incorrect predictions. The model design does not address how to correct these wrong predictions.

Answer1: As Reviewer Rb5n noted, the entropy over individual samples does not always align with the model's predictiveness because there are inevitably some corner cases in which a model produces over-confident prediction while it is incorrect. However, we indeed consider some techniques to address those overconfidence and incorrect individual predictions in our design of the method.

1. We adopted temperature scaling [15], which is mentioned in the implementation detail section of the Appendix.
2. Moreover, as we noted in L255 of Section 3.2., we leverage domain-specific offset adjustment to refine the unstable sample-wise entropy ratios of some corner cases. This offset adjustment consistently improved the performance in all cases (Please refer to Table A of the Appendix for the ablation study).

We revised the manuscript to inform this more remarkable manner at the end of Sec 3.2.

Answer2: The rationale behind using entropy is laid on the fact that it can well approximate X-entropy (the desired expertise metric). In the case where entropy is not a suitable model expertise metric such as the autoregressive language model [12,19], alternative quantities such as negative log-likelihood (NLL) or perplexity (PPL) can be adopted as an expertise metric.

• To simulate such a setting, we conduct an experiment on an autoregressive language model.
• By following LoRA Hub [4] setup that aims to fine-tune FLAN-T5 [10] via LoRA [3], we randomly sampled 20 public LoRA weights and performed the dynamic interpolation per downstream tasks (Big Bench Hard [5]).
• We adopt the negative log-likelihood (NLL) and perplexity (PPL) as model expertise metrics and leverage few-shot samples (five input-output pairs used in LoRA Hub) to compute NLL and PPL. The average matching accuracy on 25 tasks is in the table below.

• We see that DaWin using PPL and NLL both show better performance compared to LoRA Hub which requires additional training per each downstream task.

Q1) the dynamic weighting strategy per sample is another form of confidence-based selection. Given this, what if we fuse the outputs instead of the model weights? Do the tasks (e.g., ImageNet - ImageNet-A) reveal any trends in the weighting of these models?

Answer: Yes. The entropy ratio-based weighting strategy can be leveraged to fuse the model output as well as model parameter, and we already conducted the applications of DaWin for output ensemble setup in Table 5 of our manuscript. We provide a full table for the dynamic output ensemble (DOE) method.

• Similar to DaWin (weight interpolation), DOE shows relatively weak performance on ID (ImageNet) and easy OOD (ImageNetV2 which is generated by reproducing the ImageNet data collection pipeline a decade later) while outperforming on more semantically different OOD datasets compared with static weight interpolation baseline (WiSE-FT).

Q2/Q4) How can we determine whether the performance is correlated with the distribution distances between datasets? Is it possible that some checkpoints could decrease the final performance on specific datasets? Furthermore, do the semantic distances and relationships between classes from different datasets (and their corresponding checkpoints) affect the performance of one another?

Answer1: We can systematically analyze the relation between distributional distances between datasets and the performance of merged method by constructing a model pool for merging based on the distance between train datasets (where the candidate models are trained) and test datasets. Please refer the Appendix Figure G of our revised manuscript to see the the heatmap of pairwise dataset distances.

• We simulate two scenarios: 1) all models are relevant to solve the downstream task, and 2) some models are relevant but others are not to solve the downstream task. The results are in the table below.

• Although the baseline methods perform well when we have expert models trained on datasets that are relevant to the evaluation dataset, they suffer from performance degeneration as the non-expert models are included in the merging pool. This indicates that there are some interferences between models (and train corresponding datasets) that hurt post-merging performance if we inappropriately determine the interpolation coefficient.
• Meanwhile, DaWin takes benefits from even non-expert models by leveraging entropy ratio-based weighting strategy, and the performance is improved as more models participate in merging.

Answer2: We further analyze whether we can take any insight on the impact relationship between classes from different datasets on DaWin.

• Characterizing semantic distances between classes from different datasets, and relating them with post-interpolation performance is a non-trivial problem. Given that DaWin relies on the output probability distribution of each model, we measure and analyze the pairwise kullback leibler divergence (KLD) over the averaged per-class output probability distributions from each fine-tuned model.
• The mean of pairwise KLD over per-class output distributions measures how different the output probability distribution is between each class, and is related to class-wise distinctiveness and separability in the output space.
• We hypothesize that as the mean pairwise KLD of the dataset is lower, the strength of the model (trained on that dataset) expertise becomes weaker, and does not much contribute during merging when evaluating the training dataset itself or other datasets. The results are in the table below.

• We see that the mean of pairwise KLD of each dataset somewhat negatively correlated with the magnitude of the coefficient of each corresponding model on the training dataset and other datasets. For example, the model trained on Cars dataset shows the lowest contribution during merging when evaluating on Cars or other datasets.
• We speculate that class-wise relationship and separability over a dataset can be a measure of the strength of expertise of the model trained on that dataset, which is crucial to the success of DaWin.

W4) There is limited discussion on why DaWin performs well or poorly on specific datasets or distributions (e.g., SUN397 and Cars). Additionally, relationships among datasets are critical but remain under-explored.

Answer: We provide a hypothesis and supporting empirical evidence on the performance variation of DaWin depending on the characteristics of datasets, and further provide a suggestion for successful usage of DaWin. To summarize, DaWin's success may depend on the difference between a number of classes of train datasets.

• We appreciate Reviewer Rb5n's constructive suggestions on further analysis. By reflecting the feedback, we compute the distance between datasets with mean maximum discrepancy (MMD) and Frechet inception distance (FID) to analyze the performance of DaWin in terms of relationships between datasets. Please refer the Appendix Figure G of our revised manuscript to see this.
• To deepen our understanding of dataset-specific performance variation of DaWin, we analyzed the proportion of samples that the true expert (e.g., model fine-tuned on DTD train split for DTD evaluation) produced the smallest entropy so that get the largest weight for merging among individual models. This is directly matched with the entropy-dominancy assumption that we assume in Lemma 6.1. as DaWin's working condition.

• Observation: In the table above, we can see that SUN397 and Cars datasets, where DaWin's performances are farthest from the ideal individual models' performances, show the lowest proportion of samples that meet entropy-dominancy assumption thereby incorrectly weighing each model.
  • Although SVHN shows the lowest proportion in terms of 'most relevant model', SVHN dataset shows strong similarity with MNIST dataset in terms of MMD and FID, and the summation of the entropy-dominancy samples' proportions in those two datasets approach 100% likewise other datasets.
  • However, in SUN397 and Cars datasets, roughly 85% of samples only meet the entropy-dominancy assumption and 15% of samples fail to correctly give the largest merging weight for the true expert.
  • Therefore, we speculate that DaWin's success or failure depends on the proportion of samples meeting entropy-dominancy assumption.
• Possible explanation: We conjecture that the difference between number of classes of downstream tasks related to the proportion of entropy-dominancy samples in that dataset.
  • Intuitively, as the dimensionality of the output probability distribution becomes larger, the entropy over that distribution becomes larger in general because there are more possible states and outcomes.
  • Therefore, models trained on such datasets (SUN397 and Cars have 397 and 196 classes, respectively) become less confident about their prediction (higher entropy), so the proportions of entropy-dominancy samples become smaller compared to other datasets confined to a small number of classes (10 ~ 47 for remaining six datasets).
• Suggestion: As long as we adopt the entropy as an expertise metric for individual models, one should be cautious to use DaWin in the scenario where individual models are trained on datasets those the numbers of classes are very different. We recommend using DaWin in the setting where the numbers of classes are well-matched to each trained model, and if not, other quantities such as pseudo label-based X-entropy loss can be adopted as an alternative model expertise metric.
• The relationships among datasets will be further explored in the following paragraph.

W3) The dynamic interpolation adjustment can be viewed as an incremental improvement over existing interpolation methods, rather than a novel breakthrough.

Answer1: We respectfully refute novelty issues from two perspectives: the design of the method and actual performance improvement.

• In terms of design of method
  • It is worth noting that DaWin is a pioneering training-free dynamic interpolation method which is greatly underexplored so far.
  • As mentioned in the third paragraph of the introduction of the manuscript, existing methods have required additional training and extra modules to dynamically determine the interpolation coefficient. However, DaWin does not require any training while also addressing the computation complexity issue.
  • To summarize, we provided a scalable solution that addresses some intrinsic challenges of dynamic interpolation: (a) training-free: computing proper per-sample interpolation coefficients without training by just relying on the forward evaluation of each model; (b) efficient dynamic interpolation: reduced computational complexity through Beta Mixture.
• In terms of performance improvement
  • We would like to emphasize that WiSE-FT's performance on the ID-OOD performance in Figure 4 of the manuscript can not be achievable given the fact that we cannot select the interpolation coefficient based on OOD performance because we do not have test set labels in advance. Therefore, we get the performances of WiSE-FT by tuning its interpolation parameter on ID validation set in practice.
  • Given that fact, in terms of average OOD accuracy of CLIP VIT-B/32, B/16, L/14 experiments, our method achieves significantly better performance compared with existing methods including WiSE-FT and the state-of-the-art method, Model Stock [18].

Answer2: Determining appropriate per-sample interpolation coefficients is a challenging non-trivial problem, which we address in this work for the first time.

• In the table below, we provide some dynamic interpolation coefficient designs from two categories (data-independent and data-dependent).

• The results show that DaWin's entropy ratio-based coefficients greatly outperform other reasonable alternatives in terms of ID and OOD performance trade-offs. That is, some alternative methods can achieve good OOD performance but they remarkably hurt ID Acc compared to DaWin.
• This demonstrates the effectiveness of our unique design choice which is well-grounded with our unique motivation from the pilot study in Section 3 of our manuscript.

Dear Reviewer Rb5n, The authors sincerely appreciate your valuable comments that extensively cover the in-depth discussion on the technical details, algorithmic limitations, further analysis, and explanation of the results!! We strongly believe that all of this constructive feedback greatly contributes to improving the quality of this work. We also thank for the acknowledgement from you on the strengths of this paper: training-free, Beta mixture based efficiency-seeking design, outstanding performance of diverse experiments, and hyperparameter tuning-free nature! We address your concerns and questions in detail through five threads below.

---

W1/W2/Q5) There are no reported MACs or FLOPs per sample, limiting insights into computational costs. Since the model performs per-sample interpolation, it requires three inferences per sample, increasing computational demands. For the evaluation of wall-clock time, is it tested per dataset? or per sample?

Answer:

• We additionally measure FLOPs per sample for each method (CLIP ViT-B/32 robust fine-tuning setup).
• The wall-clock time in the manuscript and the table below is aggregated for all evaluation datasets.
• Meanwhile, in terms of computational demands, we can further lessen the computational complexity during inference by performing expertise estimations (forward evaluations) of individual models in parallel given multiple GPUs (two GPUs for this experiment). Note that this parallelism is orthogonal to the commonly used distributed data-parallel (DDP) inference method and is only applicable to DaWin. The results of applying those parallelizations are shown in the table below.

• Although the FLOPs of DaWin are three times larger than WiSE-FT they can be reduced by adopting the parallelization technique.
• It is worth noting that our algorithmic parallelism induces 134% faster inference compared to DDP which makes an improvement of 126% than that of vanilla inference.
• Moreover, in terms of total wall-clock time which measures the required time for all evaluations (including hyperparameter tuning of WiSE-FT), it does not increase by three or two times, unlike FLOPs, because DaWin does not require hyperparameter tuning.

Dear Reviewer sGAj, The authors greatly appreciate your invaluable comment, such as the optimality of the oracle coefficients, from a deep understanding of our work. Your productive comments and questions help the authors to further analyze and understand the strengths and weaknesses of the proposed method and the paper itself. We also are thankful for your awareness of the strengths of this work: clear and understandable writing, supporting empirical results, and addressing the computational complexity issue! We provide our responses to your query in the three threads as below.

---

W1) ... while this would make sense if we merge the output of the models (analogous to classifier selection), I cannot imagine that it is the seeked best oracle for if we perform weight averaging. This issue is about presentation and could be fixed.

Answer: As the Reviewer sGAj pointed out, $\lambda^{*}(x)$ is not guaranteed to be optimal for the weight interpolation scenario. We did not use the term 'oracle' to indicate 'optimal', and rather use it to denote the setting where we use the ground truth test labels. We revised the manuscript to avoid potential misunderstanding from readers (Please See the footnote of Sec 2.2 and Table C in the Appendix of our revised PDF).

• The rigorous 'optimality' of interpolation coefficients can be only discussed by manually searching those coefficients that induce the interpolated models to produce minimum loss per sample.
• We conduct an empirical validation investigating how close our X-entropy ratio-based oracle coefficients are to the optimal coefficients that minimize the loss. Specifically, we computed optimal coefficients that minimize the X-entropy of the interpolated model by grid-searching over the test set. Then, we compared those optimal coefficients with our X-entropy ratio-based coefficients to measure the differences between them.
• In the table below, we compared the mean squared error (MSE) between optimal coefficients and 1) Uniform(0,1) random variables, 2) WiSE-FT's single coefficient (0.8), and 3) ours.

• As we can see, compared with baseline (uniform randon varibles, a tuned WiSE-FT constant), our X-entropy ratio-based per-sample coefficients shows less MSE overall.
• Given that the standard deviation of oracle coefficients are about from 0.28 to 0.34, the closeness between optimal and X-entropy ratio-based coefficients is remarkable.

W2) minor accuracy improvement given memory overhead

Answer: Note that we could not access the true label during inference time. Accordingly, we could not achieve the best trade-off points of WiSE-FT as like in Figure 4., and the possible alternative is just tuning the interpolation coefficient of WiSE-FT on the ID validation set we have. The results from this strategy are presented in Tables 2, 3, 4. In these cases, DaWin significantly outperforms WiSE-FT in terms of OOD accuracy. The concern related to the memory overhead will be addressed in the response below.

Q3) In the experiments, DaWin sacrifices ID performance for better OOD performance compared to WiSE-FT. Could you elaborate in which application scenario I would want to choose DaWin over WiSE-FT. I would think in practice good ID performance still wins over good OOD performance if you cannot have both.

Answer: As the Reviewer sGAj noted, in robust fine-tuning experiments, DaWin slightly underperforms WiSE-FT in terms of some ID performances although remarkably outperforms on OOD datasets.

• This is because when we use the WiSE-FT method, its interpolation coefficient is tuned on the ID validation set to maximize the ID validation accuracy by assuming we can access some labeled ID samples.
• Meanwhile, DaWin does not assume accessibility on labeled samples by default and determines the interpolation coefficients based on individual models' entropy which can be not always well-correlated with the true model's expertise on a target dataset.
• We would recommend using DaWin rather than WiSE-FT (or other static merging methods) in the following scenarios.
  1. A setup where we can not access any labeled data samples from ID that can be leveraged as a prior for model design.
  2. A setup where the model is expected to encounter OOD data only during the evaluation phase.

We provide the empirical results (classification accuracy) for the second scenario in the table below.

• In this setup, we encounter the samples from unseen (OOD) data distribution during evaluation.
• We observed that DaWin consistently outperforms WiSE-FT (multiple model extended version) in terms of unseen task generalization accuracy.

Q1) Could you please comment on the memory requirement of DaWin and how it compares to other methods, for both with and without mixture modeling?

Answer 1: Here, we provide the memory requirement of DaWin in terms of storage and inference time peak memory usage. The mixture modeling does not affect the storage and peak memory usage and only affects the number of interpolation operations during inference, so DaWin in the table below denotes whether mixture modeling is applied or not.

• Compared to static interpolation methods such as WiSE-FT, DaWin requires that all individual models should be saved in storage. Thus, the memory storage requirement of DaWin is twice timed to static merging in the robust fine-tuning setup.
• Meanwhile, the inference time peak memory allocation by DaWin is roughly three times that of WiSE-FT because it requires loading two individuals and the interpolated model on memory. The results are summarized in the table below.

• We acknowledge the memory overhead of DaWin as a weakness that should be mitigated in future work, but we believe that remarkably improved accuracy on OOD datasets overwhelms that weakness given that robustness under distribution shifts is increasingly crucial for reliable AI [20].

Answer 2: If our application scenario is a parameter-efficient fine-tuning (PEFT) regime [1,2,3] rather than the full model fine-tuning scheme, this memory overhead may vanish.

• To investigate whether DaWin can be effective in such setting, we conduct an experiment on PEFT of a language model.
• By following LoRA Hub [4] setup that adapts FLAN-T5 [10] via LoRA [3], we randomly sampled 20 LoRA weights and conducted the dynamic interpolation per downstream tasks (Big Bench Hard [5]). Here we use the negative log-likelihood (NLL) and perplexity (PPL) as model expertise measure and leverage few-shot samples (five input-output pairs used in LoRA Hub) to compute NLL and PPL. The average performance (exact matching accuracy) on 25 tasks from Big Bench Hard is in the table below.

• DaWin using PPL and NLL both show better average performance compared to LoRA Hub which requires additional training per each downstream task. In this setup, compared with the static averaging or zero-shot inference method, DaWin only requires 13% more memory storage thanks to the lightweight nature of LoRA.

Q2) Definition of $H$ and a bit of elaboration. Number of model evaluations required for DaWin.

Answer:

• As noted in the caption of Table 2 of the manuscript, $H$ denotes the size of hyperparameter grid for the methods that require hyperparameter tuning such as WiSE-FT. A common choices of the hyperparameter sweep range are $\{0.1, 0.15, ..., 0.9, 0.95\}$ or $\{0.1, 0.2, ..., 0.9\}$. Then, $H$ of former and latter cases are 19 and 9, respectively.
• As the Reviewer sGAj noted, DaWin in robust fine-tuning setup requires three forward evaluations given $x$: 1) pre-trained model, 2) fine-tuned model, and 3) post-interpolation model, respectively.
• Although this is three times larger than a static interpolation method, practically, WiSE-FT requires hyperparameter tuning on the given labeled dataset to achieve competitive performances as noted in the Tables of the manuscript. Therefore, the actual runtime of DaWin would not severely be larger than WiSE-FT as analyzed in Figure 8 of the manuscript.