MERGE$^3$: Efficient Evolutionary Merging on Consumer-grade GPUs

Thank you for your detailed and constructive review. We appreciate your insights and will do our best to address your concerns within the space constraints of the rebuttal.

Methods and evaluation criteria

We report Spearman rank correlations (higher is better) using the Figure 3 setup. Due to space limits, we show averages across datasets and sample sizes (10–100); full breakdowns will appear in the paper. Results show that GMP-IRT consistently outperforms GP-IRT in ranking accuracy.

Experimental design and analyses

Ada-Merging and EMR Baselines: Thank you for highlighting these baselines—we agree they are important. Our current setup relies on MergeKit, which doesn’t yet support them. Implementing and optimizing them (e.g., for quantization) is beyond the rebuttal scope, but we plan to add support and include them in the camera-ready or follow-up.

Weaknesses

W1 — scaling number of models: Our approach scales well with the number of endpoint models because its computational cost depends on the population size (25 in our experiments) and the number of evolutionary iterations—both independent of how many endpoints are merged. We do note that the search space grows linearly with the number of endpoints, so more sophisticated initialization or mutation strategies might be needed to ensure efficient convergence.

W2 — novelty of the estimators: While inspired by tinyBenchmarks, our estimators (mpIRT and gmPIRT) are tailored for merging. By leveraging the known abilities of endpoint models and assuming linearity, we obtain more efficient and more accurate ability estimation—crucial for guiding evolutionary search.

W3 — Low-Resource Settings & Initialization: MERGE³ uses the same endpoint models as standard merging methods. The initial population is generated by sampling interpolation coefficients—not from additional models. This makes our method equally applicable in low-resource settings, and we’ll clarify this distinction in the paper.

Comments and questions

C1: Algorithm 1 explanation and position: We agree Algorithm 1 is central and will move it to the main paper in the camera-ready using the extra page. We'll also add a brief explanation to clarify its key steps.

Q1 — In-Language Merging:

While our primary focus was cross-lingual transfer, we also investigated merging models within a single language—Italian in this case. We merged a math-in-Italian model (MetaMath-Mistral-7B + Mistral-Ita-7B) with a code model with Italian capabilities (CodeNinja-1.0-OpenChat-7B), then evaluated code generation performance on the MBPP dataset (zero-shot pass@1) and math accuracy in Italian. As shown below, the merged model not only achieves higher code accuracy but also preserves math performance:

These findings demonstrate that MERGE³ can integrate multiple task-specific abilities within the same language, and we plan to include this experiment in the appendix of the revised paper.

Q2 — Baselines failure: We appreciate the reference to [2], which highlights representation misalignment as a major factor in poor performance for standard merging methods. Our baselines typically do not address this issue, so a technique like representation surgery could help reduce bias. While our work focuses on efficiently merging models via IRT-based estimators, we view representational alignment as a promising, complementary direction. We will cite [2] in the final version and consider leveraging such approaches to further enhance MERGE³.

Q3 — Initialization and "Ad-hoc Genetic Algorithm": We randomly initialize the population by creating interpolations of the same endpoint models used by our baselines—no additional models are needed. By “ad-hoc genetic algorithm,” we mean standard evolutionary operators tailored for merging. Specifically, we use Simulated Binary Crossover to recombine parents and Polynomial Mutation to introduce small perturbations. We will revise Algorithm 1 and the main text to make these steps clearer.

We thank you again for your valuable feedback. We remain available for any further questions or clarifications.

Thank you for your thorough evaluation of our work and for highlighting areas in need of clarification. We appreciate the opportunity to provide more detail on our key contributions relative to [1], our assumptions regarding the availability of a validation set, and our comparisons with in-context learning (ICL). Below, we address each of your points in turn.

Weaknesses

W1 — difference with tinyBenchmarks: We thank the reviewer for pointing this out and agree that it is essential to clarify the difference from [1] (“tinyBenchmarks”).

While [1] provides two estimators (pIRT and gpIRT) for efficient evaluation of large language models, our work proposes two new estimators—mpIRT and gmPIRT—specifically tailored for model merging. We demonstrate that plugging these new estimators into the evolutionary merging pipeline yields a fifty-fold reduction in computational costs, with negligible loss in accuracy.

Concretely, mpIRT and gmPIRT incorporate the additional prior we have in the merging setting, namely access to all endpoint models’ abilities. This lets us approximate the merged model’s ability by a linear combination of the endpoints’, rather than fitting a full IRT vector from scratch. Doing so drastically reduces the data and compute needed per iteration of evolutionary search, which is the main bottleneck. By contrast, directly applying [1] to merging without leveraging this extra prior would require re-fitting a merged model’s IRT parameters each time, significantly slowing down evolution. Thus, although we take inspiration from [1]’s general approach to efficient performance estimation, our paper addresses new challenges in model merging, such as evolving interpolation coefficients across multiple tasks/languages and making performance estimators merging-aware.

W2 — Need for a validation set: We follow the evolutionary merging setting introduced in [2], which assumes a validation (or evaluation) set is available to measure candidate models’ fitness. This convention is also common in other merging works, such as those that use a held-out set to tune the scaling factor [3]. That said, we agree that exploring fully unsupervised proxies—e.g., perplexity or entropy on unlabeled data—would be an exciting direction, as it would relax the requirement for supervised validation data in merging. We plan to investigate such approaches in future work.

[1] Polo, Felipe Maia, et al. "tinyBenchmarks: evaluating LLMs with fewer examples." International Conference on Machine Learning. PMLR, 2024.

[2] Akiba, Takuya, et al. "Evolutionary optimization of model merging recipes." Nature Machine Intelligence (2025): 1-10.

[3] Ilharco, Gabriel, et al. "Editing models with task arithmetic." The Eleventh International Conference on Learning Representations.

W3 — Comparison with In-Context Learning (ICL) We appreciate the reviewer’s suggestion. To address it, we ran the proposed few-shot in-context learning approach on our multilingual experiments, providing 20 samples as context at inference time for two baselines (TIES-DARE and Task Arithmetic). As shown in the table below, Merge³ significantly outperforms these few-shot baselines on each language:

Furthermore, using ICL makes the context significantly longer and increases memory requirements, whereas our merged model has no additional overhead at inference. Once we merge the models offline, the resulting single network can be deployed with the same resource footprint as a standard model of that size. Thus, while few-shot ICL can help in certain scenarios, model merging provides a more permanent and resource-efficient solution.

We are grateful for the time and attention you invested in reviewing our paper. Your feedback has been very helpful, and we believe that the clarifications provided address your concerns. We look forward to further refining our work in response to your comments.

We thank you for your thoughtful and detailed review. We are glad you find our framework coherent, our theoretical underpinnings solid, and our empirical results convincing. Below, we address your specific points and questions.

Limited analysis of hyperparameters: The main hyperparameter in our pipeline is the interpolation coefficient c (Equation 5), for which we followed the default from TinyBenchmarks [1] without tuning.

Following the reviewer’s suggestion, we ran a hyperparameter sweep over c. The results (included in the revised paper) show that while our original methods—MP-IRT and GMP-IRT—already outperformed competitors, the tuned GMP-IRT* further improves performance

For the evolutionary run, we used 175 individuals, split into 25 subpopulations over 7 iterations—chosen to ensure all experiments completed in under 24 hours. While not extensively tuned, this setup balanced runtime and performance well.

Ablation studies: Because evolutionary search is inherently noisy and computationally expensive to run many times end-to-end, the most feasible way to measure each module’s impact is to isolate it and evaluate it independently. In Figure 3, we focus on performance estimators, while Figure 4 examines ability estimators. Additional ablation results are included in Appendix C.2 and C.3. We agree these analyses are crucial, and we will add a concise summary of them in the main text to more clearly highlight each component’s contribution.

Benchmarks limited to one GPU. We will include multi-GPU benchmarks in the revised paper to better illustrate MERGE³’s accessibility. Below is an example using Mistral-7B on GSM8K-RO (10 examples, 4-bit models, SLERP merging):

These times show MERGE³ is practical even on older GPUs. We will add more results in the final version, including how runtimes scale with GPU class and batch size.

Weaknesses

W1 — negative transfer: In our updated experiments, we study negative transfer in cross-lingual merging using the DE, NL, RO GSM8K dataset and compare the Negative Transfer Rate (NTR) of MERGE3 against SLERP, TIES, TA. Negative transfer is observed when the merged model fails to answer a question correctly despite at least one base model having answered it correctly. Namely, NTR = (# of negatively transferred questions) / (# of questions at least one base model got right)

We see that MERGE3 substantially reduces negative transfer compared to standard interpolation methods. We will include full details and derivations in the appendix for transparency.

W2 — additional data reduction strategies: We agree that exploring non-random sampling strategies is an intriguing direction, and we experimented with two additional methods—an IRT-based clustering approach (as in [1]) and a “Representation Clustering” technique that uses concatenated embeddings from our endpoint models and applies PCA plus k-means. In both cases, we observed no clear performance benefits relative to simpler random sampling, especially after considering the added complexity and compute overhead. Thus, we ultimately opted for random sampling, but we will include these findings (presently in Appendix C.1) more prominently in the final version, highlighting why more sophisticated methods did not yield sufficient gains to justify their complexity.

Questions

Q1 — extending to more than two endpoints: Our evolutionary framework naturally extends to merging more than two endpoint models. The only change is an increase in the dimensionality of the search space, as we optimize more interpolation coefficients. The rest of the pipeline remains unchanged. We maintain a fixed population size (e.g., 25) and apply standard evolutionary operations. We demonstrated this in practice by merging three models for Jap math and four models across IT, EN, NL, and DE in the multilingual setting—both without any changes to the architecture or training procedure.

Thank you again for your thoughtful feedback. We're happy to provide further clarification if needed.

Thank you for the detailed and thoughtful review. Your feedback helped us identify key areas for clarification. Below, we respond to each point.

Claims And Evidence

• In the revision, we’ll replace “while preserving performance” with: 50× compute reduction with ~86% accuracy retained (e.g., Japanese GSM8K). The 50× figure reflects general FLOP savings; the 86% retention is task-specific and may vary. We’ll clarify this distinction to avoid overstatement.
• We will clarify the GPU used for the experiments in the main manuscript, and specify minimum VRAM requirements. We used an RTX 4090, with Batch Size 8, Quantization 4bit, Model size 7B.

Methods And Evaluation Criteria

• Please refer to W1.

Weaknesses

• W1 — true wall clock time: We agree that FLOPs don’t fully capture runtime efficiency. We’ll include wall-clock breakdowns of Estimate and Evolve steps, as well as end-to-end timing per MERGE³ iteration. To clarify: both MERGE³ and EvoMerge load the same models, so load time is similar; the gain comes from evaluating fewer examples, which significantly reduces total runtime. We’ll support this with empirical data.
• W2 — clarity issues: We now use i for data examples and j for models to remove ambiguity. “Endpoint model” is defined at first use, and MP-IRT/GMP-IRT are spelled out with brief explanations. Assumption 1 does not imply a latent space of dimension |D|, but that a merged model’s ability is a linear combination of endpoint abilities. We’ll clarify its role further in the revised paper.
• W3 — finetuning baseline: Unfortunately, full fine-tuning of a 7B model on ARC isn’t feasible within the rebuttal. We also note that this experiment has substantially different requirements from our budget-friendly MERGE3 framework, which is specifically designed for consumer-grade hardware and operates using only a small number of datapoints (20). That said, we agree this comparison would be valuable and plan to include it in the final version.

Comments

C1 — changing title: we’ve updated the abstract to explicitly mention that our method targets language models, which we agree improves clarity. Regarding the title, we’ll aim to revise it to include “evolutionary model merging” for the camera-ready version, subject to the venue’s guidelines on title changes.

C2 — emphasizing fitness as a key bottleneck: We revised the introduction to highlight that fitness evaluation is the main bottleneck in evolutionary merging.

C3 — GPU specification: Please see the “claims and evidence” section.

C4 — wall-time: Please see the answer to W1.

C5 and C6 — Clarifying notation: Thanks for pointing out this lack of clarity. Please refer to the answer to W2.

C7 — Aligning axes in figures: we will align the axes and scales across grouped figures so that all plots use consistent increments in the revised version of the paper.

Questions

Q1 — Merging in Data Flow Space (DFS): We tried DFS merging as in Akiba et al., but found no consistent gains. Their Table 1 shows DFS often underperforms parameter space (PS) merging, despite adding 3B parameters—nearly half the base model. Given our focus on efficiency for consumer GPUs, this overhead was impractical, so we prioritized size-preserving strategies.

Q2 — Estimated total time: It is a measure based on up to 12 hours runs for each method on a single NVIDIA 4090. This is an end to end measure of the entire pipeline: including model loading, estimation of abilities, and evolution of models.

Q3 — understanding of Estimate vs Evolve: Yes, that’s correct. Estimate is run once to compute endpoint abilities using the full dataset. Evolve then runs iteratively, using only the reduced dataset and our estimators to evaluate new merged models efficiently.

Q4 — scaling of compute requirements: We agree this is an important consideration and will include a discussion in the paper. In MERGE³, Estimate is run once per endpoint model on the full dataset, while Evolve runs repeatedly on a reduced subset. In practice, Evolve dominates compute because:

1. Correctness labels are often public (e.g., Open LLM leaderboard), making Estimate nearly free.
2. Estimate is one-time, whereas Evolve runs over many generations and a full population.

When correctness must be computed manually, a rough rule of thumb is: if M × N > P × K (where M = endpoint models, N = full dataset size, P = population, K = reduced subset), then Estimate might dominate. Otherwise, Evolve is the primary cost.

Q5 — dimensionality of the latent ability vectors: The dimensionality is a hyperparameter; we set it to 16, following TinyBenchmarks [1].

Q6 — see W1

We thank you again for your valuable feedback. We remain available for any further questions or clarifications.