Thank you very much for your review. We have corrected the typos you highlighted as weaknesses. In the following section, I will address the concerns you raised. If there is other questions, please let us know.

About notable correlation in line 297-299: Figure 4 and the phrase "notable correlation" are intended to emphasize that model kinship follows a pattern similar to average task performance, reflecting the progression of model evolution. The goal is to introduce a more detailed analysis of model kinship in relation to merging progress, as indicated by average task performance in the next section. Since "notable correlation" conveys the same idea as "follows similar patterns," we have removed it for clarity and conciseness, thus reducing potential confusion.

About 3 learning stages in line 311: In this section, the three model groups consist of models from two learning stages, consistent with the previous presentation, along with the initial fine-tuned models (the foundation models before the merging process). The inclusion of the initial models is intended to provide a more comprehensive view of the entire evolution process. We have revised this section for improved clarity.

About Algorithm 1:

• Line 2: It should be M_k, indicating the set of k models in the top-k set.
• Line 3: It should be 'top-k'.
• Line 7: This refers to each model pair from the top-k set, which includes the top-k models but should not include k itself.
• Line 9: We acknowledge that this state is somewhat confusing. The intended meaning is that the current best model is merged with the model having the lowest kinship with the best model from the previous generation. The key concept is to incorporate a model that may not currently perform outstandingly (and could be overlooked by a greedy strategy), but has the lowest kinship with the best model (potentially indicating better performance after merging).

Furthermore, we have updated the notation for enhanced clarity in the revised version of our paper.

About Figure Reference: Indeed, this refers to Figure 6, and we have made the necessary correction in the revised version.

About Experimental Breadth: We acknowledge that our experiments are limited in terms of architecture and task sets. We include three additional experiments:

1. Evolution experiments on the Llama-2 model
2. Ablation experiments for the greedy strategy
3. Evolution experiments on alternative task sets

Further details will be provided in the common reply section.

Thank you for your thoughtful review. We understand your concerns regarding the generalization of model kinship across datasets and architectures. We are currently conducting additional experiments to validate the generalization of model kinship in various scenarios. Most information will be included in the Common reply section.

Reason for Selecting the Current Task Set. The datasets used in our experiments are task-specific, chosen to better represent the strengths of foundation models. We selected these three datasets to maintain consistency with the task set employed in community experiments, which currently serve as the primary application environment for iterative merging techniques.

More specifically, our task set ({Winogrande, GSM8K, TruthfulQA}) is a subset of the six key benchmarks ({ARC, HellaSwag, MMLU, TruthfulQA, Winogrande, GSM8K}) featured on the Open LLM Leaderboard. These benchmarks were selected for their capacity to assess diverse reasoning abilities and general knowledge across a wide range of fields, making them applicable in various scenarios.

However, we acknowledge that our experiments are limited in task sets. At present, we are working on the addtional experiments including other task sets. Theoretically, as model kinship is rooted in the concept of task vectors, we hypothesize that it is applicable to tasks where task vectors are feasible. Additional evidence will be provided in the common reply section once the ongoing experiments are completed. We will reach out to you for this question at that time.

Generalization on Llama. We provide additional evidence in the Common Reply section regarding the Llama-2 experiment, conducted using the same task set and method as the main experiment. Further details are available in the Common Reply section.

Ablation study of greedy merging. The motivation behind Greedy Merging is to simulate the intuitive decision-making process of typical users, and it is generally regarded as the baseline strategy. For the ablation study, we implement a random selection approach. Further details are provided in the Common Reply section.

Thank you very much for your review. We notice that the correlation between biological evolution an model merging have not been fully discussed in the current version of our paper and some presentation is too short to cause confusion. We have added a new section (Appendix.E) in revised version biological concepts comparison.

Artificial Selection and Model Evolution. For biological references, we draw an analogy between artificial selection in biological evolution and model evolution through model merging. In artificial selection, desired traits are preserved by manually selecting breeding pairs in each generation, typically those exhibiting the most significant features. Similarly, model evolution (iterative model merging) follows a comparable process. Users preserve desired task capabilities by manually selecting merging pairs. Through iterative merging, they can develop a model that excels across all tasks in the desired task set.

Inbreeding Depression and Saturation Stage. One of our key findings is that the late stage of model evolution often enters a saturation stage, where models exhibit significantly low differences among themselves. This phenomenon is analogous to "inbreeding depression" in artificial selection, where breeding closely related individuals leads to a reduced gene pool and diminished fitness. Similarly, merging closely related models results in new models with minimal variation, thereby diminishing the benefits of merging, particularly with weight averaging method. To address this, we propose measuring the differences between models, termed model kinship, to mitigate this issue during model evolution.

Concern about generalization in task sets: We acknowledge that our experiments are limited in terms of architecture and task sets. At present, we are working on the addtional experiments including other task sets. Theoretically, as model kinship is rooted in the concept of task vectors, we hypothesize that it is applicable to tasks where task vectors are feasible. Additional evidence will be provided in the common reply section once the ongoing experiments are completed. We will reach out to you for this question at that time. If there is other questions, please let us know.

Thank you very much for your review. Based on your feedback regarding the first two questions, we have restructured Section 2 (Background) to improve clarity and presentation. New contents are highlighted in the blue color. If there is other questions, please let us know.

Problem Definition and Research Motivation. We focus primarily on the iterative merging process (model evolution). To the best of our knowledge, this technique was first adopted by the Hugging Face merging community with Mistral-7B [1]. The community iteratively refines models through repeated applications of model merging, resulting in the creation of several powerful models, such as YamShadowExperiment28-7B.

The table below demonstrates the effectiveness of iterative merging compared to the single-merge method. The results indicate that iterative merging can more effectively acquire generalized models, achieving higher overall performance.

However, iterative merging lacks a general merging strategy, which limits its overall effectiveness. Common methods often employ greedy strategies, such as merging models with the highest average performance. While this approach is relatively effective, our observations indicate that the community frequently attempts to merge models with nearly identical weights. This practice leads to resource waste and stagnation, as no further improvements are achieved. Our work aims to develop a more effective metric to guide iterative merging, potentially avoiding self-merging and enabling further advancements.

Reason of Kinship Metric Formulation. Model kinship leverages the structure of task vectors [2], which have proven effective in representing task specific information. Our approach extends this idea to a more complex multi-task scenario. Here, the similarity between the delta parameters (task vectors) of two models, calculated using similarity metrics (three of which have been tested in our analysis), can reveal differences in overall task capabilities, thereby enhancing the quality of the merging process.

In the revised version of our paper, we have incorporated additional background to clarify the motivation for formulating model kinship.

Quantitative Analysis to validate Model Kinship. We agree that more quantitative analyses are necessary to validate the concept of model kinship and support our conclusions. Since iterative merging is a novel concept in the field of model merging, existing experiments are limited. To address this, we conducted additional experiments to evaluate the generalization of model kinship across architectures and task sets.

Details of these experiments are provided in the common section. Note that the experiments are still ongoing, and we will provide updates upon its completion.

References

[1] Huggingface Open LLM Leaderboard - v1

[2] Gabriel Ilharco et al. Editing Models with Task Arithmetic. ICLR 2023

Thank you for your effort in revising and enhancing the manuscript. I truly appreciate the improvements made. However, after a closer examination, I still have several questions and concerns that I hope the authors can address to clarify and strengthen the arguments presented in the paper:

1. On the relationship between Kinship and performance gain:

(1) Support for the qualitative conclusion in Section 3.2: In Section 3.2, the manuscript suggests a positive relationship between Kinship and Performance Gain. However, the experiments do not seem to explicitly validate this assertion. While the Kinship concept is introduced to “identify task-related differences between models to maximize the outcomes of merging,” the conclusions drawn in Section 3.2 appear insufficient to substantiate this claim. Could the authors provide additional experimental evidence or a clearer explanation to reinforce this point?

(2) Theoretical basis of Kinship in Section 2.3: The definition of Kinship as the similarity of model weight differences is stated to reflect “the similarity of task capabilities between models.” However, it is unclear why weight difference similarity necessarily correlates with task capability similarity. Could the authors elaborate on the theoretical foundation for this claim? Furthermore, are there potential counterexamples (e.g., models with similar weights but divergent task performance) that could challenge this assertion?

(3) Key statement in Section 3.2.1: The manuscript suggests that Kinship “may serve as a key factor in determining the upper limit of merge gains, thus indicating potential improvements.” While this statement is intriguing, it lacks direct experimental validation. Could the authors provide quantitative evidence demonstrating that Kinship reliably predicts the upper limit of performance gains, perhaps through controlled experiments or ablation studies?

2. On the main experimental results:

(1) Criteria for stage division in Section 3.4: In Section 3.4, the thresholds for “Saturation Stage (≥ 0.75)” and “Learning Stage (< 0.75 and ≥ 0.73)” are introduced. However, the basis for this division is not clearly explained. Are these thresholds derived from statistical analysis, theoretical insights, or empirical observations? If they are empirically determined, additional experimental justification would be helpful. If they are theoretically motivated, the rationale should be explicitly stated.

(2) Relationship between Kinship and higher average task performance: The manuscript claims that “model Kinship increases with the average task performance, even across models that follow different evolution paths yet share similar average performance levels.” However, this relationship is not sufficiently clarified. For instance, the proximity of parameter similarity near a local optimum (using the base model as the initialization path) might merely reflect optimization path adjacency rather than providing evidence for the core conclusion. Could the authors provide a more rigorous explanation or analysis to strengthen this argument?

Summary and Suggestions: To enhance the clarity and rigor of the manuscript, I suggest the following:

Provide stronger theoretical and experimental support for the definition of Kinship and its contribution to performance improvements. Elaborate on the causal relationships and logical connections in the main experimental results, especially regarding the relationship between Kinship and performance gains. Offer more detailed explanations or acknowledge potential limitations in the current analysis to improve the overall scientific rigor.

Notably, the related exploration models (Exp-2 in both Mistral-7b and Llama-2) exhibit significantly different strengths across task sets compared to other models within the same generation. We hypothesize that this distinction is a key factor contributing to the effectiveness of model kinship-based exploration.

We conduct our experiments on the Llama-2 architecture, adhering to the same settings outlined in the main experiments of our paper.
The foundation models used are fine-tuned on the TruthfulQA, Winogrande, and GSM8K datasets.

The following table presents the evaluation results. Each column represents:

• Model: The name of each model. Note that the first three entries are fine-tuned foundation models used in our experiments.
• TruthfulQA_mc2, Winogrande, GSM8K: The benchmark results for each dataset, indicating the model's task-specific capabilities.
• Average: The average score across all benchmarks, reflecting the model's overall generalization performance.
• Model Kinship: The kinship score (Here, we use cosine similarity to measure model kinship) of the parent models involved in the merge, indicating their relatedness.
• Parent-1 and Parent-2: The names of the parent models used in the merging process.

Additionally, '-greedy' indicates that the model is generated by the greedy algorithm, while '-exp' indicates that the model is generated during the exploration step.

In the Llama-2 experiments, the kinship-based method outperforms the vanilla greedy method (e.g., Child-6-Exp > Child-2-1-Greedy). The greedy method becomes trapped in a local optimum by Generation-6, whereas the kinship-based exploration identifies a new model with better overall performance.

We conduct our ablation experiment on the Mistral-7B architecture, adhering to the same settings outlined in the main experiments of our paper.

The only difference is that we use the random-merge strategy, where models in each generation are merged with random models (excluding themselves) from the repository (as illustrated in our main figure).

The following table presents the evaluation results. Each column represents:

• Model: The name of each model. Note that the first three entries are fine-tuned foundation models used in our experiments.
• TruthfulQA_mc2, Winogrande, GSM8K: The benchmark results for each dataset, indicating the model's task-specific capabilities.
• Average: The average score across all benchmarks, reflecting the model's overall generalization performance.
• Model Kinship: The kinship score (Here, we use cosine similarity to measure model kinship) of the parent models involved in the merge, indicating their relatedness.
• Parent-1 and Parent-2: The names of the parent models used in the merging process.

In the random-merge strategy, the average performance in each generation fluctuates. The highest average performance achieved is 68.55, which is lower than the 68.72 obtained in the greedy experiment.
Although the random-merge strategy avoids falling into local optima, its unstable search process may make it inefficient.