Whoever Started the interference Should End It: Guiding Data-Free Model Merging via Task Vectors

We appreciate the reviewers’ valuable feedback and have addressed each point as follows:

Concern 1: The Use of Validation Data

1. Whether task arithmetic is data-free: We summarize TA as a data-free method as it can select empirical parameters for merging without requiring data. Although it can be improved by seaching rescaling coefficient, it is reasonable to summarize TA as a data-free method.
2. How we run other methods: We run other methods in the same way as described in their paper and report the results accordingly in the paper if the experimental settings are same.
3. Whether we use validation data: Our proposed method is entirely data-free, and did not require any validation data. This is because our approach is rescale-free and solely relies on our data-free loss.

Concern 2: Focus on Multi-Task Learning

Our work focuses on model merging rather than model editing. In current model merging field, multi-task learning is the primary goal to pursue [1,2,3,4,6,7], where TA and task vector is a foundamental concept for analysis the interference and conflict[1,2,6] .

Concern 3: Related References

1. Linearity claims: Our work focued on the linear layer but not linearizing the whole model. Therefore, these linearization works are not highly relevant to us.
2. Reconstructing data：Our work used the reconstruction as an intermediate step in our derivation and did not use reconstruction data in our method. Therefore, the data reconstruction attacks works methoded are not highly relevant to us.
3. Other works: We appreciate the references you provided and will incorporate their discussion in our revision. The conclusions and methods in these works do not significantly overlap with ours. Therefore, we consider them to be "Related References Not Discussed" rather than "Essential References Not Discussed"

W1: The Claim of Local Optima and Definition of Interference Vector

1. The Claim of local optima What we want to express is that the task vectors for same task are at similar positions, which indicated that the models are fine-tuned to convergence, so we think they are optimized to local optimal. But these task vectors may not be the optimal solution in the entire parameter space, so using local optimal is a more rigorous statement. Also we have not stated that this local optimal is unique, and this has no significant connection with our method.
2. The Definition of Interference Vector: We do not claim that any changes to task vector constitute interference, we define the interference as the difference between outputs of merged model and expert model, which is composed of $\delta x$, if $\delta$ is orthogonal to $x$, then $\delta$ is not cause interference. Similar concepts are adopted in methods like Eq.7 in Alpha-Edit [5] and Eq.1 in Regmean[4]. Furthermore, the success of TA and other methods doesn‘t mean "you can get disentangled vectors that do not interfere" since the current model merging method is far from allowing model merging without reducing any performance.

Q1: Convex Hull

Not falling in convex hull is not unique to our method, as other methods do not constrain the sum of rescaling coefficients equal to 1. The reason why our method effective compared to previous data-free methods is our method implicitly leverages input data information via task vectors.

Q2: The Choice of RoBERTa and LoRA Results

We selected RoBERTa because it's a commonly used benchmark, as used in works such as [3]. We further provide results for other widely adopted LoRA benchmarks [1][2][3]. Due to character limitation, you can view the results in Tab1.1 and Tab1.2 in our rebuttal to reviewer VcLV.

Q3: The Least Similar Problem to Our Work

1. Least Similar Problem Most existing methods focus on reducing the interference and conflicts [1,2,3,4,6,7], so it's hard to say what is the least similar problem to our work.
2. Combine with other methods Since our objective is a simple optimization problem, we think different initialization will only provide limited influence. The result in Tab.3.1 demonstrated that using different methods as initialization yields similar results.

Tab 3.1 Results of using different methods as initialization

---

We thank the reviewer again for their valuable feedback and hope our detailed responses address the concerns. We look forward to further discussion.

[1] TIES-Merging: Resolving Interference When Merging Models

[2] Parameter Competition Balancing for Model Merging

[3] Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging

[4] Dataless Knowledge Fusion by Merging Weights of Language Models

[5] AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models

[6] Task Singular Vectors: Reducing Task Interference in Model Merging

[7] DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling

We appreciate the reviewers’ valuable feedback and have addressed each point as follows:

W1: Reconstruction Error

For calcuating the reconstruction error in Equation (13), we first obtain the input for each layer from a set of samples and then compute the reconstruction coefficients using the least squares method with its corresponding task vector. The reconstructed vector, $x_{\text{recon}}$, is derived using these coefficients along with the task vector. Then we calculate the Relative Reconstruction Error (RRE) for the sample from task $i$ as follows:

\text{Relative Reconstruction Error (RRE)} = \frac{\|x - x_{\text{recon}}\|}{\|x\|}, \quad \text{where} \quad x_{\text{recon}} =\tau_i\(\tau_i^T\tau_i)^{-1}\\tau_i^T x .

Table 2.1 shows the result of different layers and tasks. which demonstrate that the relative reconstruction errors across different layers are extremely small, which further validates our theoretical analysis.

Table 2.1: Reconstruction Error Results on eight tasks of different layers

---

W2: Resource Consumption Comparison

We report the computational time and GPU memory usage of different method on ViT-B-32 tasks. In comparison to the Adamerging method, our approach not only improves performance but also significantly reduces computational cost. The details are summarized in the following table:

Table 2.2: Detailed computational time and gpu memory requirements on ViT-B-32 tasks.

Q1 & Q2: How to Use Random Vectors or Subsets of the Task Vector for Optimization

For the subset of task vectors, we randomly sample a subvector from the original task vector as follows:

\tau^{\text{sub}}_i = \tau_i [\text{rand\\_index}, :]

The corresponding loss is computed as:

$\mathcal{L}_{\text{sub}} = \sum\_{i=1}^{n} \frac{1}{\|\tau_i\|\_F^2}\ \delta_i(\tau^{\text{sub}}\_i)^\top = \sum\_{i=1}^{n} \frac{1}{\|\tau_i\|_F^2}\ (\tau_m - \tau_i) (\tau^{\text{sub}}_i)^\top$

For random vectors, we sample from a Gaussian distribution whose mean and standard deviation are computed from the original task vectors:

$\tau^{\text{random}}_i \sim \mathcal{N}(\mu_i, \sigma_i^2), \quad \text{where} \quad \mu_i = \text{mean}(\tau_i) \quad \text{and} \quad \sigma_i = \text{std}(\tau_i)$

The loss in this case is given by:

$\mathcal{L}_{\text{random}} = \sum\_{i=1}^{n} \frac{1}{\|\tau_i\|_F^2}\delta_i(\tau^{\text{random}}_i)^\top = \sum\_{i=1}^{n} \frac{1}{\|\tau_i\|_F^2}(\tau_m - \tau_i)(\tau^{\text{random}}_i)^\top$

All reported results are averaged over 5 sampling runs. We thank the reviewers for their suggestions and will include these additional details in the revision.

---

We thank the reviewer again for their constructive feedback and hope that our detailed responses address the concerns. We look forward to further discussion.

We appreciate the reviewers’ valuable feedback and have addressed each point as follows:

---

W1: Results on More Models and LoRA

To further demonstrate the generalizability of our method on different models and LoRA, we supplemented the experiments on Flan-T5-base and Qwen-14B. For merging LoRA, We first restore BA back into the original matrix $(\tau_i=B_iA_i)$ , then apply WUDI-Merging directly to $\tau_i$ to obtain $\tau_m$, then merging it into $\theta_{base}$. The experimental results obtained from merging Flan-T5-base (LoRA fine-tuned) models and Qwen-14B (LoRA fine-tuned) are shown in the table below:

Tab 1.1: Experimental results of merging Flan-T5-base (LoRA fine-tuned) models on all eight tasks.

Tab 1.2: Experimental results of merging Qwen-14B (LoRA fine-tuned) models on all four tasks.

While there is a slight degradation relative to individual models, our method demonstrates SOTA performance on merging LoRA fine-tuned models.

---

W2: Potential Limitation

We think that merging heterogeneous models may be the potential Limitation, as such models might not adhere to the assumptions underlying WUDI-Merging and current merging methods. Addressing these challenges is a key direction for our future research.

---

Q1: On Using Only the Task Vector of the Linear Layers

In the context of homogeneous model merging, we think that "using only the task vector derived from the linear layers is usually enough". Removing the nonlinear task vector from the fine-tuned model can be viewed as applying a small offset to a limited number of parameters. For a $C$-Lipschitz continuous model, the change in the output is bounded by:

$||f_{\theta+\tau_i}(x) - f_{\theta+\tau_i^{linear}}(x)||\le C·||\theta+\tau_i - \theta+\tau_i^{linear}|| = C·||\tau_i^{non-linear}||$

Considering that the parameters of the nonlinear layer accounting for only a small fraction (Qwen-14B$\approx$0.007%, LLama3.1-8B$\approx$0.003%) and has a small offset. The affect brought by only using the task vector of the linear layer is small. Therefore, we think that using only the task vector derived from the linear layers is usually enough. In all our other experiments, we found that utilizing only the task vector of the linear layer achieves performance comparable to that of the full expert model, which also confirms this point.

However, if the parameter's offset are large,our method may have limitations in use, which is also a major limitation in the current field of model merging [1]. Current model merging methods need to limit the change of fine-tuned expert models to base model in a relatively small range. We will explore this limitation in our future work.

---

Q2: Results on Different Optimization Strategies

We evaluate the effects of applying different optimizers and learning rates in our method, which is shown in Tab 1.3 and Tab 1.4. Tab 1.3 shows that Adam (85.2%) and SGD (85.1%) achieve similar performance, which suggest that our method is not sensitive to kind of the optimizers. Tab 1.4 demonstrate that a smaller learning rate can better ensure the stability of optimization and enhance the performance of results.

Tab 1.3: Experimental results of applying different optimizers in our method, the average accuracy of ViT-B-32 are reported.

Tab 1.4: Experimental results of applying different learning rate in our method, the average accuracy of ViT-B-32 are reported.

---

Q3: Typo

We appreciate the reviewers' attention to detail and will correct the identified typographical errors in the final revision.

---

We thank the reviewer again for their constructive feedback and hope that our detailed responses address the concerns. We look forward to further discussion.

[1] Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch