$\texttt{UQ-Merge}$: UNCERTAINTY GUIDED MULTIMODAL LARGE LANGUAGE MODEL MERGING

We sincerely appreciate your careful review and a great summary of our contributions as intuitive, useful, promising and effective. And thank you very much for your construction comments. We provide our response as follows:

Q1: Discussion of evaluation using limited test sets

A1: Thanks for your valuable suggestions. We present our results with limited datasets for UQ:

From the results above we can see that the performance between full or limited datasets for UQ is very close. These results validate our design and prove its data efficiency.

We further tested our method with limited evaluation data from [1], and the results are shown below, where our method still performs better compared to baselines.

[1] Kaichen Zhang, undefined., et al, "LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models," 2024.

Q2: Computationally cost of UQ

A2: Thank you for the question. Indeed the running time of our method will increase as the dataset and MLLM enlarge. But our method is a one-time effort and the overhead (5h for a single model on a RTX A6000 Ada) will be amortized by the deployment and the performance benefits make the effort worthwhile.

Q3: Ablation of different perturbation strategies

A3: Thank you for the great question. We tested our method with three perturbation variants (image-only, text-only, and conformal prediction [2] as the UQ). The results are below:

The results demonstrates the effectiveness of our design that utilizes both image and text perturbation to evaluate models.

[2] Vasily Kostumov, undefined., et al, "Uncertainty-Aware Evaluation for Vision-Language Models," 2024.

Dear Reviewer 8odd,

We are very grateful for your constructive comments. We have made a substantial effort to respond to your questions. As there are only two days left for the author-reviewer discussion, we sincerely hope that you can provide us feedback before the discussion phase ends, and are happy to answer any follow-up questions.

Best regards,
Authors

Dear Reviewer 8odd,

We appreciate your insightful review and dedication to the review process. Due to the approaching deadline for rebuttal, we would like to respectfully convey our willingness to address any further concerns you may have. In our previous response, we carefully studied your comments and made detailed responses summarized below:

• Further description about the settings of our rebuttal experiments
• Results on different ratios of samples in the UQ datasets.

Thank you once again for your support.

Warm regards,
Authors

Dear Reviewer 8odd,

We are very grateful for your constructive comments. We have made a substantial effort in responding to your questions and listed all the paper revisions and additional experiments in the general response. As the deadline for rebuttal is approaching, we sincerely hope you can provide us feedback before the discussion phase ends, and we are happy to answer any follow-up questions.

Best regards,
Authors

Thank you Reviewer aFmm for your comments. we provide pointwise responses below to address your questions:

Q1: Generalize our method to LLM

A1: Thank you for raising the question. We would like to point out that: Our major motivation is that MLLM merging and UQ for MLLM are research areas less explored, as mentioned by Reviewer V122 and Reviewer 8odd. So we aim to conduct pioneering efforts in this direction, addressing (1) merging candidate selection and (2) when to stop adding new models. Our design performs modality-related perturbations which also include text perturbation and can be applied to LLM merging. Our future research goal is to generalize UQ-Merge to LLM and other modalities.

Q2: Comparison with other uncertainty-based merging methods.

A2: Thank you for pointing out this related work. In [1], they provided a model merging method that utilizes gradient-based UQ. However we would like to point out that their work is orthogonal to our method as our method aims to aggregate models in a training-free manner without using any labels, which is more feasible in many real-world applications as acknowledged by you and Reviewer V122.

[1] Nico et al. Model merging by uncertainty-based gradient matching. In ICLR, 2024.

Q3: Some collected knowledges lack essential commonsense elements

A3: Thank you for the suggestion. Our method aims to aggregate knowledge from diverse domains and these models may not all focus on commonsense. In our experiments, A-OKVQA [2] is a dataset used for fine-tuning and SeedBench is used for UQ [3], and both of them are considered with commonsense in Figure 2 of [4]. The model selection from our method includes the model tuned with A-OKVQA, highlighting its recognition for the importance of commonsense. We further removed A-OKVQA tuned model and results are shown below, which demonstrates the importance of commonsense in our model selection.

[2] Chen et al. M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought. In ACL 2024.
[3] Dustin Schwenk, undefined., et al, "A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge," 2022.
[4] Bohao Li, undefined., et al, "SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension," 2023.

Q4: The necessity of visual modality in ScienceQA

A4: We respectfully disagree with the point that visual modality in ScienceQA may be needless. ScienceQA is a benchmark focusing on multimodal reasoning, which we view as important for an MLLM. While for certain questions the MLLM may only use the text and its knowledge to answer, we can easily find examples like "Which term matches the picture?" in the dataset. In our method, we apply slight perturbation to the image to preserve the visual information, and in most cases the visual information is still valid.

Q5: The omission of error metrics in Table 1

A5: Thank you for the question. We included error metrics in Table 1 and think you may refer to Table 2. In our experiments we noticed the performance is very stable across runs and baselines always merge all models and have limited randomness. We tested 3 runs under Table 2's setting and the results are below:

The results show our method still achieves better performance compared with baseline methods.

We appreciate Reviewer ck8W's feedback, especially finding our work inspiring and experiment comprehensive. We provide pointwise responses to address your questions:

Q1: The method appears time-intensive due to 1) its instance-specific nature, requiring model merging for each input; 2) uncertainty scores that rely on multiple perturbations per step; and 3) an iterative merging process.

A1: We respectfully disagree and would like to point out that our method is not instance-specific. As shown in Figure 2 and Algorithm 1, our method is 1) model-specific rather than instance-specific and 2) merges models once and applies the merged model to all the inputs. For UQ, our method is also efficient as it requires only a one-time effort to obtain the uncertainty of all candidates, which can be amortized after deployment, and has no additional inference overhead.

Q2: How to remove harmful models remains unclear.

A2: As described in Figure 2, Section 3.2, and Section 3.4, we exclude over-confident models by sorting models in descending order of uncertainty and pick the merged model with the lowest uncertainty. We have provided extensive and comprehensive quantitative definitions and results to show our performance improvements compared to existing state-of-the-art baselines.

Q3: Merged model underperforms LLaVA-1.5 on some tasks. Some results appear within the average ± standard deviation range of baseline numbers.

A3: This confusion could be addressed by clarifying our setting. Motivated by training single MLLMs from scratch can be very computation-heavy [1, 2], and direct merging may include harmful models [3], we aim to design a training-free merging method to aggregate knowledge from existing MLLMs selectively.

To create MLLMs with different expertise, we used LLaVA-1.5 architecture and trained on tasks from different domains (Section 4.1 Model Preparation). Thus, the fair baseline methods we consider in experiments are 1) single trained model, and 2) baseline merging methods. We compare our method to the best single model and direct merging model in the table below, where our method outperforms these baselines.

For the second point, we argue that this is due to the downward fluctuation of performance, and the results of 3 runs for random model selections in Table 8 and Table 9 show that our method is consistently better compared to baselines.

[1] "Meta-Llama/Llama-3.2-11B-Vision-Instruct ⋅ Hugging Face", 2024.
[2] Tong, S., et al, "Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs," 2024.
[3] Zhao, X., et al, “Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild,” 2024.

Q4: Method applicability to well-calibrated models

A4: Thanks for raising this question. No, our method doesn't assume models are poorly calibrated and can still perform well when models are well-calibrated.

We tested the expected calibration error (ECE) of all models to merge before and after calibration [4], and bold means models excluded by our method. As shown in the results above: 1) the ECE values of models to merge vary in big range, showing that we are not only considering poorly calibrated models. 2) Our method excludes both a high ECE model and low ECE models, suggesting no inclination the calibration situation. 3) After calibration, our method selects the same group of models and shows close performance before calibration in the second table.

[4] Chuan Guo, et al, "On Calibration of Modern Neural Networks," 2017.

Q5: What are the oracle model merging numbers for each task?

A5: We would like to respectfully point out that this is infeasible. We have 10 models for merging and the oracle number requires 1023 runs for a single task, which takes more than 1000 GPU hours and is unaffordable during the rebuttal session.

Dear Reviewer ck8W,

Thank you for all your efforts in reviewing our work. We have carefully addressed all your questions in detail. Please check our response and let us know if you have any further questions.

Best regards,
Authors

Thank you for the response! For point 1, I was referring to the fact that you need to compute $UQ$ for each instance. For further clarification, how is $\bar{UQ}$ computed? For Line 284-293, do you mean you average $UQ$ over all the test instances of these benchmarks and then merge them based on the score?

For point 2, I understood how you filtered the uncertainty models, my point was not "how to remove harmful models remains unclear", but that you need to support the claim that there are indeed "over-confident, potentially harmful models" and your method does target removing them. In other words, you need to explain why getting a merged model with low uncertainty would remove "over-confident, potentially harmful models". If all the models are over-confident and wrong, shouldn't their merged model have low uncertainty and "harmful" behaviors regardless of how you merge them?

For point 3, I understood your setting, but my point was that a baseline with weak performance (e.g. ~70 VQAv2 score) is unreliable.

we average the model’s uncertainty on all the samples from all the datasets

do these samples include the test instances?

Your extreme example indeed challenges our algorithm

My point was exactly that there are cases where your algorithm wouldn't work as you claimed, thus it felt unconvincing and not technically sound to me.

My previous question "Does your method assume that some models are poorly calibrated" was exactly trying to figure out what assumptions you need to make for your methods to work.

baseline performance (70.21) already surpasses the largest BLIP-2 (65.2)

The largest BLIP-2 achieves 65.2 VQAv2 score in their zero-shot setting and ~82 VQAv2 score after finetuning. Your training dataset includes VQAv2 and should not be compared with their zero-shot number.

We have 10 models for merging and the oracle number requires 1023 runs for a single task,

You can do a greedy search as you did for your algorithm.

Dear Reviewer ck8W,

We appreciate your insightful review and dedication to the review process. Due to the approaching deadline for rebuttal, we would like to respectfully convey our willingness to address any further concerns you may have. In our previous response, we carefully studied your comments and made detailed responses summarized below:

• We used test instances but labels are not used.
• We have no assumption of calibration. And in real-world applications where not all models are wrongly trained, our method could exclude harmful models.
• Close performance with LLaVA-v1.5 on datasets not used for training.
• We don’t rely greedy search in our algorithm.

Thank you once again for your support.

Warm regards,
Authors

Dear Reviewer ck8W,

We are very grateful for your constructive comments. We have made a substantial effort in responding to your questions and listed all the paper revisions and additional experiments in the general response. As the deadline for rebuttal is approaching, we sincerely hope you can provide us feedback before the discussion phase ends, and we are happy to answer any follow-up questions.

Best regards,
Authors

Dear Reviewer ck8W,

We sincerely thank you for your valuable comments and time invested in our work. We have revised our paper and added relevant discussions and experiments. At present, all your concerns are responded to in the rebuttal and revised version of the paper. However, as there is only 1 day remaining, we kindly request your feedback to confirm that our response and revision effectively address your concerns. If there are any remaining issues, we would greatly appreciate the opportunity to address them to ensure the quality of our work. We sincerely hope that you find our response convincing. Please consider revisiting your rating.

Best regards,
Authors

Thank you Reviewer V122 for acknowledging our work as novel and well-organized. To address Reviewer V122’s questions, we provide pointwise responses below:

Q1 Missing baseline of single model trained on combined dataset

A1: In this study, our goal is to develop a training-free method for aggregating knowledge from pre-trained models created by other users, as outlined in our Abstract and Introduction. Training on combined datasets can be prohibitively expensive [1, 2], and current model merging methods often include over-confident models, as shown in Figure 1 and Table 2. To address this, we propose a training-free UQ-based approach that efficiently aggregates knowledge while excluding harmful models.

[1] "Meta-Llama/Llama-3.2-11B-Vision-Instruct ⋅ Hugging Face", 2024.
[2] Tong, S., et al, "Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs," 2024.

Q2 Experiment on more model families and applicability of UQ to other models.

A2: Thank you for the insightful comment. We fine-tuned InternVL-2-1B using the same datasets described in Section 4.1 (Model Preparation) and applied our method to the fine-tuned models. The results are shown below:

These results demonstrate that our perturbation-based UQ can generalize to other model family, both shown in the result above and in Section 3.3 that input perturbation and model perturbation don't rely on specific architectures.

Q3: Exploring merging different model architectures.

A3: Thanks for raising a good question. In the MLLM domain, model merging is still under-explored and a novel area, as acknowledged by Reviewer 8odd and Reviewer aFmm. Our study addresses the two fundamental challenges in model merging: 1) candidate selection and 2) pipeline optimization (including merging method and process). While merging different architectures presents an intriguing area for future research, current merging techniques primarily operate on models with identical architectures [3]. We plan to explore MoE-based approaches for merging different architectures in future research [4, 5].

[3] Goddard, C., et al. “Arcee’s MergeKit: A Toolkit for Merging Large Language Models,” 2024.
[4] Ding, N., et al, "Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models," 2024.
[5] Sukhbaatar, S., et al, "Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM," 2024.

Dear Reviewer V122,

We appreciate your efforts in reviewing our paper. We have tried our best to address all your questions in detail. Please check our response and let us know if you have further questions. Once again, thank you very much for your time, help, and consideration.

Best regards,
Authors

Dear Reviewer V122,

We appreciate your insightful review and dedication to the review process. Due to the approaching deadline for rebuttal, we would like to respectfully convey our willingness to address any further concerns you may have. In our previous response, we carefully studied your comments and made detailed responses summarized below:

• Explanation of model merging and examples that combining datasets and train is infeasible.
• Comparison with a baseline trained on a combined dataset.

Thank you once again for your support.

Warm regards,
Authors

Dear Reviewer V122,

We are very grateful for your constructive comments. We have made a substantial effort in responding to your questions and listed all the paper revisions and additional experiments in the general response. As the deadline for rebuttal is approaching, we sincerely hope you can provide us feedback before the discussion phase ends, and we are happy to answer any follow-up questions.

Best regards,
Authors

Dear Reviewer V122,

We sincerely thank you for your valuable comments and time invested in our work. We have revised our paper and added relevant discussions and experiments. At present, all your concerns are responded to in the rebuttal and revised version of the paper. However, as there is only 1 day remaining, we kindly request your feedback to confirm that our response and revision effectively address your concerns. If there are any remaining issues, we would greatly appreciate the opportunity to address them to ensure the quality of our work. We sincerely hope that you find our response convincing. Please consider revisiting your rating.

Best regards,
Authors