Fusing Models with Complementary Expertise

Major concern: the fusion based method relies on a validation set (+ additional assumption that the validation set should be similar to the test set).
Response: Please kindly refer to our general response.


Concern: If there is available validation data, can we train a parameterized fuser for traditional methods? Therefore, this is not fair in a sense, or can I understand that this is a setup for a new fusion task?

Response: We kindly request that the reviewer provides further clarification on their concern, as we are not entirely clear on the issue.

When comparing our proposed FoE method to traditional methods, the key difference lies in the approach. Traditional ensembles aggregate experts' outputs through methods like simple averaging or majority voting. Thus, not like FoE, it is test sample agnostic. The MoE method, on the other hand, requires joint training of experts with the gating mechanism. In contrast, FoE allows the use of pre-developed expert models, which are then fused for enhanced performance. This is achieved by training a lightweight fuser model, which is responsible for selecting the most suitable expert for each test sample.


Concern: Using a single expert was almost as good as using all experts: the result of an illogical experimental setup? It looks like the knowledge between multiple experts is not complementary but redundant.

Response: We would like to argue that the experts are indeed complementary, especially when considering Tables 2 and 3, where each expert performs well on only one task, and there is an expert performing well on each task. The term "redundant" might refer to our MMLU experiment, as discussed in the paper. In that experiment, we explored the concept of "weak experts," where none of the LLMs are true experts in any of the MMLU categories. Consequently, it is very likely that their knowledge contains redundancy. However, our experiment shows that even in such a scenario (i.e., beyond our initial problem setting), FoE can still deliver a decent performance boost.


Concern: Some more explanations about the results in Table 3.

Response: "CNN DM Expert'' refers to the accuracy of the CNN DM expert on each sentiment analysis task, serving as a baseline to indicate individual expert performance. "CNN DM Only" assesses the fuser's performance when only the embedding feature of the CNN DM expert is used for training. This demonstrates that FoE is inherently frugal on the summarization task, as relying on a single expert's embedding is sufficient for accurate expert selection. And that explains why there is a large performance gap between those results. We modified Table 3 to make this less confusing, thanks for pointing it out!


Concern: Are "Fusion of Experts" and "Mixture of Experts" two different concepts, and what is the essential difference between them?

Response: Let's discuss this under three conditions:

First, when all experts are trained jointly with the gating network in MoE, it results in a new model architecture. Training MoE, such as a SwitchTransformer, can be computationally and resource-intensive. In contrast, FoE focuses on utilizing any pre-trained experts and fusing them, requiring only a small fuser model to be trained. Thus, the development cost of FoE can be much lower than MoE in this scenario.

Second, when all experts are pre-trained, and one aims to build a MoE using these experts with a gating network that directly processes the data (similar to what the reviewer 2UmK asked about), the gating network must be complex, such as a convolutional neural network or ViT for images, or a BERT/RoBERTa model for sequences of tokens. Compared to such scenarios, FoE requires a much simpler fuser architecture, needing nothing more complex than an MLP or kNN model.

Third, when all experts are pre-trained, and one wants to build a MoE with the gating network using experts' outputs or intermediate results, FoE is conceptually identical to MoE in this scenario, however we are unaware of prior work considering MoE in such a setting, nor extending it to the frugal setting as in our paper.

Concern: How reasonable is that assumption that one knows the correct model for each input? Do we also know "exactly" which model should be selected?

Response: Please kindly refer to our general response.


Concern: Clarifying Equations 4.1, 4.3, and 4.4.

Response: Thank you for bringing up these questions regarding our equations. We have made changes in the main text to make things clearer. Regarding Eq. 4.1, we sum over $k:f_k \in \mathcal{S}$ because, in our notation, $\mathcal{S}$ includes all experts in $\mathcal{\tilde{S}}$ (queried experts) plus some extra expert we want to query in the next step; therefore, it should not be constant when choosing a new expert. Regarding Eq. 4.3 and 4.4, we assume access to the experts’ outputs for all data points in a validation set. For example, if we have a validation set with a hundred data points and five experts, we have a total of $100\times 5 = 500$ predictions/generations in the validation set. In the first equation, we just choose the expert which had the best average performance in the validation set, while in the second we choose a new expert which is expected to reduce the loss by the biggest amount. The intensive part here is training many fuser models to combine experts in different ways. Still, that cost can be alleviated using k-nearest neighbors as explained in the paragraph “Obtaining fusers for subsets of experts” (Section 4.2).


Concern: Using $c_k$ in practice.

Response: The $c_k$’s can include different types of costs. It could include energy consumption if experts are run locally or API costs when a third party provides experts. We made this point more explicit in the text.


Concern: More explanation on why FrugalML performed so poorly, and adding error bars for the random expert baseline.

Response: FrugalML considers composing up to two experts. Thus, in our setting, its performance saturates after two experts (i.e., allowing FrugalML to query more experts is not going to improve its performance). However, when querying either one or two experts, FrugalML still outperforms the approach of selecting random experts. We updated Figure 2 to include error bars for the random expert baseline.


Concern: Though sentiment analysis is essentially a classification task, we train the fuser using the generative model strategy 3.3. I believe this is a typo and should be "using the classification model strategy".

Response: No, it’s not a typo. The key difference between our generative model strategy and the classification strategy lies in their approaches: while the generative model strategy is designed to predict "which expert to use," the classification strategy directly employs the fuser for classification tasks. In our experiment, even though sentiment analysis inherently involves prediction, we train the fuser to determine which expert should be used for the sentiment analysis task.


Concern: Notation inconsistency.

Response: Thank you for pointing this out. We have corrected this issue in the main text.

Dear authors,

Thanks for the detailed and concise answers to my questions. I greatly appreciate the effort you put in the rebuttal, specially running extra experiments that help clarifying the overall proposal.

The rebuttal provided answers my most important questions, and the experiments make the paper stronger in my opinion. Given the above, I am willing to upgrade my score.

Congratulations on your great work.

Concern: assuming data available for training the fuser. Not realistic as there is only a few labeled data. Fine-tune models if labeled data is available? Generalize the trained fuser to other tasks?

Response: Please refer to our general response to this point.

Regarding the suggestion that “If a large labeled dataset is available, it might be more efficient to fine-tune a foundational model or employ few-shot learning,” we contend that in FoE, only a very small fuser model needs to be trained, typically a small MLP or even a kNN, where no training is required. In terms of the number of parameters, the fuser models are usually orders of magnitude smaller than the expert models. This approach is computationally much cheaper than fine-tuning a foundational model, as the reviewer suggested, and also cheaper than fine-tuning any individual expert model.


Concern: Various choices of the fuser network? More details about the hyper-parameters. Why choose a certain $\kappa$ in the kNN setting?

Response: Thanks for pointing it out! A promising aspect of the FoE is its flexibility in the architecture of the fuser model; it doesn't require any specific design. In our experiments, we explored using both simple MLPs and kNNs as fusers and found that kNNs offer certain advantages, particularly in frugal settings. Additionally, we experimented with simple linear models during the development of the FoE method, although these results are not yet included in the paper. The table below compares the performance on the CIFAR-100 dataset using both MLP and linear models as fusers, each trained on the combined training set of all experts. We observed that using models as complex as MLPs are sufficiently effective, as evidenced by our results in summarization and sentiment analysis tasks where our fuser performs comparably to the oracle models. Therefore, we believe there is no need to explore more complex models like transformers, especially since the input for the fusers is not sequence data, and doing so would compromise the FoE's advantage of having an easily trainable fuser. We will include a discussion on the specific choices of fuser architecture in the revised draft.

For questions regarding experimental details, we will revise the experiment section to include more comprehensive descriptions of our setups. The details of our hyper-parameters are already specified in Appendix C. The fuser training process is so simple that there aren’t many hyper-parameters to report.

As for the different choices of $\kappa$, our approach is to select a relatively small and odd number, as is common practice in such experiments. We will update the manuscript with a plot showing results for various values of $\kappa$ shortly, as these experiments are currently in progress. For additional experimental details, we have provided the code used in our experiments, which contains all the necessary information to replicate our results.


Concern: Expanding experimental details?

Response: We will expand our experiment section in the final version.

We would like to thank the reviewer again for the detailed reviews and constructive comments on our paper. As promised, we have updated the manuscript with a plot showing results for various values of $\kappa$. Please refer to the new Figure 2 in the revised manuscript, where one can clearly observe that the performance of FoE does not appear to be sensitive to various values of $\kappa$.

Please let us know if our response has addressed your concerns. Thank you again!

Thank you so much for joining the discussion and for increasing your rating! We discuss your remaining questions below.

They mention that the fuser network is better because 1) it can be learned with a few examples, but few-shot in-context learning can also achieve this; 2) it can be trained more efficiently since it has fewer parameters, but many parameter-efficient training methods exist.

An important aspect of FoE is the ability to reuse pre-trained expert models. It is not clear how to use PEFT to fuse multiple pre-trained LLM experts. One naive approach could be to fine-tune a single expert (or some base model) on all experts’ training data using PEFT. This approach would require fine-tuning an LLM, even with PEFT this could be a fairly expensive task in comparison to training an MLP model as in FoE. For instance, in our experiments, we simply downloaded expert models already available on HuggingFace and did not need to fine-tune any LLM. Meanwhile, fine-tuning an LLM on a combination of experts' domains does not allow one to reuse existing experts and is likely to also require a larger base model to perform well on every domain simultaneously with a single LLM. We believe combining pre-trained experts is more appealing as it can take advantage of the many fine-tuned models open-source community releases.

Regarding in-context learning, the performance is typically worse than that of an expert model. Prior work has demonstrated that training a smaller expert model is typically more efficient than prompting even the largest general model (we provide some references in the first paragraph of the introduction). We also note that the “weak” experts considered in our MMLU experiment are 5-shot prompted general models (which is the standard practice for evaluating on MMLU) and our results demonstrate that it is also beneficial to fuse such ICL-prompted general LLMs.

However, it is still unclear how other complex architectures, or just MLP with more layers, could improve the task. It is also a bit weird that the authors ablate K=7,9,11 in KNN instead of more general Ks.

We are willing to explore more complex fuser model architectures and will demonstrate the results in the final paper draft. However, we have already demonstrated that FoE is not overly sensitive to the fuser’s model architecture, and even simple MLPs and linear models work sufficiently well. We will also explore more $\kappa$ values in our final paper draft, but we believe it is evident that FoE is reasonably robust to the choice of $\kappa$.

Could you explain more about what the "FoE w/ TFN Expert Features Only" is? How can you train the fuser with only one expert and apply the fuser to multiple experts?

FoE w/ TFN Expert features only implies that only the TFN Expert's features are used for training the fuser. For testing, it is also necessary to use only the TFN Expert’s features. Let’s consider a concrete example: suppose the TFN Expert’s embedding dimension is 768, and there are six experts. In this case, the fuser (e.g., an MLP) only takes a 768-dimensional input to perform a six-class classification, i.e., to determine which expert to query for a given test data sample. In FoE w/ TFN Expert features only, we train the MLP using only the TFN Expert's 768-dimensional features. During testing, we also use only the TFN Expert’s features to predict which expert to query. In contrast, in the regular FoE, the fuser takes a 6x768=4608-dimensional input for the same six-class classification task. During test time, all experts need to be queried for each test data sample to construct the 4608-dimensional input. This is precisely why we claim that FoE is naturally frugal for the sentiment analysis and summarization tasks, as using only one expert during the entire fusion process (both in training the fuser and using the trained fuser during testing) is sufficient to achieve good results.

Concern: The correct baseline for the proposed algorithm is to compare it to an equal number of parameters with the sum of all individual experts or Sparse MoE.

Response: We would like to respectfully disagree with the reviewer’s point. We view FoE as a lightweight paradigm that allows us to easily fuse pre-trained expert models to achieve better performance on a mixture of downstream tasks and domains.

It is not very clear how to directly adopt Sparse MoE in a scenario where $K$ pre-trained experts are provided with the data used to build them. One possible approach is to freeze all pre-trained experts and only train the gating network using the available datasets. In such a case, the gating network would need to directly process data, including images and sequences of tokens, requiring a complex architecture like a BERT, a ViT, or a convolutional neural network. This requirement significantly increases the computational difficulty and model complexity needed to build such a Sparse MoE.

Regarding the suggestion to use a model with the combined number of parameters of all experts, we believe this approach is not quite practical. FoE allows us to leverage the resources created by the entire open-source community, such as trained and fine-tuned expert models or LLMs, to achieve better results. For example, one can easily download ten 13B expert LLMs from Hugging Face with just a few lines of code. However, obtaining and running inference on a 130B expert model, as suggested by the reviewer, is far more challenging. In most cases, such a 130B expert LLM does not even exist, and constructing one poses even greater challenges.