InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion

Thanks for your insightful feedback. Our responses to each point are as follows:

W1 & Q1: Thank you for highlighting the importance of statistical validation. While we follow the previous works experimental design like FuseLLM and FuseChat, we acknowledge that reporting confidence intervals or statistical tests would strengthen the empirical evidence. Importantly, many of the gains we observed (e.g., +35.6 on Multistep Arithmetic and 37.06 on Causal Judgement) are substantial and consistent across datasets. It can make sure that the results are not noise.

We also make additional experiments with 4 different seed settings, which provides the variance and interval:

To demonstrate the effectiveness and generalization of our GLD, we also provide additional experiments on the small size distillation with variance and interval:

W2: That`s a good point! We agree that InfiGFusion have the problem of resolving factual conflicts between source models. However, the source re-weighting and selection strategy, which aims to resolve the factual conflicts between source models, is a complementary research direction, and our work mainly focuses on aligning semantic structure and design a better GLD loss. In fact, our framework can be naturally extended by integrating factual confidence estimation or external verifiers. We have tested some existing factual conflicts resolving strategy, but the results is not significant. We thank you for pointing out this practical limitation and will discuss this more thoroughly in the final version. Designing a better factual conflicts resolution is viewed as one of our future works in our plan. But it is not the concerns in this work. W3: We appreciate the reviewer’s insightful observation. Due to space limitations, our discussion on when structural alignment is less impactful was brief in the main text. However, Appendices F–I provide a detailed analysis. We offers a per-task performance breakdown over 84 benchmarks (BBH + MMLU), clearly showing that GLD brings the most gains in reasoning-intensive tasks like Multistep Arithmetic, Causal Judgement, while maintaining parity in fact-centric domains, such as Biology, US History in Appendix F. Appendix G includes qualitative case studies illustrating how GLD improves multi-hop and dependency-based reasoning, while not harming performance on fact-centric prompts. Appendix H reports robustness trends and failure cases, confirming that GLD is beneficial or neutral—but not detrimental—even in tasks where token identity dominates. In our updated version, we will include this more dedicated subsection discussing this, and refer readers to detailed case studies and task-specific analyses in the appendix: Our qualitative analysis (Section 4.6) further illustrates this: InfiGFusion provides nuanced causal disambiguation by modeling reasoning steps (e.g., intent → misfire → injury, or criminal motive → assassination → explosion), while SFT models tend to respond with shallow pattern-based answers. In contrast, on factual recall tasks such as International Law, Ruin Names, or US History, where precise token identity or entity retrieval dominates, GLD offers limited gains (+0–1%). However, importantly, we observe that InfiGFusion does not degrade performance on such tasks. For example, on MMLU factual domains (e.g., Biology, Psychology), InfiGFusion matches SFT within ±1.5%, indicating that our method is robust even when structural signals are not essential.

Q2: Thank you for the insightful question. The gains on Multistep Arithmetic and Causal Judgement are consistent and not isolated. As detailed in Section 4.3 and Appendix F, InfiGFusion significantly outperforms both individual sources and strong fusion baselines across multiple structure-heavy BBH tasks. Expect for Multistep Arithmetic and Causal Judgement, additional consistent gains are observed on Formal Fallacies, Boolean Expressions, and Word Sorting. It is the overall results across different scenarios that demonstrates the general significant abilities in complex reasoning.

Thank you for your careful feedback and valuable suggestions. Our responses to each point are as follows:

W1: Thanks for your suggestion! We agree that introducing the concept of cross-tokenizer fusion, as exemplified by [34] , earlier in the paper can enhance clarity and better contextualize InfiFusion’s innovations. In the revised manuscript, we incorporate a brief mention of cross-tokenizer fusion in Section 1 to highlight its relevance to model fusion and to clarify how cross-tokenizer distillation builds upon and extends this paradigm. Specifically, we will note that while [34] introduced a universal logit distillation loss for aligning models with different tokenizers, InfiGFusion advances this by capturing semantic dependencies through graph-based alignment, addressing limitations in token-level methods for complex reasoning tasks. W2: For Scale of GW Distance and Relative Error Analysis, we analyzed the scale of the GW distance in our experiments, 14B model and 130K data, using datasets like BBH and MMLU (supplement in Appendix F.1 & F.2 ) and make additional experiments with other approximation algorithm. Due to the complexity, it`s hard to directly calculate the original GW distance.

The relative error analysis is included, computed as the absolute error bound divided by the GW distance (absolute errors are calculated following the original paper [1][2]). The GW distance scale ranges from 0.0210 to 0.0614 across BBH subsets, with the absolute error bound fixed at 0.00015, resulting in relative errors of 0.244% to 0.714%. This indicates suitability, as errors remain below 1%, decreasing with larger GW distances. Empirical results show Sorting-Based’s absolute error aligns with the O(1/N) bound (n=150k and m=100k), while Sinkhorn and Entropic GW offer tighter approximations, validated by performance gains.

For the upper bound of the performance, due to this cost, we conducted ablation studies on 7B (Qwen 2.5) and sub-1B models (GPT2-120M, OPT-350m, Bloomz-560m) with LLama3 as the teacher. Results show GW consistently achieves higher or equal performance than GLD (e.g., Qwen 2.5-GPT2-120M: 17.81% vs. 17.58% average), with smaller variance (e.g., 0.15 vs. 0.31), indicating better alignment stability. The upper bound performance gain is modest (e.g., 0.23–0.31% on average), suggesting GLD’s O(NlogN) efficiency retains most benefits. Computational time for GW (2.6–4.7h) exceeds GLD (2.8–5.3h), confirming the trade-off, making GLD suitable for large-scale applications.

W3: It’s a really insightful point! We make additional analysis, for real-world concepts like “cities,” GLD’s feature-wise graph captures token co-activation patterns, enabling relational alignment across models. We analyzed a sample sentence from the Causal Judgement task: “New York and Boston are closer than Los Angeles and Seattle.” and plot the token distribution of GLD and without GLD. Figure without GLD shows the token distribution, with higher activation weights for “New York” and “Boston” due to proximity, while Figure with GLD reveals stronger edge connections between these tokens in the learned graph, reflecting semantic similarity, unlike token-level baselines, which overlook these relations. To enhance interpretability, we will add these case study and figures in Appendix F using this example, demonstrating how GW distance aligns these relations, and compare it to a baseline.

[1] Scetbon, Meyer, Gabriel Peyré, and Marco Cuturi. "Linear-time gromov wasserstein distances using low rank couplings and costs." International Conference on Machine Learning. PMLR, 2022. [2] Rioux, Gabriel, Ziv Goldfeld, and Kengo Kato. "Entropic gromov-wasserstein distances: Stability and algorithms." Journal of Machine Learning Research 25.363 (2024): 1-52.

Thank you for your patient review. Our responses to each point are as follows:

W1: We have added an additional comprehensive simulation as Appendix I in the revised manuscripts. To verify the generality of our GLD approximation, we conduct experiments in the traditional knowledge distillation setting, where a compact student model learns from a single larger teacher model. Due to the O(N4) complexity of exact GW computation, we conducted an ablation study on 7B (Qwen 2.5) and sub-1B models (GPT2-120M, OPT-350m, Bloomz-560m) with LLama3 as the teacher, comparing generally sinkhorn computied GW and our GLD.

(1) Performance: GW outperforms GLD (e.g., 17.81% vs. 17.58% average for Qwen 2.5-GPT2-120M) with lower variance (0.11–0.15 vs. 0.19–0.51), reflecting exact alignment. (2) Computational Cost: GW requires 3.3–6.7 hours vs. GLD’s 2.8–5.3 hours, due to O(N4) vs. O(NlogN)complexity. (3) Efficiency Trade-off: GLD achieves near-equivalent performance (e.g., 18.26% vs. 18.43% for Qwen 2.5-OPT-350m) with 15–30% less time, ensuring scalability. Detailed Results and Comprehensive data and comparisons will be provided in Appendix I in the final version.

W2: We agree that ideally, a fusion method should not only improve average performance but also consistently outperform all source models and leverage additional sources effectively. However, the focus of this work is to introduce and evaluate the GLD formulation itself, source re-weighting and selection strategies is another complementary research direction, we plan to do research about it in the future work. While our results already demonstrate competitive performance across diverse tasks, we acknowledge that further robustness can be achieved through source re-weighting and selection strategies that address factual conflicts and model redundancy. In fact, our GLD without this strategy is compatible with existing source weighting or selection methods [1], and we view their integration as a promising future direction.

W3: All results in Table 4, including singleton source models and InfiGFusion variants, are evaluated under the same setting of top-k = 30 to ensure a fair comparison. While the best top-k in Table 2 is tuned per method (e.g., k = 10 for best final performance), we intentionally choose k = 30 for ablation because at smaller k, the only ULD component struggles to retain sufficient semantic information, especially on complex reasoning tasks. Using k = 30 ensures that all model variants are provided with enough information to showcase the relative contribution of each component.

W4: Thank you for highlighting this concern. We acknowledge that the term “internal reasoning” may be ambiguous and will revise the manuscript to clarify our intent. Specifically, our use of “internal reasoning” refers to the relational structure among semantic dimensions in the logit space, which often reflects intermediate concept alignment or multi-step inference patterns. The proposed GW term models these structures by aligning the graph of inter-logit relationships across models via Gromov-Wasserstein distance, rather than treating logits as independent vectors. While this does not model reasoning steps explicitly, it provides a structural inductive bias that benefits reasoning-intensive tasks. We will revise the manuscript to more precisely describe this as capturing relational patterns indicative of reasoning behavior, rather than reasoning itself.

[1] Wan, Fanqi, et al. "Knowledge Fusion of Large Language Models." The Twelfth International Conference on Learning Representations.

Thank you for taking the time to review our submission.

We believe there may have been a mix-up, as your review appears to describe a paper titled INFOBLEND, which focuses on KV-cache reuse for multimodal LLMs, a topic that is entirely unrelated to our submission.

We kindly ask you to verify whether this review may have been mistakenly attached to our submission.

Thank you again for your time and understanding.