Divide And Conquer: Efficiently Decoupling Consensus And Divergence For Federated Large Language Model Fine-Tuning

W1. The authors claim that this is the first study on non-iid federated LLM fine-tuning, which appears to be an overstatement. Related work [1] is not discussed or compared. Furthermore, most federated LLM fine-tuning studies have performed non-iid experiments (with different Dirichlet parameter $α$ values), even without specific non-iid designs.

W2. While evaluating divergence by comparing directions is straightforward, it is not clear why $||θ^i||_c$ is referred to as consensus and why consensus can be directly averaged.

W3. The rationale for using average cosine similarity as a weight in model averaging is not clearly explained. Suppose two clients, A and B, have identical data and model parameters, and another client, C, has low similarity to A or B. This implies that updates from clients A and B will dominate the training process due to the high similarity. It is unclear why this outcome is expected, and further justification is needed to clarify the reasoning behind this approach.

W4. The experimental results do not convincingly demonstrate the superiority of the proposed method. The proposed method performs best only on MT-1 (it is unclear if MT-final is an independent evaluation set).

W5. The experimental scope is limited, with inadequate ablation studies. It is unclear what "client" and "server" represent in Table 2. The contribution of consensus and divergence to the performance improvement is not demonstrated, and the effect of imbalance level (Dirichlet parameter) is not studied.

Minor Suggestions

M1. Lines 107, 133: "LLama" should be "LlaMA".

M2. The dataset names in Section 4.1 do not match the names in Table 1, making it unclear what each column refers to.

References

[1] Cho, Yae Jee, et al. "Heterogeneous LoRA for federated fine-tuning of on-device foundation models." International Workshop on Federated Learning in the Age of Foundation Models in Conjunction with NeurIPS 2023. 2023.

1. The notations and the relation between the notations in the paper need to be further revised and be more concise. The notations in the current version make it hard for the reader to understand the core algorithm in the paper. Section 3.2 contains many notations in addition to the $\theta$ used for LoRA, including $M$, $N$, $v$, $V$, $L_k$ and $G_k$. It is hard to understand how those notations relate to the parameter of LoRA. Although some explanation efforts can be observed in the subsection, it brings more questions, such as what the "vector" refers to or corresponds to in LoRA.
2. Some key parts of the algorithm may need to be further illustrated, including what are the parameters that local clients share and which part of the computation is done on the client/server, respectively.
3. The motivation for the Gauss-noise masking is unclear and somehow not self-contained. In the "motivation" paragraph, it mentions the parameters with fewer updates can slow down the global convergence; however, introducing noise on those parameters can not solve such a problem in common belief but can even make the convergence slower. (This can refer to differential privacy papers where Gaussian noise is usually applied.) More theoretical or empirical support or existing literature references are required to justify such a design.
4. The experiment setting and explanation is not persuasive. 1) It is unclear why the scores from different datasets can be added together, given that their scales already seem different. 2) Only one set of experiment datasets are evaluated. Although such data setting can be one of the most heterogeneous settings, it may expected to show the effectiveness of the algorithm on more datasets, such as different clients have different coding languages. 3) It is unclear what "client" and "global" mean in Table 2. 4) Only the evaluation on MT-bench is presented, but the results on Code and others are not presented.

1. Many claims sound sensible on paper, but are not supported either empirically or theoretically. An example is how current Fed LLM training overlooks "interpretability of LoRA and the instability" that arises from client drift. We don't see this demonstrated rigorously in the paper. For example, how does the extent of client drift affect the results?

2. The idea of consensus-divergence decomposition is just another name for well-established methods that look at the parameter update's magnitude and direction. It just feels like the authors now apply it to LoRA. As to how this method is made to be specific to LoRA, nothing much was suggested.

3. The paper looks incomplete. The experiments are few, for a paper that has no theoretical backings.

Insufficient methodological innovativeness: the inclusion of Gaussian-Noise Mask by simply judging the importance of the parameters through L2 Norm seems to be simplistic, and the authors do not explain clearly why this is effective. Even if it is effective, the Gaussian-Noise Mask only masks the non-important information, so will the important information be easily recognized. Does this really protect the privacy of important information?

Experimental limitations: The experimental setup of this paper is not robust enough to convincingly demonstrate the effectiveness of the proposed method. It lacks ablation studies, which are crucial to understand the contribution of individual components such as consensus-divergence aggregation and Gaussian noise masking. In addition, the benchmark comparison is outdated and mainly involves studies before 2020, which do not reflect the state-of-the-art in federated learning and LLM fine-tuning.

Unexplored trade-offs: In federated learning, there are inherent trade-offs between model accuracy, data privacy, and fairness among participating clients. These trade-offs are not discussed or analyzed in this paper.