Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning

Reviewer 1yUZ

We would like to thank the reviewer for the insightful comments. Below are our responses to the comments in Weaknesses and Questions.

---

Respond to W1

Answer: We thank the reviewer for their suggestions. Personalized FCL enables clients to maintain individual task histories and learn customized models through cooperation [1,2]. Unlike centralized approaches, decentralized learning eliminates the need for a central server [2,3], better aligning with our goal of optimizing individual client performance (Eq.1) rather than global objectives. As discussed in Section 1 (Paragraphs 2-3), decentralized topologies are more suitable for personalized FCL. The comparison of CFL, Per-FedAvg, and centralized methods (Section 1, line 39) demonstrates that selective cooperative aggregation is essential. We apologize for any lack of clarity and will refine these definitions in the revised manuscript.

---

Respond to W2

Answer: We thank the reviewer for arguing this point. Our work addresses the key challenge of how to achieve personalized continual learning in federated systems with strong spatio-temporal catastrophic forgetting as stated in Section 1.

1. FCL Challenges FCL introduces two key challenges: (1) Local temporal catastrophic forgetting from sequential task learning (2) Spatial catastrophic forgetting during aggregation of heterogeneous models

2. Assumptions and Initial Evidence We first establish that federated aggregation can mitigate forgetting when models share homogeneous knowledge (Fig.1a-b), explaining central server architectures [4,5].

3. Limitations of Complete Aggregation Complete aggregation proves suboptimal under spatial heterogeneity due to knowledge interference (Fig.1c), showing aggregation efficacy depends on methodology.

4. Connection to Personalized FL While methods like ClusterFL (decentralized) and Per-FedAvg handle spatial heterogeneity, they fail to address continual learning challenges (lines 45-49).

This motivates our DCFCL framework featuring dynamic grouping to:Handle continuous task arrivals，Prevent spatio-temporal forgetting，Maintain decentralized personalization

---

Respond to W3

**Answer:**We thank the reviewer for raising this point. We are the first to propose a dynamic grouping strategy for personalized model learning in FCL which is entirely original. We would like to clarify our contributions as follows: 1.Theoretical Foundation Section 3.1 formally establishes the problem of personalized FCL in decentralized topologies. Section 3.2 introduces our novel dynamic grouped aggregation approach for model updates. Section 3.3 applies cooperative game theory to determine optimal groupings, with the cooperative equilibrium as our target solution.

2.Core Innovations in Chapter 4 We extend static cooperative games to dynamic settings to achieve dynamic grouping. While inspired by coalitional affinity games for relationship modeling, all algorithmic components are original:(1) Knowledge distillation (creatively applied) maintains feature consistency for cooperator identification (2) Our benefit table framework uses: Original overall similarity metric,Coalitional affinity principles (3) We created:Merge-blocking algorithm,Dynamic cooperative evolution algorithm,to obtain Dynamic Cooperative Equilibrium

3.Theoretical Analysis Appendix A contains our original arguments about coalitional affinity game applications. Extensive experiments validate:KD,OS,CE.All aspects from problem formulation to equilibrium analysis represent original FCL contributions, except: affinity-based relationship concept,KD's core mechanism. We believes that this can serve as theoretical analysis and guarantee to a certain extent. If we have any misunderstandings, please do not hesitate to question us. We will do our best to clear up your doubts.

---

Respond to W4

Answer: Our experimental results (Fig.7, Appendix D.4) demonstrate that while conventional equilibrium computation exhibits exponential time growth as client numbers increase from 10 to 50, DCFCL maintains near-linear scalability - a performance gap that widens significantly with system scale. The proposed Merge-Blocking Algorithm achieves substantial efficiency gains, including faster equilibrium formation and fewer iterations, while significantly reducing both time complexity and communication costs compared to baseline methods.

We acknowledge the combinatorial complexity challenge, though our current focus remains on moderate-scale federations to study fundamental collaboration dynamics. While computational cost isn't our primary focus given the priority of precise collaborator identification, our framework demonstrates strong performance/speed trade-offs versus alternatives ([6]'s loss distance, [7]'s FC-based, [8]'s gradient-based). Like most group-based FL methods, pairwise similarity computations inherently limit scalability for very large systems. To address scalability we plan to use:1.Sinkhorn Approximation: For large client counts, we employ Sinkhorn's entropy-regularized optimal transport [9] to efficiently approximate similarity matrices through iterative refinement. 2.Localized Assessment: Clients compute similarities only with Top-K neighbors [10] or historical collaborators.3.Sparse Matrix Construction: We retain only high-similarity edges in the similarity graph.4.Algorithmic Optimization: Our merge-blocking algorithm processes large coalitions first and only traverses non-zero matrix entries.

In summery, we will leave these promising extensions to future work and clarify this in revision.

---

Respond to Q1

Answer: We clarify our decentralized FCL approach through four key aspects:

Decentralization Advantage: Unlike centralized aggregation, our decentralized topology better enables personalized model formation, achieving client-satisfactory group aggregation without central server involvement.

Cooperative Game Foundation: Our dynamic group strategy is theoretically grounded in cooperative games, where rational clients form performance-optimizing coalitions while excluding non-contributors - fundamentally differing from clustering methods.

Behavioral Personalization: Client autonomy in alliance formation/betrayal embodies personalization, with the decentralized structure naturally aligning with cooperative game theory's agent-centric dynamics.

Traditional Comparison: While traditional decentralized methods ([11]'s gossip-SGD, [12]'s doubly-stochastic weights) enforce global consensus ([13]), they falter in non-IID settings ([14]). Like [15,16], we adopt decentralization for personalized modeling, but uniquely integrate it with continual learning.

---

Respond to Q2

Answer: Thank you for your questioning. Our overall optimization goal is specific to each client, which is precisely what personalized FL defines[17,18], as described in Section 3.1 Problem Setup. At each task phase $t$, model parameter of client $k$ is $θ_k^t$ , and optimization goal is foreach client. Because it is personalized model and adopts a decentralized structure, we do not have a global optimization objective. This is the main difference from the centralized method, as the latter may optimize a global objective.

---

Respond to Q3

Answer: Thank you for your questioning. In the revised version, we will cite the article of cooperative game theory.

---

Respond to Q4

Answer: We provide comprehensive runtime comparisons with all baselines across datasets in Appendix D.4 (full training wall-time), along with time complexity analysis versus non-optimized cooperative strategies in Appendix B.4. Our method avoids four-loss optimization, and we detail communication costs (Appendix D.2) and computation costs (Appendix D.4). We apologize for any unclear presentation and will clarify these analyses in revision.

[1] J. Xie, FedMeS: Personalized Federated Continual Learning Leveraging Local Memory.

[2] S. Li, Learning to Collaborate in Decentralized Learning of Personalized Models,.

[3] S. Kharrat, DPFL: Decentralized Personalized Federated Learning.

[4] Yoon, J. Federated continual learning with weighted inter-client transfer.

[5] Shenaj, D.. Asynchronous federated continual learning.

[6] M. Zhang, "Personalized Federated Learning with First Order Model Optimization.

[7] Wu, S.. PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers.

[8] Sattler, F.. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints..

[9] Sinkhorn, R. A relationship between arbitrary positive matrices and doubly stochastic matrices.

[10] S. Li, T. Zhou, X. Tian and D. Tao, "Learning to Collaborate in Decentralized Learning of Personalized Models.

[11] Michael Blot, . Gossip training for deep learning.

[12] Zhanhong . Collaborative deep learning in fixed topology networks.

[13] Xiangru Lian, . Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent.

[14] Kevin Hsieh, The non-iid data quagmire of decentralized machine learning.

[15] S. Li, "Learning to Collaborate in Decentralized Learning of Personalized Models.

[16] P. Vanhaesebrouck, , "Decentralized Collaborative Learning of Personalized Models over Networks," arXiv preprint arXiv:1610.05202, 2017.

[17] S. Li, "Learning to Collaborate in Decentralized Learning of Personalized Models," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18] Wu, S., . PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers. .

Reviewer ckij

We would like to thank the reviewer for the positive and insightful comments. Below are our responses to the comments in Weaknesses and Questions.

---

***Weakness 1& Question 1

Answer: We thank the reviewer for their suggestions. While our current experiments focus on moderate-scale federations to study fundamental collaboration dynamics, we acknowledge the combinatorial complexity challenge. Though computational cost isn't our primary focus given our goal of precise collaborator identification for personalized clients, we recognize scalability limitations commonly in methods like [1]'s loss distance, [2]'s FC-based, or [3]'s gradient-based approaches. This reflects a common challenge in group-wise federated systems where pairwise similarity computations are unavoidable.

To address scalability we plan to use:

1. Sinkhorn Approximation: For large client counts, we employ Sinkhorn's entropy-regularized optimal transport [4] to efficiently approximate similarity matrices through iterative refinement.

2. Localized Assessment: Clients compute similarities only with Top-K neighbors [5] or historical collaborators, using compressed parameter representations/PEFT adapters for efficiency.

3. Sparse Matrix Construction: We retain only high-similarity edges (above threshold) in the similarity graph.

4. Algorithmic Optimization: Our merge-blocking algorithm processes large coalitions first and only traverses non-zero matrix entries, with BC-triggered merging/splitting per Algorithm 1.

In summery, we will leave these promising extensions to future work and clarify this in revision.

[1] M. Zhang, K. Sapra, S. Fidler, S. Yeung, and J. M. Alvarez, "Personalized Federated Learning with First Order Model Optimization," in Proc. Int. Conf. Learn. Represent..

[2] Wu, S., Jia, Y., Liu, B., Xiang, H., Xu, X., & Dou, W. (2025). PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers. .

[3] Sattler, F., Müller, K.-R., and Samek, W. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. .

[4] Sinkhorn, R. A relationship between arbitrary positive matrices and doubly stochastic matrices. The Annals of Mathematical Statistics, 35(2), 876-879, 1964.

[5] S. Li, T. Zhou, X. Tian and D. Tao, "Learning to Collaborate in Decentralized Learning of Personalized Models," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition .

---

***Weakness 2& Question 2:

Answer: Thank you for pointing that which inspires our consideration. 1.Nonlinear Separability: When clients share high feature similarity but opposing labels, the proxy may misgroup them. While rare in practice, XOR-style distributions demonstrate this limitation. 2.Extreme Drift: Abrupt task shifts (e.g., CIFAR-10→MNIST vs EMNIST→CT-RATE) can make knowledge transfer harmful, favoring local training over similarity-based aggregation.

---

Weakness 3& Question 3:

Answer: Thank you for your sincere remind. We analyze the convergence properties of our method from two perspectives:

1. Optimization Convergence at Client Level: problem setting

$

\min_{\theta_k}L_k^t(\theta_k)=L_{\mathrm{class}}^t(\theta_k)+\lambda L_{\mathrm{dis}}^t(\theta_k)

$

client update:

$

\theta_{k}^{\tau+1}=\sum_{i\in S}\alpha_{i}(\theta_{i}^{\tau}+\Delta\theta_{i}^{\tau})

$

assumptions: 1.L-smoothness

$

L_k^t(y) \leq L_k^t(x) + \langle \nabla L_k^t(x), y - x \rangle + \frac{L}{2} |y - x|^2

$

2.bounded variance

$

\mathbb{E}_\xi \left[|\nabla L_k^t(x, \xi) - \nabla L_k^t(x)|^2\right] \leq \sigma^2

$

3.unbiased estimation

$

\mathbb{E}_\xi \left[\nabla L_k^t(x, \xi)\right] = \nabla L_k^t(x)

$

proof:

$

\theta_k^{{\tau}+1} &= \sum_{i\in S} \alpha_i (\theta_i^{\tau} + \Delta\theta_i^{\tau}), \quad \Delta\theta_i^{\tau} = -\eta \nabla_\theta L_k^{\tau} (D_k^{\tau}, \theta_k^{\tau}) \\ &= \alpha_k (\theta_k^\tau + \Delta\theta_k^\tau) + \sum_{i\in S} \alpha_i (\theta_i^\tau + \Delta\theta_i^\tau), \quad \sum_{i\in S} \alpha_i = 1 \\ &= \theta_k^\tau + \Delta\theta_k^\tau - \sum_{i\in S \setminus {k}} \alpha_i (\theta_k^\tau + \Delta\theta_k^\tau) + \sum_{i\in S \setminus {k}} \alpha_i (\theta_i^\tau + \Delta\theta_i^\tau) \\ &= \theta_k^\tau + \Delta\theta_k^\tau + \sum_{i\in S \setminus {k}} \alpha_i [\theta_i^\tau - \theta_k^\tau + \Delta\theta_i^\tau - \Delta\theta_k^\tau] \\ &= \theta_k^\tau - \eta \nabla_\theta L_k^\tau (D_k^\tau, \theta_k^\tau) \\ & + \sum_{i\in S \setminus {k}} \alpha_i [\theta_i^\tau - \theta_k^\tau - \eta (\nabla_\theta L_i^\tau (D_i^\tau, \theta_i^\tau) - \nabla_\theta L_k^\tau (D_k^\tau, \theta_k^\tau))]

$

define the disturbance term:

$

\delta_k^\tau = \sum_{i\in S \setminus {k}} \alpha_i [\theta_i^\tau - \theta_k^\tau - \eta (\nabla_\theta L_i^\tau (D_i^\tau, \theta_i^\tau)) - \nabla_\theta L_k^\tau (D_k^\tau, \theta_k^\tau)]

$

norm upper bound of disturbance term:

$

| \delta_k^\tau | \leq \epsilon := \sup | \delta_k^\tau |

$

where

$

\epsilon_{\text{model}} &= \left| \sum_{i\in S \setminus {k}} \alpha_i (\theta_i^\tau - \theta_k^\tau) \right| \\ \epsilon_{\text{grad}} &= \eta \left| \sum_{i\in S \setminus {k}} \alpha_i \left(\nabla_\theta L_i^\tau (D_i^\tau, \theta_i^\tau) - \nabla_\theta L_k^\tau (D_k^\tau, \theta_k^\tau)\right) \right| \\ \text{satisfying} & \quad | \delta_k^\tau | \leq \epsilon_{\text{model}} + \epsilon_{\text{grad}} = \epsilon

$

In our dynamic cooperation, $S$ differs with times.

$

\theta_k^{\tau+1} - \theta_k^\tau &= -\eta \nabla L_k^\tau(\theta_k^\tau) + \delta_k^\tau \\ L_k(\theta_k^{\tau+1}) &\leq L_k(\theta_k^\tau) + \langle \nabla L_k(\theta_k^\tau), -\eta \nabla L_k(\theta_k^\tau; \zeta_k^\tau) + \delta_k^\tau \rangle \\ &\quad + \frac{L}{2} | -\eta \nabla L_k(\theta_k^\tau, \zeta_k^\tau) + \delta_k^\tau |^2 \\ \mathbb{E}[L_k(\theta_k^{\tau+1})] &\leq \mathbb{E}[L_k(\theta_k^\tau)] - \eta \mathbb{E}[|\nabla L_k(\theta_k^\tau)|^2] + \mathbb{E}[\langle \nabla L_k(\theta_k^\tau), \delta_k^\tau \rangle] \\ &\quad + \frac{L\eta^2}{2} \mathbb{E}[|\nabla L_k(\theta_k^\tau, \zeta_k^\tau)|^2] + \frac{L}{2} \mathbb{E}[|\delta_k^\tau|^2] \ &\quad + L\eta \mathbb{E}[\langle \nabla L_k(\theta_k^\tau, \zeta_k^\tau), \delta_k^\tau \rangle]

$

with bounded conditions:

$

\mathbb{E}[|\nabla L_k(\theta_k^\tau, \zeta_k^\tau)|^2] &\leq |\nabla L_k(\theta_k^\tau)|^2 + \sigma^2 \\ |\delta_k^\tau|^2 &\leq \epsilon^2

$

we have:

$

\mathbb{E}[L_k(\theta_k^{\tau+1})] &\leq \mathbb{E}[L_k(\theta_k^\tau)] \\ &- (\eta - C_1\eta^2 - C_2\eta \epsilon - C_3 \epsilon) \mathbb{E}[|\nabla L_k(\theta_k^\tau)|^2] \\ &+ C_4 (\eta^2\sigma^2 + \epsilon^2)

$

set:

$

\eta &= \frac{1}{\sqrt{\tau}} \quad \Delta := L_k(\theta_k^0) - L_k^* \leq C \\ \gamma &:= \eta - C_1\eta^2 - C_2\eta \epsilon - C_3 \epsilon \\ \tau &= 0 \cdots T-1 \\ \mathbb{E}[L_k(\theta_k^{\tau})] &\leq \mathbb{E}[L_k(\theta_k^0)] - \sum_{\tau=0}^{T-1} \gamma \mathbb{E}[|\nabla L_k(\theta_k^\tau)|^2] \\ &+ TC_4 (\eta^2\sigma^2 + \epsilon^2) \\ \sum_{\tau=0}^{T-1} \mathbb{E}[|\nabla L_k(\theta_k^\tau)|^2] &\leq \frac{L_k(\theta_k^0) - L_k(\theta_k^{\tau})}{\gamma} + \frac{TC_4}{\gamma} (\eta^2\sigma^2 + \epsilon^2) \\ \frac{1}{T} \sum_{\tau=0}^{T-1} \mathbb{E}[|\nabla L_k(\theta_k^\tau)|^2] &\leq O\left(\frac{1+\sigma^{2}+\epsilon^{2}}{\sqrt{T}}\right)

$

which indicates optimization convergence on each client.

2. Coalition evolution convergence: Our merge-blocking strategy ensures that the coalition structure evolves toward a stable state.

---

***Weakness 4& Question 4:

Answer: Our framework design explicitly addresses this concern through experimental validation of coalition update frequencies. Results across 10-communication-round task phases demonstrate that per-round coalition evaluation, while computationally intensive, is essential for handling unpredictable data arrival patterns during task learning.

Timely collaborator identification is critical as mini-batches arrive during task training. Infrequent coalition updates risk failing to adapt to emerging patterns - e.g., when learning new classes like "dog", delayed updates maintain outdated collaborations, increasing catastrophic forgetting and accuracy degradation.

---

Limitations 2:

Answer: Thank you. We acknowledge this parameter sensitivity and plan to implement auto-tuning in future work.

---

Limitations 3

Answer: Thank you. In practice, an impartial third-party (e.g., industry association) handles model collection, benefit assessment, and cooperation publishing in our federated learning paradigm. Crucially, our approach differs from centralized learning in two aspects: Only evaluates optimal coalitions (preventing self-interest) without performing model aggregation/distribution, which remains client-autonomous. Preserves client-specific learning trajectories via decentralized knowledge transfer, avoiding global model imposition.

We appreciate the privacy concerns raised and will address trusted third-party aspects in future work.

Reviewer 88x1

We would like to thank the reviewer for the useful comments. Below are our responses to the comments in Weaknesses.

---

Weakness 1: In the Introduction, the authors claim that spatial catastrophic forgetting is caused by aggregation. However, the subsequent preliminary experiments demonstrate that federated aggregation helps alleviate forgetting, which appears contradictory. Could the authors clarify this seeming inconsistency?

Answer: Thanks for the useful comments. We will clarify our statement from following aspects.

1. Federated Aggregation Benefit: We acknowledge that federated aggregation can mitigate forgetting, as knowledge from other clients helps recall prior tasks. This is illustrated in Section 2, Paragraph 2, via Fig. 1(a)(b), highlighting FCL’s value and motivating the use of central aggregation [1,2].

2. Critical Limitation: However, we question whether aggregating all models is always beneficial. We argue that blind aggregation is suboptimal—especially under strong spatial heterogeneity, which may cause severe knowledge interference, termed spatial catastrophic forgetting in prior work [3,4].

3. Empirical Validation and Motivation: As shown in Paragraph 3 and Fig. 1$c$, we empirically verify this concern. We emphasize that selective and cooperative aggregation is essential for personalized continual learning, motivating the design of our DCFCL framework.

[1] Yoon, J., Jeong, W., Lee, G., Yang, E., & Hwang, S. J. (2021). Federated continual learning with weighted inter-client transfer. In Proceedings of the 38th International Conference on Machine Learning (pp. 12073-12086). PMLR.

[2] Shenaj, D., Toldo, M., Rigon, A., & Zanuttigh, P. (2023). Asynchronous federated continual learning. In *2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE.

[3] X. Yang, H. Yu, X. Gao, H. Wang, J. Zhang and T. Li, "Federated Continual Learning via Knowledge Fusion: A Survey," in IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 8, pp. 3832-3850, Aug. 2024.

---

Weakness 2: The manuscript does not provide a clear definition of Personalized Federated Continual Learning (FCL). What exactly distinguishes personalized FCL from decentralized FCL? A clearer conceptual distinction between the two is necessary.

Answer: Thanks for the useful comments. Here we give the definition of two concepts.

1.Personalized FCL: Personalized FCL is a scenario where clients aim to preserve their individual task histories and learn models tailored to their own data distributions across time. In PFCL, clients can share models to facilitate their own performance [1,2].

2.Decentralized Learning: Decentralized learning refers to the absence of a central server in communication topology [2,3]. Clients spontaneously aggregate models and don't rely on central server. Our work emphasizes the personalized continual learning objective under decentralized settings.

3.Framework Objectives and Rationale: Since our personalized FCL framework is based on dynamic cooperation, the objective is to optimize the performance of each rational client rather than a global objective as shown in Eq.1.:

$

\underset{\theta_k^t}{\operatorname*{\operatorname*{argmin}}}\underset{\mathcal{D}_k^t}{\operatorname*{\operatorname*{E}}}[L_k^t(\mathcal{D}_k^t,\theta_k^t)]

$

which naturally requires no central architecture. Thus, we argue decentralized topology better suits personalized FCL than centralized, as discussed in Section 1 (Paragraphs 2-3).

4. Revision Plans: In the revised version, we will make two definitions more precise in Section 1.

[1] J. Xie, C. Zhu, and S. Li, "FedMeS: Personalized Federated Continual Learning Leveraging Local Memory," arXiv preprint arXiv:2404.12710, 2024. [2] S. Li, T. Zhou, X. Tian and D. Tao, "Learning to Collaborate in Decentralized Learning of Personalized Models," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 2022.

---

Weakness 3: Regarding the client grouping strategy for utility computation: is it based on exhaustive enumeration or another approach? Given that the benefit comparison involves dynamic equilibrium, which can be computationally expensive, are there any strategies proposed to reduce time complexity?

Answer: Thanks for the comments. Our coalition formation avoids exhaustive enumeration. Instead of exploring all grouping states, we iteratively merge coalitions under the PT condition (Profitable Transition: merging occurs only when clients gain higher benefits). For complexity reduction, we employ these strategies:

1. Benefit Calculation via Coalitional Affinity Game: To avoid prohibitive computation/communication costs from exhaustive local aggregation, we propose overall similarity to quantify pairwise client benefits. Using coalitional affinity game, we prove multi-client benefits can be derived from 2-client relationships via function $f(\cdot)$ (Appendix A). This reduces complexity from $O(\sum_{k=1}^K C_K^k)$ to $O(C_K^2)$ (Appendix B.4).

2. Merge-Blocking Algorithm: Our iterative algorithm (Section 4.2) avoids traversing the exponential partition space. While each iteration remains $O(K2^K)$, it significantly improves over the $O(B_K 2^K)$ complexity of full state-space traversal.

3. Pruning in Merge-Blocking: Redundant subset comparisons are eliminated each round. Empirical results show fast convergence regardless of client count, with formation times substantially reduced versus non-optimized approaches.

---

Weakness 4: Does the proposed method involve storing state spaces? If so, what is the associated space complexity?

Answer: We thank the reviewer for raising this concern. Our method avoids explicit storage of the entire state space by maintaining only a small cache of previous-round model states and utility values for current candidate coalitions. The space complexity analysis shows this design achieves linear scaling: each client requires $O(d)$ local storage for knowledge distillation (significantly less than rehearsal-based CL approaches that store prior task data/generators), while the third-party coordinator maintains $O(2Kd)$ space for current/previous models of all clients (detailed in Section 4.3 and Appendix B.1). We will further clarify this space complexity analysis in the final manuscript.

---

Weakness 5: In Table 3, two important hyperparameters significantly affect model performance. The selection of these hyperparameters is critical for practical deployment. Additionally, on the EMNIST-shuffle dataset, model performance appears more sensitive to these hyperparameters. Is there any analysis or discussion provided on this issue?

Answer: We thank the reviewer for raising this concern. To further illustrate, we conduct an extended sensitivity analysis in the following tables.

Sensitivity Analysis of λ and ε on MNIST-SVHN-F

We compared the maximum fluctuation in accuracy when one value was fixed and the other changed. $\lambda=0.4,max(Range_\varepsilon) =3.6$ $\varepsilon =0.2,max(Range_\lambda) =3.8$

Sensitivity Analysis of λ and ε on EMNIST-Shuffle

Similarly, for the emnist-shuffle dataset: $\lambda =0.0,max(Range_\varepsilon) =17.7$ $\varepsilon =1.0,max(Range_\lambda) =10.3$

EMNIST-shuffle shows greater sensitivity to $\lambda$ and $\varepsilon$ due to extreme class imbalance, requiring larger $\varepsilon$ to better leverage global model information for collaborator identification - highlighting the critical importance of proper similarity weight settings for highly heterogeneous datasets.

Response to W1:

Answer: We thank the reviewer for their insightful suggestion. While our current work investigates continual learning from scratch to analyze task coordination in a controlled setting, we acknowledge the growing importance of pre-trained models (e.g., LLMs, vision backbones) [6] and parameter-efficient tuning (e.g., adapters, LoRA) [7] in practical FCL. Here we concisely address this concern from three perspectives:

1. Method Comparision: While these methods benefit from strong initialization and existing knowledge, leading to reduced training costs and improved early-stage performance, our approach concentrates on learning from scratch. This allows us to directly study coalition-based knowledge retention and generalization dynamics in federated continual learning without relying on pre-trained priors. Notably, our method is complementary to PEFT-based approaches and could potentially be integrated with them in future work.
2. Revised Version: In the revised version, we will add a dedicated paragraph in the Related Work section discussing these approaches.
3. Method Extension: To demonstrate our framework's adaptability to pre-trained foundation models, we conducted preliminary experiments applying our DCFCL approach with PEFT. Specifically:Replaced local models with a pre-trained ViT-B/16 backbone (iBOT-21K). Implemented adapter-based PEFT, where only PEFT parameters are updated and transmitted. Maintained our original coalition similarity computation using PEFT parameter embeddings. Preserved our benefit calculation and coalition evolution framework. Compared against FedAvg+adapter [6] and ClusterFL+adapter [8] baselines on CIFAR100. Our experimental results are as follows

As shown in the Table, DCFCL+Adapter achieves the best accuracy and the lowest forgetting rate on CIFAR-100. These results validate the generalizability of our mechanism in PEFT scenarios.

---

Response to W2:

Answer: Thank you for pointing this out. In our experiments, we follow recent works [9,10] and use average accuracy and average forgetting as evaluation metrics. Let $\alpha_k^{t,i}$ denote the test accuracy on task $i$ after learning task $t$ in client $k$. Average forgetting measures backward transfer, defined as the difference between peak and final accuracy of each task. A weighted average is applied in its calculation:

$

\text{Average Forgetting} = \frac{1}{\sum_{k=1}^{K}\sum_{i=1}^{T-1} n_k^i} \sum_{k=1}^{K}\sum_{i=1}^{T-1} \max_{t \in {1,\cdots,T-1}} (a_k^{t,i} - a_k^{T,i}){n_k^i}

$

In our revision, we will supplement this point in the experimental section.

---

Response to Q1

Answer: Thank you for your valuable comment. We would like to clarify a possible misunderstanding regarding the use of KL loss in our method.

1. Clarification: Our method does not rely on KL divergence to measure task coordination. Instead, knowledge distillation is used as an auxiliary tool to preserve feature consistency during continual learning (Section 4.1). As new tasks alter the classifier, it becomes harder to identify cooperators for knowledge recall. By maintaining feature space consistency, we enable efficient cooperator matching using existing model information, with minimal communication overhead.
2. Empirical Validation: We experimentally verify that our method, with KD as an auxiliary tool, better mitigates catastrophic forgetting than naive KD. Even applying KD to all clients fails to achieve optimal performance without our framework, as shown in Section C.2, line 530 ("To control variables during local training, we adopted the same KD method into all FL baselines, which still obtained suboptimal results.").
3. Relation to Identifying Cooperators: We use knowledge distillation between the classifier’s current and past outputs to maintain feature space consistency and prevent drift during continual learning. This helps retain discriminative features from previous tasks, enabling accurate identification of similar clients without additional model exchange. As shown in Fig.2, clients choose different cooperators at each task stage based on the trade-off between acquiring new knowledge and retaining prior learning. KD helps clients find suitable partners to preserve prior knowledge. In short, distillation is not for coordination estimation but for preserving useful representations to support similarity-based cooperation under dynamic, heterogeneous conditions.

---

Respond to Q2

Answer: Thank you for raising this. Based on our investigation, many existing works estimate task similarity via the following several mainstream metrics:

1. L2-based: the $l_2$ distance across client models. [1]
2. Gradient-based: only use the cosine similarity of client gradients.[2,3]
3. EDC: truncate gradients onto key directions via Singular Value Decomposition and computes similarity in this reduced space. [4]
4. FC-based: the cosine similarity between client classifier parameters.[5]

Under our experimental settings, we only replace the overall similarity part. The experimental results are as follows:

Using gradient-based methods alone leads to severe forgetting, as gradient similarity struggles to identify models aligned with prior tasks, hindering knowledge retention. EDC reduces computation by truncating the gradient matrix and approximating similarity but yields suboptimal performance. In contrast, our overall similarity accurately identifies cooperators and evaluates benefits, achieving the best results in both accuracy and forget rate.

[1] M. Xie, G. Long, T. Shen, T. Zhou, X. Wang, and J. Jiang, “Multi-center federated learning,” arXiv preprint arXiv:2005.01026, 2020.

[2] Sattler, F., Müller, K.-R., and Samek, W. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8): 3710-3722, 2021. doi: 10.1109/TNNLS.2020.3015958.

[3] Li, Z., Lin, T., Shang, X., & Wu, C. (2023). Revisiting Weighted Aggregation in Federated Learning with Neural Networks. arXiv preprint arXiv:2302.10911.

[4] M. Duan, D. Liu, X. Ji, R. Liu, L. Liang, X. Chen, and Y. Tan, "FedGroup: Efficient Federated Learning via Decomposed Similarity-Based Clustering," in Proc. IEEE Int. Symp. Parallel Distrib. Process. Appl , 2021, pp. 228-237.

[5] Wu, S., Jia, Y., Liu, B., Xiang, H., Xu, X., & Dou, W. (2025). PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20), 21572-21580.

[6] Cai, D., Wu, Y., Wang, S., Lin, F. X., and Xu, M. FedAdapter: Efficient Federated Learning for Modern NLP. arXiv preprint arXiv:2205.10162, 2023.

[7] Yi, L., Yu, H., Wang, G., Liu, X., and Li, X. pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning. arXiv preprint arXiv:2310.13283, 2024.

[8] Sattler, F., Müller, K.-R., and Samek, W. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8): 3710-3722, 2021. doi: 10.1109/TNNLS.2020.3015958.

[9] Wuerkaixi, A., Cui, S., Zhang, J., Yan, K., Han, B., Niu, G., Fang, L., Zhang, C., and Sugiyama, M. Accurate Forgetting for Heterogeneous Federated Continual Learning. In The Twelfth International Conference on Learning Representations (ICLR), 2024.

[10] Qi, D., Zhao, H., and Li, S. Better Generative Replay for Continual Federated Learning. In The Eleventh International Conference on Learning Representations (ICLR), 2023.

Dear reviewers,

If you have discussed well with the authors regarding the potential concerns, thank you very much for your effort.

If you haven't done so, please follow the official review policy of NeurIPS 2025 during the author-review discussions.

• Reviewers must participate in discussions with authors before submitting “Mandatory Acknowledgement”. The non-participating reviewers will receive possible penalties of this year's responsible reviewing initiative and future reviewing invitations.
• Clicking “Mandatory Acknowledgement” prematurely does NOT release Reviewers from participating in discussions.
• Please note “Mandatory Acknowledgement” button is to be submitted only when reviewers fulfill all conditions below (conditions in the acknowledgment form):
  • read the author rebuttal
  • engage in discussions (reviewers must talk to authors, and optionally to other reviewers and AC - ask questions, listen to answers, and respond to authors)
  • fill in "Final Justification" text box and update “Rating” accordingly (this can be done upon convergence - reviewer must communicate with authors first)
• To facilitate discussions, the deadline of Author-Reviewer discussions has been extended to Aug 8, 11.59pm AoE. Please discuss with the authors regarding your concerns about this submission.

Best,

AC