We would like to thank the reviewer for the insightful comments. Below are our responses to the comments in Weaknesses and Questions.

---

Question 1: The authors' use of feature correlation for replay training is intriguing, though the batch-wise weighting method employed seems somewhat simplistic. Recent research has explored orthogonal training techniques to mitigate the influence of uncorrelated or biased old features. Consequently, a more nuanced feature weighting mechanism might yield improved results.

Answer: Thanks for the insightful advice. The incorporation of orthogonal training and our accurate forgetting method is a promising direction. We have supplemented the discussion in the paper and cited the work of orthogonal training in the FCL scenario [1].

[1] proposed to modify the subspace of model layers in learning new tasks such that it is orthogonal to the global principal subspace of old tasks. By distinguishing the subspace inside the model for each task, catastrophic forgetting of old tasks is mitigated, and it also relieves the influence of unrelated tasks. We will continue to explore the employment of orthogonal training in our method.

Our method explicitly quantifies the correlations of generated features through probability calculations. Moreover, we facilitate selective forgetting by assigning lower weights to erroneous old knowledge, thus enabling the classifier to discard biased features and achieve improved overall performance. We are happy to know that orthogonal training techniques could mitigate the influence of uncorrelated or biased old features. We will continue this topic in the future.

[1] Bakman, Yavuz Faruk, et al. "Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning." arXiv preprint arXiv:2309.01289 (2023).

---

Question 2: It would be beneficial for the authors to provide a more detailed rationale for the application of the Normalization Flow (NF) model in this context. It is unclear why the NF model was chosen specifically. It may be valuable to conduct experiments to demonstrate the necessity of the NF model in comparison to other generative models.

Answer:

We would like to thank the reviewer for the professional comments. We would like to explain it as follows:

1. The NF model is able to accurately estimate the probability density of observed data. With such unique capabilities absent in other generative models, we evaluate the benefits of generating features for the current task and achieve accurate forgetting through probabilistic estimation.

2. NF model could map an arbitrarily complex data distribution to a pre-defined distribution losslessly through a sequence of bijective transformations. Such invertability enables the NF to have a lossless memory of the input knowledge.

3. We have supplemented ablation studies validating that the NF model is superior to the GAN model in our method; thereby, the correlation estimation module of our method also loses effects. As the results shown in the table below, AF-FCL-GAN degrades into naive generative-reply based method, thus the performance has significantly declined.

---

Dear reviewer hHgp:

We would like to appreciate your precious efforts in reviewing our paper. Meanwhile, we have done our best to clarify all your concerns in our rebuttal. For example, for the issue you are concerned about: rationale for the application of the Normalization Flow (NF) model, we would like to clarify that we utilize the unique characteristic of probability estimation of the NF model in our method, which we elaborate on in the answers to Question 2.

Therefore, would you mind checking our response, and is there any unclear point so that we could further clarify?

Best regards,

Authors

We would like to thank the reviewer for the insightful comments. Below are our responses to the comments in Weaknesses and Questions.

---

Question 1: It is not straightforward to understand how difficult the baselines are because FedAvg and FedProx, without any continual learning mitigation mechanism, have about 8% forgetting. Maybe this is due to the fact that the performance of the models is not good even in the current tasks.

Answer: Thanks for the valuable comments. We would like to explain as follows:

1. We fine-tuned different hyper-parameters to guarantee convergency of each task and achieved optimal performance. As shown in the table below, we chose a local iteration count of 400 as it leads to near convergence in our experiments.

2. The relatively low degree of forgetting observed can be attributed to the complexity of each task, combined with the collaboration among clients, which mitigates the forgetting effect. The CIFAR100 dataset is a challenging dataset for the FCL problem. In this context, the federated model is tasked with classifying 100 distinct classes, where the baseline accuracy of random guessing stands at 1%. Consequently, achieving 26.3% accuracy with the FedAvg baseline represents a reasonable performance benchmark. Models trained on limited data for a single task perform poorly; however, when they learn more tasks and collaborate with other users, their performance relatively improves, mitigating the impact of catastrophic forgetting. For example, as the accuracy of each task in each time step shown in the table below, the accuracy of task 1 increases after the model is trained on the second task.

---

Question 2: In all the experiments, N (number of clients) is very small (only 10).

Answer: Thanks for the practical comments. We would like to explain as follows:

1. We follow existing work in FCL for consistency. The number of clients is set as 5 in related works [1].

2. The dataset with a small number of clients is more challenging in the FCL scenario. When there are numerous clients, a client is highly likely to spot collaborative clients with the same current task and previous tasks, which relieves the forgetting problem in FCL.

3. We supplement experiments on the EMNIST-LTP dataset with more clients ($N$ denotes the number of clients) as displayed in the table below. As previously noted, an increase in the number of clients results in diminished forgetting across all methods, thereby enhancing accuracy. Moreover, naive FL baselines, such as FedAvg and FedProx, exhibit performance comparable to those with explicit memorization techniques when the client count surpasses 14. Our method, which more effectively manages statistical heterogeneity among clients, consistently outperforms these baselines in most cases.

[1] Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.

---

Question 3: Do task transitions for the clients happen simultaneously, or do tasks change in different clients independently?

Answer: Following the existing work of FCL, we assume task transitions for the clients happen simultaneously.

---

Question 4: Is any example saved in the memory for AF-FCL? What is the memory size for the memory-based baselines (GFCL and FLwF2T)?

Answer: (1). There is no example saved in memory for AF-FCL. Following the setting in the original paper, we set the exemplar memory as shown in the table below.

(2). We set the same size of exemplar memory for GFCL and FLwF2T. Besides, for fair comparison, we add perturbations to all prototype samples, as in GFCL.

---

Question 5: How robust is AF-FCL to data heterogeneity? Could you please include the information regarding how heterogeneous clients' data distribution is?

Answer: We would like to thank the reviewer for the helpful advice.

1. By selectively utilizing global knowledge, AF-FCL is highly robust to statistical heterogeneity. We accurately identify benign knowledge through probability estimation, thus mitigating the adverse impact of statistical heterogeneity and erroneous information.

2. For the experiments in the paper, the statistical heterogeneity among clients is simulated by randomly assigning different classes of data to each client. For example, in the EMNIST dataset, each client possesses 12 classes of data among all 26 classes. In the table below, we display the probability of any two different clients possessing the same class of data and the probability of any two tasks for two different clients possessing the same class of data. We calculate the probability using the actual class overlaps in the constructed datasets.

3. To further examine the influence of statistical heterogeneity, we supplement experiments on the EMNIST dataset distributed with a Dirichlet distribution to represent real-world data distribution patterns and control heterogeneity. We partition the dataset into 48 tasks with Dirichlet distributions $Dir_{48}(0.1)$ and $Dir_{48}(0.1)$ following the work in [4]. And we randomly assign 6 tasks to each of the 8 clients. The results of our method and baselines are shown in the table below. Under a more heterogeneous setting ($Dir_{48}(0.1)$), the collaboration among users has weakened, and the potential bias has intensified, resulting in a significant decline in the performance of all methods. While AF-FCL outperforms all baselines under both high and low degrees of heterogeneity by selectively utilizing learned knowledge.

---

Question 6: What is the communication cost for AF-FCL?

Answer: The communication mode of AF-FCL is the same as FedAvg. After training locally, the clients send gradients of the model to the server. Then the server transmits the aggregated gradients back to clients. Therefore, same as other baselines, the communication cost for AF-FCL is: number of clients $\times$ number of communication rounds $\times$ number of model parameters.

---

Question 7: What is the number of parameters in NF models?

Answer: The number of parameters in NF models from AF-FCL is shown in the following table. In comparison, we also show the number of paraemters in GAN models from FedCIL.

From the above table, the number of parameters in the NF models is comparable to the GAN models.

---

Question 8: Are all the clients participating in the training every round?

Answer: Following the setting of existing works, all the clients participate in the training every round in both the proposed method and baselines.

---

Question 9: How is g initialized? What is the forgetting in the NF model?

Answer: We would like to explain it as follows:

1. The parameters of the NF model are initialized randomly from a normal distribution. While learning in the first round of the first task, the NF model is only trained but not used.
2. The NF model evaluates the benefits of generating features for the current task through probabilistic estimation. It enables the classifier to forget erroneous features by assigning lower weights to biased features. Then, by learning from the feature space of the purified classifier, the NF model also forgets erroneous memories.

Dear reviewer T35E:

We would like to appreciate your precious efforts in reviewing our paper. Meanwhile, we have done our best to clarify all your concerns in our rebuttal. For example, for the issue you are most concerned about: the number of clients is small in the experiments, we would like to clarify that we followed existing works and have supplemented experiments, which we elaborate on in the answers to Question 2.

Therefore, would you mind checking our response, and is there any unclear point so that we could further clarify?

Best regards,

Authors

We would like to thank the reviewer for the positive and insightful comments. Below are our responses to the comments in Weaknesses.

---

Question 1: The main contribution of this work is disentangling and removing biased harmful features, while FCL merely serving as a relevant scenario. It would be better if the authors introduced the FCL formulation briefly and focused on the biased features in the memory bank.

Answer: Thanks for the valuable comments. As you mentioned, our main contribution of this work is disentangling and removing biased harmful features. More importantly, we find that forgetting could be beneficial for model learning. FCL is a natural scenario where statistical heterogeneity exists, making it possible to improve performance by accurating forgetting. We agree that our method may be applicable in other scenarios besides FCL. Following the advice, we have revised our paper and discussed the practicability of our method in other learning scenarios.

---

Question 2: It would be better if the authors could illustrate or formally define "biased features."

Answer: Thanks for the constructive advice. We have supplemented the definition of biased features in the paper.

Researchers have employed various definitions for biased features, one of which involves defining them as spurious correlations. We denote $\mathcal{X}$, $\mathcal{Y}$ as the input and output spaces of machine learning algorithms. An algorithm learns a mapping from the data $x\in\mathcal{X}$ to the prediction $\hat{y}\in\mathcal{Y}$: $\hat{y}=f(x)$. We assume there are attributes $\gamma_1, \gamma_2,...$ abstracted from the data $x$. For example, $\gamma_1$ represents the shape of the object in the input image $x$, and $\gamma_2$ denotes the number of black pixels in the input image $x$. The machine learning algorithm actually relies on many attributes to infer: $\hat{y}=f(\gamma_{i_1}, \gamma_{i_2},..., \gamma_{i_N})$. We define an attribute $\gamma$ as a biased feature if it does not comply with the natural meaning of the target $y$ [1]. Relying on such a biased attribute would result in poor generalizability of the algorithm. The biased features could be attained through a biased training dataset, and the learned mapping $f$ relying on the biased features may not perform well in the testing dataset. For instance, if in the training image dataset all cows are standing on the grass, the machine learning model may rely on the attribute 'grass' for classifying images of cows.

[1] Jeon, Myeongho, et al. A conservative approach for unbiased learning on unknown biases. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

---

Question 3: Despite EMNIST-noisy, the authors haven't explained the reason why using all other datasets could introduce biased features.

Answer: The reasons other datasets could introduce biased features are as follows:

1. There could be noisy data in all datasets, including images with gaussian noise or falsely labeled data. These noisy data could result in biased features.

2. Statistical heterogeneity could exacerbate the harmful effects of biased features. The datasets in our study are distributed across various clients, exhibiting statistical heterogeneity. This heterogeneity, coupled with instances of insufficient or biased training data, can lead to the emergence of biased features. Recent research has indicated that such statistical heterogeneity can exacerbate the adverse effects of these biased features [2].

3. Task heterogeneity could introduce biased features in most FL scenarios. In all datasets, there are many tasks. And there could be some unrelated tasks that result in spurious correlations with other tasks. For instance, in the MNIST-SVHN-F dataset, different tasks rely on different features. Shape features that are relevant to digit classification in MNIST differ significantly from those that are important for classifying clothing items in FashionMNIST. If clients collaborate naively, it may result in a model that relies too heavily on spurious correlations, thus neglecting the significance of task-specific features.

[2] Hong, Junyuan, et al. Federated robustness propagation: Sharing adversarial robustness in federated learning. (2021).

---

Question 3: A client could theoretically generate raw data by first using the NF model from the previous FedAvg to generate numerous features, then converting these back to the data space using deconvolution methods. AF-FCL permits each client to generate a feature space distribution, which includes information from all clients in the previous round. This potential problem is not found in standard FL methods. Is there a more effective solution to this data privacy concern?

Answer: Thanks for pointing out the issue. We would like to clarify it as follows:

1. Compared with related works in the FCL domain, our approach incorporates more considerations regarding privacy. For instance, a global generative model for raw data is applied in [4]. Compared with baselines, AF-FCL generates features rather than raw data, which mitigates privacy leakage. And we only transmit gradients of model parameters between clients and servers, as in FedAvg. Besides, there are several works in FL utilizing generative models in the feature space of classifiers to aid collaboration among clients [5,6].

2. Our method can be easily modified to provide enhanced privacy protection and avoid the leakage of raw data through deconvolution methods. Instead of sharing the complete classifier among clients, we can keep the feature extrator of the classifier personalized and local for each client, only sharing the classifier head and NF model between the server and clients. By concealing the local feature extrator, which is the only model processing raw data, we can better protect client privacy. Without access to the feature extrator, one cannot attain raw data by deconvolution methods.

3. Our method is also compatible with existing privacy protection techniques like differential privacy. By elaborately adding noise to the local model updates and homomorphic encryption during model aggregation, we can train a global model adhering to a predefined privacy budget.

[4] Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.

[5] Zhu, Zhuangdi, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. International conference on machine learning. PMLR, 2021.

[6] Wu, Yuezhou, et al. Fedcg: Leverage conditional gan for protecting privacy and maintaining competitive performance in federated learning. arXiv preprint arXiv:2111.08211 (2021).

---

Question 4: The proposed AF-FCL appears to "accurately forget" prior correct data. The effects are unclear in the following scenarios: (a) The malicious (falsely labeled) data occurs in the middle of the FL task. (b) Increased heterogeneity among clients and tasks during the FL task, and comparison of AF-FCL to the baseline.

Answer: Thanks for the very constructive comments. We would like to clarify it as follows:

1. For scenario (a), we guess this problem is similar to Question 1. Please refer to the answer to Question 1. If we misunderstand this problem, please feel free to point it out at any time.

2. For scenrio(b), we experimentally validated that AF-FCL could effectively utilize memorized knowledge in scenarios with increased heterogeneity. We supplement experiments on the EMNIST dataset distributed with a Dirichlet distribution to represent real-world data distribution patterns and control heterogeneity. We partition the dataset into 48 tasks with Dirichlet distribution $Dir_{48}(0.1)$ and $Dir_{48}(0.1)$ following the work in [4]. And we randomly assign 6 tasks to each of the 8 clients. The results of our method and baselines are shown in the table below. Under a less heterogeneous setting ($Dir_{48}(0.5)$), clients collaborate more closely with each other, thus the accuracy is higher. From the table below, AF-FCL outperforms all baselines under both high and low degrees of heterogeneity.

[4] Wang, Jianyu, et al. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems 33 (2020): 7611-7623.

---

We would like to thank the reviewer for the insightful comments. Below are our responses to the comments in Weaknesses and Questions.

---

Question 1: In Section 4.2's experiment, noise interference is limited to the initial three tasks. This doesn't adequately assess the impact on each method when the noise occurs at random or during intermediate stages, which would be a more general scenario.

Answer: Thanks for the insightful comments. We would like to clarify it as follows:

1. In fact, our method is naturally suitable for cases where the noise occurs at random or during intermediate stages. In this paper, we address the scenario where biased information resides in the memory bank and propose methods to mitigate its detrimental impact on the current task. While our method can also manage the situation where noisy tasks occur at random stages or during intermediate stages. When the model learns on the current task containing noise, AF-FCL measures the probability $p_D(h_{a}(x_i))$ of local data $x_i$, rather than the generated data, within the global data distribution using the NF model. And the loss objective training on local data is as follows:

$$\mathcal{L}_{ce}^x(h; D_k^t)=\frac{1}{n_k^t}\sum\_{i=1}^{n\_k^t}k\cdot p\_D(h\_{a}(x\_i))\cdot\mathcal{L}\_{ce}(h(x\_i), y\_i),$$

where $k\cdot p_D(h_{a}(x_i))$ denotes the weights of the local data in the loss objective $\mathcal{L}_{ce}^x$ and $k$ is a hyper-parameter. The noise in the current task typically has relatively lower probability compared with the beneficial feature. By using the global probability as the weight in the loss objective, we mitigate the adverse effects of low-probability biased data of current task.

2. We experimentally validate that AF-FCL could address the noise when it occurs at random or during intermediate stages. We supplement experiments when the noisy tasks in Section 4.2's experiment occur at random stages. The average accuracy and forgetting of normal tasks are displayed in the table below. Our method consistently surpasses all baselines by alleviating the negative influence of noisy clients.

---

Question 2: The problems in Section 4.2 appear to overlap with those discussed in studies on Concept Drifts in Federated Learning (FL). To emphasize the importance and influence of this research, it would be beneficial to distinguish it from Concept Drifts in Federated Learning, incorporate these into the related works, or conduct additional experiments.

Answer: Thanks for the insightful advice about expanding the horizons of this research. Following your kind advice, we have supplemented the discussion and cited the related work regarding the studies on Concept Drifts in Federated Learning.

Different from the studies about Federated Continual Learning, the evaluation in the concept drift studies is conducted at each time step. Therefore, there is no memorization requirement or catastrophic forgetting problem in the concept drift studies. [1] proposed a novel clustering algorithm for reacting to concept drifts. Adaptive-FedAVG adapted the learning rate to react to concept drift [2]. [3] proposed to detect concept drift through the magnitude of parameter updates and designed a novel adaptive optimizer.

[1] Ellango Jothimurugesan, Kevin Hsieh, Jianyu Wang, Gauri Joshi and Phillip B. Gibbons. Federated Learning under Distributed Concept Drift. International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain.

[2] Canonaco, Giuseppe, et al. Adaptive federated learning in presence of concept drift. 2021 International Joint Conference on Neural Networks (IJCNN). IEEE, 2021.

[3] Panchal, Kunjal, et al. Flash: Concept Drift Adaptation in Federated Learning. 2023.

---

Dear reviewer cH3G:

We would like to appreciate your precious efforts in reviewing our paper. Meanwhile, we have done our best to clarify all your concerns in our rebuttal. For example, for the issue you are most concerned about: situations where noise occurs at random stages, we would like to clarify that our framework is naturally capable of dealing with such situations, which we elaborate on in the answers to Question 1 and Question 4.

Therefore, would you mind checking our response, and is there any unclear point so that we could further clarify?

Best regards,

Authors

Dear reviewer cH3G:

We apologize for any inconvenience our request may cause during your schedule. As the rebuttal phase is drawing to a close, we would be grateful if you could review our responses. We have made every effort to address the concerns you raised thoroughly and thoughtfully. If you have any other questions, we hope you can communicate with us again. Your insights are crucial in helping us refine and improve our research.

Best regards,

Authors

Question 3: In Section 5, it's more the bijective property than the invertibility that allows for a lossless memory of the NF model. Please consider rephrasing.

Answer: Thanks for the insightful comments. We have rephrased the statements. Although an invertible function is inherently bijective, it is specifically the bijective property that guarantees the lossless memory characteristic of the NF model.

---

Question 4: In Section 5, does a large distance of encodings in the latent space of the NF (i.e. low likelihood) imply that tasks/samples/features are contradictory? I think it does not, although your proposed way to measure this might have a certain level of accuracy since the distance and the extent of being contradictory probably correlate. It would be beneficial to see how strong the correlation between distance of tasks in the latent space and "harmfulness" during learning is.

Answer: Thanks for the very constructive comments. We would like to explain it as follows:

1. As the reviewer mentioned, the distance and the extent of being contradictory probably correlate. Statistically, the method could help mitigate the undermining of malicious memory. We perform probability estimation of generated data with respect to local distribution to measure the relevance of the generated feature to current tasks. Outlier features could potentially be detrimental features. For example, we assume the NF model has memorized the mislabeled feature of label $Y$ from previous malicious tasks. While learning the correct task from data of label $Y$, the probability of memorized malicious features would be low since they are widely divergent from the correct feature with the same label $Y$.

2. As the reviewer stated in Question 6, memorized knowledge from unrelated tasks may also present a large distance of encodings in the latent space, but it may not imply contradictory. The impact of irrelevant knowledge and malicious knowledge on model training differs. Irrelevant knowledge, which does not contradict correct knowledge, can be retained during the acquisition of new tasks. For a more detailed explanation, please refer to the response provided in Question 6.

3. Following the advice, we supplement the statistics of the measured weights for malicious features, relevant features, and irrelevant features in the following table. On the EMNIST-noisy dataset in Section 4, 'malicious features' refer to the features whose label are the same with the subject features but are in the malicious clients; 'relevant features' refer to the features whose labels are the same with the subject features and are in the normal clients; 'irrelevant features' refer to the features whose labels are different from the subject features and are in the normal clients. These three kinds of features represent biased memory, benign memory, and irrelevant memory. The average probabilities of the three kinds in normal clients calculated by the NF model—in other words, the distance of encodings in the latent space of the NF model—are shown in the Table below. Relevant features present high probabilities in each client, and malicious features have low probabilities. Therefore, the negative impact of biased memory could be mitigated by assigning low weight to it. The probabilities of irrelevant features are lower than relevant features and higher than malicious features. Despite the somewhat low weight, the irrelevant features would not be forgotten by the model, as they do not conflict with the correct features, which we elaborate on in the answer of Question 6.

---

We would like to thank the reviewer for the positive and very valuable comments. Below are our responses to the comments in Weaknesses and Questions.

---

Question 1: In Section 4, it lacks to clearly support the claim that biases in the data have severe impact on the model performance in FCL. Related work already indicated that this is the case. It would be great to have some references to fully cover the claims in this section as well.

Answer: Thanks for the valuable advice.

1. As the reviewer mentioned, we concur that discussion regarding the impact of bias on the model performance in FCL, supported by references, would indeed enhance the manuscript's rigor and depth. For instance, [1] demonstrated that statistical heterogeneity in FCL results in severe performance degradation. [2] theoretically and empirically revealed that bias could propagate and be exaggerated through federated learning. [3] designed a synthetic biased dataset and found that it critically reduced the performance of continual learning methods.

2. Besides, we supplement experiments on the EMNIST-noisy dataset without malicious clients ($M=0$) to study the impact of biases. The results of the last 3 tasks in the EMNIST-noisy dataset are shown in the Table below. The dataset has an increasing number of malicious clients, denoted as $M$. The efficacy of the methods declines with more malicious clients. Furthermore, introducing bias into the dataset (from $M=0$ to $M=1$) significantly undermines the performance of both the baselines and the proposed method.

[1] Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.

[2] Hongyan Chang, Reza Shokri. Bias Propagation in Federated Learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023.

[3] Lee, Donggyu, Sangwon Jung, and Taesup Moon. Issues for Continual Learning in the Presence of Dataset Bias. AAAI Bridge Program on Continual Causality. PMLR, 2023.

---

Question 2: In Section 5, learning in feature space does not protect privacy if the model is leaked because one could still run e.g. membership inference attacks given the first feature extractor. Without any additional protection mechanism (e.g. differential privacy) privacy guarantees are not given. Please rephrase that part.

Thanks for pointing out the issue. We would like to explain it as follows:

1. Indeed, we concur that learning in feature space does not guarantee absolute privacy protection. Accordingly, we have rephrased the statement.

2. We mentioned privacy protection because our approach incorporates more considerations regarding privacy compared with existing works in the FCL domain. For instance, [1] proposed learning a global generative model for raw data to mitigate catastrophic forgetting in FCL.

3. Our method can be easily modified to provide enhanced privacy protection and avoid membership inference attacks. Instead of sharing the complete classifier among clients, we can keep the feature extrator of the classifier personalized and local for each client, only sharing the classifier head and NF model between the server and clients. By concealing the local feature extrator, which is the only model processing raw data, we can better protect client privacy. Without access to the feature extrator, membership inference attacks cannot be implemented.

[1] Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.

---