From the above mathematical results, we can confirm that the reservoir sampling allows the buffer to contain samples from previous tasks. However, as the reviewer mentioned, given a limited buffer size, the number of samples the buffer can maintain for old tasks is bounded above. In Fig. R3-3, we reported the effect of buffer size on the accuracy of the trained model for old tasks. At the end of each task, we measured the accuracy of the trained model with respect to the test data for task 1. We tested on S-CIFAR100 dataset, and considered task incremental learning (task-IL) setup.

One can observe that when the buffer size $B$ is small, the accuracy drops as the model is trained on new tasks. On the other hand, when $B \ge 100$, the task-IL accuracy for task 1 is maintained throughout the process. Note that training with our default setting $B=200$ does not hurt the accuracy for task 1 throughout the continual learning process.

[Figure R3-3. Accuracy (%) for Task 1, under the Task-IL setting on S-CIFAR100. We tested on FedGP with different buffer sizes.]

https://docs.google.com/document/d/e/2PACX-1vQfbETzt5SUSDrgcCAypbjrxTia1uBgTjNZ4_KnY6tYZ7pcpRo20SA-FiDctxgQNKUwTgP_RZSRxgdZ/pub

In Table R3-2c, we compared three different buffer updating algorithms:

• Random Sampling: randomly replaces a data point in the buffer with incoming new data
• Sliding Window Sampling: replaces the earliest data point in the buffer when new data arrives
• Reservoir Sampling (Ours): given $N$ (the number of observed samples up to now) and $B$ (the buffer size), we do the following
  • when $N \le B$, we put the current sample in the buffer
  • when $N > B$, with probability $B/N < 1$, we replace a random sample in the buffer with the current sample

Note that when the number of incoming data is $N$, those $N$ samples have the same probability of getting into the buffer, for the Reservoir Sampling method used in our paper. Thus, when Reservoir Sampling is used, the buffer contains approximately equal number of samples for each task (when each task has the same number of samples), throughout the continual learning process. This is the underlying intuition why we choose such buffer updating algorithm. To support this claim, we report the performance of different sampling methods in Table R3-2c. Here, one can confirm that our sampling method is outperforming other sampling methods.

[Table R3-2c. Effect of different buffer updating algorithms on the accuracy (%) of FedGP, tested on CIFAR100]

The below figure compares the sample distribution across different tasks, for different buffer updating algorithms (uniform-random sampling, reservoir sampling and sliding window sampling). This clearly shows the difference of buffer updating algorithms, which guided to the different performances reported in Table R3-2c.

https://docs.google.com/document/d/e/2PACX-1vRzMZ8Ha3DaG79IhAR6lLEFj7UYjQt-W3pJkBvgkPHLtSIv5KkCE7DcAUIBf7VuxnUVM3l78oduC54N/pub

All in all, the reservoir sampling method used in FedGP allows us to have balanced sample distribution across different tasks, thus allowing us to mitigate the catastrophic forgetting and to improve the accuracy in the continual federated learning setting.

In the revised manuscript, we will include our new results as well as the discussion we made during the rebuttal.

Q3: I am skeptical about whether the use of a buffer can solve the problem of catastrophic forgetting. Since the buffer size is limited, one is unable to store the entire past information in it, and has to discard stale information. According to Algorithm 4, when the buffer is full the algorithm randomly selects an old entry in the buffer and replace it with a new data point. How can this avoid discarding useful information, e.g., the data point stored in previous rounds?

Thanks for the great question. The reviewer is correct that a finite buffer cannot maintain the entire history of data regardless of buffer update algorithm. Instead, our goal is to continuously maintain the buffer so that it represents all the tasks that have been seen in the past. Under the Reservoir Sampling, it is easy to prove that after seeing $N$ samples for any $N$, the samples stored in the buffer are $B$ samples chosen uniformly at random. Before proving this property, let us recall the definition of Reservoir Sampling: for a given $N$ (the number of observed samples up to now) and $B$ (the buffer size),

• when $N \le B$, we put the current sample in the buffer
• when $N > B$, with probability $B/N < 1$, we replace a random sample in the buffer with the current sample

We now prove that for reservoir sampling, the probability of a sample contained in the buffer is $\frac{B}{N}$. We prove this by induction; suppose this statement holds when $N-1$ samples are observed, and we show that this holds when one more sample is observed. Note that the probability of a sample contained in the buffer can be computed as $\frac{B}{N-1} * (1-\frac{B}{N}*\frac{1}{B})=\frac{B}{N}$, where

• $\frac{B}{N-1}$ is the probability of the sample initially contained in the buffer;
• $(1-\frac{B}{N}*\frac{1}{B})$ is the probability of a sample not being kicked out of the buffer;
• $\frac{B}{N}*\frac{1}{B}$ is the probability of a sample being kicked out of the buffer

We will include these details in the revised manuscript.

We thank the Reviewer for the detailed review and constructive suggestions. We appreciate your acknowledgement that the experiment part is comprehensive, solid and well-written. Our responses are detailed below.

Q1: The novelty of the proposed method is questionable. The key steps of the proposed FedGP algorithm, including using buffers and performing gradient projection, can all be found in previous works [1]. The only major difference, as far as I can tell, is to extend the centralized setup in [1] to a decentralized/federated setup.

We consider our main contribution lies in the development and presentation of a simple yet highly effective algorithmic solution for continual federated learning. We have devoted considerable effort to an extensive empirical validation of our approach. Especially, to support our claim, we added new extensive experimental results in the revised manuscript. The detailed explanations can be found in a related question [R1-Q1] by Reviewer MNXy.

[1] Efficient lifelong learning with A-GEM.

Q2: The algorithm design of FedGP is mostly heuristic and not well motivated. For example, the authors do not rigorously explain why the gradient projection operation and the buffer updating algorithm will solve the catastrophic forgetting problem and fit into the FL setting. A theoretical analysis and more ablations on algorithm design will help to show the effectiveness of FedGP.

Thanks for the sharp question. To address the reviewer's concern, below we added extensive experimental results for ablations on our algorithm design, which is decomposed into (1) gradient manipulation algorithm, and (2) buffer updating algorithm.

First, we considered different ways of manipulating the gradient $g$, given the reference gradient $g_{\text{ref}}$. In the below table, we compared four different methods of updating $g$:

• Average: define $g \leftarrow (g+g_{\text{ref}})/2$.
• Rotate : rotate $g$ towards $g_{\text{ref}}$ while keeping the magnitude.
• Project: project $g$ to the space that is orthogonal to $g_{\text{ref}}$.
• Project & Scale: apply Project and then scale up the vector such that the magnitude is identical to the original $g$

Recall that our FedGP applies Project method only when the angle between $g$ and $g_\text{ref}$ is larger than 90 degree, i.e., when the reference gradient $g_\text{ref}$ (measured for the previous tasks) and the gradient $g$ (measured for the current task) conflicts to each other. Our intuition for such choice is, it is better to manipulate $g$ if the direction favorable for current task is conflicting with the direction favorable for previous tasks. To support that this choice is meaningful, we compared two ways of deciding when we manipulate the gradients, denoted below:

• (>90): update the gradient $g$ only when the angle($g,g_\text{ref}$)>90
• (Always): update the gradient $g$ always

We compared the performances of above choices in Table [R3-2a], for S-CIFAR100 dataset. One can confirm that our FedGP (denoted by Project (>90) in the table) far outperforms all other combinations, showing that our design (doing projection for conflicting case only) is the right choice. If we check each component (Project and (>90)) independently, one can check that choosing Project outperforms Average, Rotate and Project & Scale in most cases, and choosing (>90) outperforms Always in all cases.

[Table R3-2a. Effect of different gradient manipulation method on the accuracy (%) of FedGP, tested on S-CIFAR100]

We also tested whether doing the projection is helpful in all cases when $\text{angle}(g, g_\text{ref}) > 90$. We considered applying the projection for $p\%$ of the cases having $\text{angle}(g, g_\text{ref}) > 90$, for $p=10, 50, 80$ and $100$. Note that $p=100\%$ case reduces to our FedGP.

Table R3-2b shows the effect of projection rate $p\%$ on the accuracy, tested on S-CIFAR100 dataset. In both class-IL and task-IL settings, increasing $p$ always improves the accuracy of the FedGP method. This supports that our way of projection is suitable for the continual federated learning setup.

[Table R3-2b. Effect of projection rate $p\%$ on the accuracy (%) of FedGP, tested on S-CIFAR100]

Q5: The paper focus on an interesting continual federated learning problem and has several issues that can be improved: The proposed FedGP method suffer from more communication costs than comparison methods. Moreover, additional gradients need to be uploaded to the server which increases the risk of the local data leakage.

We thank the reviewer for pointing out the possible limitations of our method. Regarding the communication cost, it is true that vanilla FedGP has 2x communication cost compared with baselines. However, we consider a variant of FedGP which updates the model and buffer gradient less frequently (i.e., reduce the communication rounds for each task by half or more), which has reduced communication than the vanilla FedGP.

During the rebuttal period, we ran extensive new experiments which explore the accuracy of FedGP methods for different communication frequencies. Table R2-5 below reports the performance for different datasets, when the communication overhead is set to 2x, 1x, 0.5x and 0.2x. Values in brackets indicate differences from the FL baseline.

First, in most cases (except S-CIFAR10 Class-IL setup), FedGP with equalized (1x) communication overhead is outperforming FL. In addition, for most of tested datasets including R-MNIST, P-MNIST and S-CIFAR100, FedGP outperforms FL with at most 0.5x communication overhead. This means that FedGP enjoys a higher performance with less communication, in the standard benchmark datasets for continual federated learning.

In the revised manuscript, we will include Table R2-5 our discussions written above.

[Table R2-5. Effect of communication on the accuracy (%) performance, with values in brackets indicating differences from the FL baseline]

Here's the updated table with the differences compared to the first row (FL) included:

Regarding the data leakage issue, we are running experiments measuring the privacy leakage of FedGP and other baselines. We will report the result as soon as we get it.

Q3: The adopted comparison methods seem to be outdated, it is better to adopted the SOTA methods to further validate the effectiveness of the proposed FedGP method.

As per the reviewer's suggestion, we compared our method with SOTA paper [1] proposing Federated Orthogonal Training (FOT) algorithm for continual federated learning.

In FOT, at the end of each task, each local client transmits the activation vectors information (computed for local data points) of each local model to the server. Then, the server computes the subspace $S$ spanned by the aggregated activations. This subspace is used during the global model update process; the gradient $g$ is updated in the direction that is orthogonal to the subspace $S$. Note that FOT has several advantages compared with existing baselines; the privacy leakage is mitigated, the communication cost is reduced, and the solution has theoretical guarantees. We very much appreciate their work.

While both FOT and our FedGP project the gradients $g$ on the subspace $S$ specified by previous tasks, they have two main differences. First, the subspace $S$ is defined in a different manner: FOT relies on the representations of local model activations to define $S$. FedGP, on the other hand, relies on the gradient of model computed on its local buffer data. Second, FOT projects the gradient computed at the server side, while FedGP projects the gradient computed at each local client.

The below table compares the accuracy of FL, FOT and FedGP (with buffer size 200) for P-MNIST and S-CIFAR100 under the task incremental learning scenario, consistent with the paper's benchmarks. Assuming that the local buffer is available, FedGP outperforms FOT.

We will include this in the revised manuscript.

[Table R2-3. Comparative accuracy (%) performance analysis of FL, FOT, and FL+FedGP]

[1] "Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning"

Q4: The literature[1] also addressed the forgetting by using gradient projection, which has a strong connection with the proposed method. It is better to taken as a comparison method.

We thank the Reviewer for bringing this work [1] to our attention. As the reviewer mentioned, both our method (FedGP) and the gradient projection memory (GPM) method suggested in [1] consider gradient projection for mitigating the forgetting issue.

However, FedGP (ours) and GPM (proposed in [1]) have two clear differences.

First, the reference vector used for the projection is different. In our FedGP, the reference vector is the gradient $g_{\text{ref}}$ of the global model computed by the buffer data stored at clients. In contrast, in [1], the network representation (activations) approximated by top singular vectors are used as the reference vector, which is measured when each task ends.

Second, GPM requires task boundaries, while FedGP does not. Thus, FedGP can be applied in more realistic scenarios where the task boundaries are not given.

Table R2-4 compares the performance of FedGP and GPM (applied in federated learning) for various datasets. We compared the two methods assuming that task boundaries are given. Note that our method does not use this information, while GPM does. Still, we observe that our FedGP outperforms GPM. The potential reasons could be (1) FedGP utilizes buffer gradients which could provide more information than the top singular vectors derived from the activations in GPM. (2) FedGP updates its reference gradients more often, like after every communication round, while GPM only updates after finishing a task.

[1] “Gradient projection memory for continual learning.”

[Table R2-4. Comparative accuracy (%) performance analysis of FL, FL+GPM, and FL+FedGP across various datasets]

We will include this in the revised manuscript.

We thank Reviewer Nmd1 for the detailed review and very helpful suggestions. We thank the Reviewer for appreciating the proposed projection method, extensive experimental results, and our efforts in making a well-written paper. Our responses are detailed below.

Q1: The gradient projection that conflict with the previous task update direction would aggravate forgetting, while directly drop it would impair the current task learning. The proposed method can be addressed the forgetting to some extent while ignoring its impact on the current task.

We thank the Reviewer for pointing this out. To address your concern, below we provided additional experimental results on the performance measured for the current task. The below plot shows the Class-IL accuracy of FedGP (with buffer size 200) and FL for S-CIFAR100, where the total number of tasks is set to 10. During the continual learning process, we measured the accuracy of each model for the current task. One can confirm that using FedGP does not hurt the current task accuracy, compared with FL. Note that this shows that FedGP does not impair the performance of the current task, while also alleviating the forgetting in upcoming rounds.

We will include this result and discussion in the revised manuscript.

[Figure R2-1. Class-IL Accuracy (%) of current task for FL and FedGP on the S-CIFAR100]

https://docs.google.com/document/d/e/2PACX-1vSqmIgnlzdcX2Pvvjw3acXsM1tINU-Jj385cYadSk1jUI9RDm6jr7hZQ3mtomf3Mxo6nDYBq4ZR9Lre/pub

Q2: In Table 3, the two times communication get an inferior performance than the equalized communication with FedGP in P-MNIST, can you give an explanation?

We thank the Reviewer for carefully reading our work. After reading our result again, we noticed that the performances of two methods (2x communication stragety and equalized communication strategy) in P-MNIST are quite close to each other, and they are within the errors.

During the rebuttal period, we ran additional experiments; instead of the initial 5 runs (which we reported in the submitted manuscript), we did 20 runs per method, each with a unique random seed, the result of which is given below.

[Table R2-2. Effect of communication on accuracy (%) performance averaged across 20 runs, for P-MNIST]

As in Table R2-2, 2x communication strategy has a slightly better performance than equalized communication strategy, when evaluated across 20 runs. Note that this result is consistent with our finding for other datasets (R-MNIST, S-CIFAR10 and S-CIFAR100) we reported in Table 3 of the submitted manuscript.

Based on this observation, we will revise the result for P-MNIST in the updated manuscript.

We thank Reviewer MNXy for the detailed review. We are happy to see the acknowledgement of the paper's clarity and thorough experiments. Please find our answers to comments and questions as follows.

Q1: Observing FedGP, it seems to be a federated take on A-GEM, where g_ref is replaced by a federated aggregation of local buffer gradients. Although FedGP shows promising results when combined with other methods, I'm still pondering its overall novelty and contribution.

Thanks for your comments.

We consider our main contribution lies in the development and presentation of a simple yet highly effective algorithmic solution for continual federated learning. Our FedGP is a simple variant combining existing methods, but it highly outperforms baselines in various scenarios. To show this, we have devoted considerable effort to an extensive empirical validation of our approach, including

• image classification
• language processing
• object detection data from the CARLA simulator

This empirical justification is a critical part of our contribution, demonstrating the general applicability of our algorithm.

To further strengthen our claim, we added extensive new results during the rebuttal period. To be specific, we conducted experiments exploring the effect of

• gradient manipulation (projection, rotation, averaging) methods -- result is in R3-Q2
• buffer data sampling methods (reservior, random, sliding-window)-- result is in R3-Q2
• buffer size -- result is in R3-Q3
• communication frequency -- result is in R2-Q5
• the reference gradient used for projection (by comparing with related baselines [1], [2]) -- result is in R2-Q3 & R2-Q4

[1] Saha, Gobinda, Isha Garg, and Kaushik Roy. "Gradient projection memory for continual learning." arXiv preprint arXiv:2103.09762 (2021). [2] Bakman, Yavuz Faruk, et al. "Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning." arXiv preprint arXiv:2309.01289 (2023).

Q2: In the experimental section, Table 1 presents the performance of all baseline methods. It's noticeable that, in the without FedGP column, even though CFL methods are designed for continual learning challenges, some, especially FL+iCaRL/L2P, performed better than CFL methods. Can the authors clarify the reasons behind these results?

We thank the reviewer for the sharp question. First, it is worth nothing that FL+CL methods could still outperform some CFL methods under certain conditons, and the similar findings have been made in the previous work too. For example, according to a recent work [1], FL+CL methods (FL+EWC and FL+ER) outperforms CFL methods (GLFC and FedCIL) for S-CIFAR100 dataset.

Second, we note that FL+L2P's high performance is due to a different reason -- it utilizes a pretranied Vision Transformer (ViT), which helps mitigate the catastrophic forgetting. This is why we wrote the numbers in gray with a caveat in the caption, but we will further clarify this in the revision.

Third, regarding a CFL method called CFeD [2], it is important to note that its performance is heavily dependent on the choice of surrogate data (we used CIFAR for our experiments), and no direct comparison with state-of-the-art FL+CL methods (e.g., iCaRL) has been reported in [2].

In the revised manuscript, we will include this discussion.

[1] Y. F. Bakman et al., "Federated Orthogonal Training: Mitigating Global Catastrophic Forgetting in Continual Federated Learning", arXiv 2023 [2] Ma, Yuhang, et al. "Continual federated learning based on knowledge distillation." Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. Vol. 3. 2022.