Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective

We thank the reviewer for providing the valuable review. Please note that all our new experiment results (i.e., the tables we quote below) and our codes can be accessed via the link https://anonymous.4open.science/r/repo-c14014

Q: Are there any risks of overfitting in sequential rehearsal?

A: We appreciate the reviewer’s question. During training, the risk of overfitting indeed exists due to the limited size of memory data. To reduce the overfitting risk, our experiment has adopted a more conservative learning rate if a ‘dissimilar task’ is learned separately, as presented in Table 3 of the paper.

Q: About the sequential rehearsal, did you try training with the rehearsal samples for several rounds? For example, in figure 1, after training $M_{t,t-1}$, go back to train on $M_{t,1}$. What would it be?

A: We appreciate the reviewer’s insightful question. Despite the limited time, we made every effort to expand our experiments to train dissimilar tasks for multiple rounds, as presented in Table R3 in the anonymous link. The Averaged Final Accuracy(Acc) of multiple-round training is not changed significantly, while the Forgetting(Fgt) is slightly improved compared to single-round training. It could be because the same memory data is repeatedly learned when sequential training is conducted over multiple rounds, which may help recover more previous knowledge while introducing an unavoidable risk of overfitting.

Please note that all our new experiment results (i.e., the tables we quote below) and our codes can be accessed via the link https://anonymous.4open.science/r/repo-c14014

==Questions about Experiments==

Effectiveness of the method under label corruption: We conduct experiments under label corruption (see Table R2 in the anonymous link). Under varying levels of label corruption, our hybrid rehearsal also outperforms concurrent rehearsal, demonstrating the effectiveness of our method under label corruption.

Experiments on Tiny-Imagenet200: Results are in Table R1 via the anonymous link. For this more complex dataset, hybrid rehearsal substantially outperforms concurrent replay. On the original dataset, hybrid rehearsal improves accuracy by 2.19% and reduces forgetting by 3.66%. On the corrupted dataset, it improves accuracy by 2.38% and reduces forgetting by 13.32%.

Add std in Table 2: We marked standard deviation in Table R4 via the anonymous link. As can be observed, the improvement of most results are well beyond the error bars.

Impact of number of memory splits and ordering of dissimilar samples: Thanks for the suggestion. We are currently exploring these issues with full efforts.

Improvement of CIFAR-100 is marginal compared to MNIST and CIFAR-10: This may not be due to the more complex dataset, as our new experiment on Tiny-Imagenet200 shows significant improvement. It could be because the random selection of classes in our experiments did not have high dissimilarity among tasks.

Other questions: In our experiments, each sample is used the same number of times in both hybrid and concurrent rehearsal for a fair comparison. The unit of y-axis is a scalar of 1.

==Questions about Other Issues==

Whether objective in L186 is equivalent to $\text{argmin} (Xw - y )^2$ plus some regularization: The optimization objective in Line 186 comes from the convergence point of SGD on the linear model with the starting point of $w_{t-1}$ without any regularization [2,3,5]. This optimization is not equivalent to $\text{argmin}_w (X^\top w-y)^2+\lambda \|w\|$ because we force $X^\top w-y=0$ (Line 188) while $\text{argmin}_w (X^\top w-y)^2+\lambda \|w\|$ usually leads to a non-zero $(X^\top w-y)^2$ due to the need of balance between $(X^\top w-y)^2$ and $\|w\|$.

Extension of linear model to more complex data: A natural next step is to extend our analysis to neural networks in the NTK regime, which is effectively a linearized model. Further, our analysis of rehearsal-based CL can also integrate with recent advances in analyzing over-parameterized neural networks and attention models, to study CL under these more complex models.

Comparison to multitask learning: Previous studies [1,4] have shown that conventional concurrent rehearsal outperforms multitask learning. We expect our method to outperform the multitask learning baseline, as it subsumes concurrent replay as a special case.

Intuition about coefficients in Theorem 5.1: The coefficients in Theorem 5.1, which are given in (32) and (35) in appendix, suggest the following intuitions about how task ordering impacts forgetting. When memory size $M$ is small, introducing dissimilar tasks early reduces forgetting by encouraging broader feature exploration, aligning with [3] (which has $M=0$). As $M$ increases, delaying dissimilar tasks helps, since early introduction leads to frequent rehearsal, and their conflicting nature can disrupt learning of later tasks.

L 223 to 237: $M$ and $p$ affect the theorem via the coefficients $c_i$, $c_{ijk}$, $d_{0T}$ and $d_{ijkT}$ (see Proposition B.2 in appendix).

Similarity of sequential rehearsal with Curriculum Learning: Although they appear similar, their underlying training objectives differ significantly. In curriculum learning, the model is presented with data in a structured progression, typically organized by increasing difficulty, with the goal of optimizing an overall learning performance. In contrast, sequential rehearsal involves data drawn from different (possibly conflicting) tasks, where the model is trained by rehearsing over these tasks with a goal of mitigating forgetting. The forgetting issue is not addressed in curriculum learning.

Thank you for your insightful comments. We hope our responses addressed your concerns and would greatly appreciate your kind consideration in increasing your score.

Reference:

[1] Goodfellow, et al. "An empirical investigation of catastrophic forgetting in gradient-based neural networks." arXiv.

[2] Gunasekar, et al. "Characterizing Implicit Bias in Terms of Optimization Geometry." ICML 2018.

[3] Lin, et al. "Theory on forgetting and generalization of continual learning." ICML 2023.

[4] Wu, Zihao, et al. "Is multi-task learning an upper bound for continual learning?" ICASSP 2023

[5] Zhang, et al. "Understanding deep learning requires rethinking generalization." arXiv 2016.

We thank the reviewer for providing the valuable review. Please note that all our new experiment results (i.e., the tables we quote below) and our codes (including code for Fig. 2) can be accessed via the link https://anonymous.4open.science/r/repo-c14014

Q: Clarify the choice of $\mathcal{L}_i(w)$, and justify the performance metric in the overparameterized setting

A: We first clarify that $\mathcal{L}_i(w) = (w -w^*_i)^2$ is equivalent to the test error $\mathbb{E}[y - Xw]^2$. To see this, we derive $\mathbb{E}[y - Xw]^2 = \mathbb{E}[X(w -w^*_i)]^2 + \sigma^2= \mathbb{E}[(w -w^*_i)^\top X^\top X (w -w^*_i) ] + \sigma^2 = (w -w^*_i)^2+ \sigma^2$, where the last equality follows because $X$ has i.i.d. standard Gaussian entries, and $\sigma$ is noise level. Such a loss has been commonly adopted in recent theoretical studies of CL [1,2].

The example given by the reviewer is not valid, because the loss function (i.e., the test error) needs to take the expected value over the distribution of $X$, not for a specific value of $X$.

Further note that the model parameter $w$ depends on input data $X$, because the model is trained based on data. Thus, the performance of forgetting and generalization defined in (4) and (5) is evaluated using the expected value of model errors, which reflects the overall performance across a set of inputs.

Q: Provide reference for the statement "the convergence point of SGD for MSE is the feasible point closest to the initial point in $l_2$-norm, i.e., minimum-norm solution."

A: In [3,4], it has been mathematically shown that in overparameterized linear models, SGD/GD converges to the minimum-norm solution, i.e., the feasible point closest to the initial point with respect to $l_2$-norm. Further, such a property has been widely used to simplify the theoretical model in CL [1,2].

Q Numerical results are within error margin?

A: In Table 1 of the paper, the improvement of both generalization and forgetting on corrupted datasets (with higher level of task dissimilarity) is substantial and far more outside the error, which validates our theoretical observation. The improvement is marginal on original datasets(Split-MNIST, Split-CIFAR-10, Split-CIFAR-100) because their task dissimilarity level is not significant.

We further conducted new experiments on Tiny-Imagenet200 (see Table R1 via the anonymous link). For this more complex dataset, where task similarity is much higher, on the original dataset, hybrid rehearsal improves the Averaged Final Accuracy by 2.19% and reduces forgetting by 3.66%. On the corrupted dataset, it improves the Averaged Final Accuracy by 2.38% and reduces forgetting by 13.32%. These results clearly demonstrate the benefits of our hybrid rehearsal.

Q: Will resampled input-output pairs in memory alter $w_i^*$ that perfectly solves the corresponding task?

A: We clarify that in this paper, ​$w_i^*$ represents the ground-truth model parameters, which are fixed for each task. Then based on such a ground-truth $w_i^*$, data are generated by $Y_i=X^\top_iw^∗_i+z_i$. Clearly, resampling of input-output pairs will not change ground-truth parameters.

Q: Explain influence of $\tau$. Why was it not tested in your experiments? If only one task is treated as dissimilar, doesn’t that bypass the role of $\tau$?

A: Thank you for highlighting this point. Our experiment is a simplified implementation of Algorithm 1, where at most one task is designated as the ‘dissimilar task’ for sequential rehearsal. $\tau$ still serves the role of threshold as follows. If multiple tasks have scores below $\tau$, then the most dissimilar task is set for sequential rehearsal. Otherwise, if all tasks have positive scores, then no task is chosen for sequential rehearsal. Note that this is a reasonable experiment setup in CL [5].

Further note on choice of $\tau$: Since the task similarity score is defined to be cosine similarity between gradient of each previous task and concurrent task, positive score indicates their alignment, and negative score indicates their confliction. Hence, in our experiment, we set threshold $\tau$ to be $0$ naturally. In practice, $\tau$ can be set negative to ensure the task dissimilarity is high enough.

Thank you again for your insightful comments. We hope our responses addressed your concerns and would greatly appreciate your kind consideration in increasing your score.

References:

[1] Evron, et al. "How catastrophic can catastrophic forgetting be in linear regression?." COLT 2022.

[2] Lin, et al. "Theory on forgetting and generalization of continual learning." ICML 2023.

[3] Gunasekar, et al. "Characterizing Implicit Bias in Terms of Optimization Geometry." ICML 2018.

[4] Zhang, et al. "Understanding deep learning requires rethinking generalization." arXiv 2016.

[5] Lin, et al. "Trgp: Trust region gradient projection for continual learning." ICLR 2022.

We thank the reviewer for providing the valuable review. Please note that all our new experiment results (i.e., the tables we quote below) and our codes can be accessed via the link https://anonymous.4open.science/r/repo-c14014

Q1: Results with more / larger datasets (e.g. (mini)ImageNet), and under different settings could have strengthened the paper further.

A1: Thank you for your constructive suggestion. Despite the limited time, we made every effort to expand our experiments to Tiny-Imagenet200, as presented in Table R1 via the anonymous link. Our experiments on Tiny-ImageNet demonstrate that hybrid rehearsal achieves a substantial improvement over conventional concurrent rehearsal. On the original dataset, hybrid rehearsal improves the Averaged Final Accuracy by 2.19% and reduces forgetting by 3.66%. On the corrupted dataset, it improves the Averaged Final Accuracy by 2.38% and reduces forgetting by 13.32%, which also aligns with our theoretical observation that benefits of our hybrid rehearsal increase as task dissimilarity rises. Furthermore, we observe that the advantage of our method is even more obvious in Tiny-Imagenet200 compared to MNIST, CIFAR-10 and CIFAR-100. This is because Tiny-Imagenet200 is a more complex dataset and exhibits a higher level of task dissimilarity, and hence benefits of our hybrid rehearsal are more salient.

Q2. I doubt whether the observed differences, which have only been shown in a task-incremental setting, are worth the extra complexity in a practical setting, and concurrent rehearsal may remain the go-to solution.

A2: We sincerely appreciate the reviewer’s insightful observation. We agree that hybrid rehearsal can introduce additional complexity in practical deployments, and that conventional concurrent rehearsal may offer greater simplicity and convenience. We also appreciate the reviewer’s recognition that one goal of our paper is to highlight, from a scientific perspective, that concurrent rehearsal, while practical, can be suboptimal in terms of performance, as we have demonstrated in task-incremental settings. Our findings suggest that alternative strategies, such as hybrid rehearsal, warrant further exploration for their potential benefits.

We thank the reviewer again for your inspiring comments. We hope that our responses resolved your concerns. If so, we wonder if the reviewer could kindly consider to increase your score. Certainly, we are more than happy to answer your further questions.