We sincerely appreciate your valuable suggestions and would like to express our gratitude.

Q1 computational complexity of the proposed method and Fisher Information Matrix (FIM) for large-scale applications.

A1 Thank you for your question! Although the FIM calculation is costly, our method circumvent this difficulty by calculating the FIM in the dual parameter space of exponential family distribution, i.e., the expectation parameter space. According to theorem 3.1, the FIM in the natural parameter space is equal to the gradient in its dual space, i.e., the expectation parameter space. Therefore, the computation cost of FIM is not high. In addition, we conducted experiments to compare the running efficiency by integrating our proposed method (MACL) with multiple CL approaches, including DER++, ER-ACE, and LODE. The training time (in minutes) for a single epoch on ImageNet-R with ViT as backbone is shown below. Our method only marginally increases the training time by approximately 56% to 64%. This indicates that our method only introduces small computation cost and is efficient in practice.

running time on ImageNet-R (minutes)

A2 Online Continual Learning (CL) focuses on learning from a data stream with unclear task boundaries. This poses the challenge of not being able to utilize task identity to calculate the FIM. Additionally, we need to maintain computational efficiency. To address these challenges, we explore calculating the FIM in an online CL setting using a moving average approach. Specifically, we approximate the diagonal Hessian matrix at each time step using the gradient outer product, i.e., $H_t \approx g_tg_t^T$. To improve training efficiency, we calculate the FIM only in the first two residual network blocks (closer to the inputs) of ResNet (which consists of four residual network blocks). This is based on existing studies that show earlier layers, which calculate the input data representations, are more sensitive to task drift. We invite you to refer to the global response Q1 and A1 for the results in online CL setting.

Q3 hyperparameter optimization.

A3 Thank you for your suggestions! We have explored automatic hyperparameter optimization to enhance the method's usability by utilizing the SMAC3 framework [1], a sample-efficient and automatic Bayesian optimization hyperparameter search strategy. The primary goal of Bayesian optimization for hyperparameter search is to optimize a black-box function. In the context of CL, the objective function is the average validation accuracy on the first three tasks, following [4]. We set the hyperparameters to be optimized, specifically the learning rate $\eta$, within the range of $[1e-6, 1e-2]$. SMAC3 then performs the hyperparameter optimization process, iteratively suggesting new configurations to evaluate and updating its model based on the results. The selection of which hyperparameter configuration to evaluate next is determined by an acquisition function, which balances exploration (trying new configurations) and exploitation (focusing on configurations that are known to perform well). Initially, we perform hyperparameter optimization on the first three tasks, then use these optimized hyperparameters for subsequent tasks, following existing CL works [4]. The performance of the searched optimal hyperparameters on TinyImageNet with a memory buffer size of 500 is shown in the following table.

Hyperparameter Optimization for $\eta$ on Tiny-ImageNet

Q4 Datasets outdated

A4 Thank you pointing out this! We conducted experiment on the recent CL datasets of ImageNet-R [2] and CUB200 [3] with pre-trained Vision Transformer (ViT), i.e., vit-base-patch16-224 as the backbone following the codebase of DER++. The results (memory size of 500) are shown in the following table.

ImageNet-R Results

CUB200 Results

Q5 Can the authors provide more insights into the computational overhead introduced by the proposed method, particularly in large-scale datasets or real-time applications? Have the authors considered any strategies for optimizing hyperparameters in a more automated fashion to improve the method's usability across different datasets and tasks? Are there any plans to extend this approach to online continual learning scenarios? If so, what challenges do the authors anticipate?

A5 Thank you for your questions! We invite you to refer to the question and answer in Q1 to Q4 and A1 to A4.

Reference:

[1] SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization, JMLR 2022

[2] The many faces of robustness: A critical analysis of out-of-distribution generalization. ICCV 2021

[3] The caltech-ucsd birds-200-2011 dataset, 2011

[4] Efficient Lifelong Learning with A-GEM, ICLR 2019

Thank you for your feedback! We really appreciate your support!

We deeply appreciate the thoughtfulness of your feedback and support!

Q1 whether the assumption of CL model parameters following a normal distribution could be valid for all models.

A1 Thank you for your question!

• The assumption that model parameters follow a Gaussian distribution is intended to balance good performance with computational efficiency. This assumption is widely adopted in Bayesian inference for neural network parameters because of its simplicity and scalability. The Gaussian distribution is computationally efficient to work with and can be easily integrated into various optimization and inference algorithms. This balance between performance and efficiency makes the Gaussian distribution a practical choice for modeling neural network parameters, particularly in complex tasks such as continual learning.

• We extend the standard normal distribution of current CL model parameters into a more general Gaussian distribution update. Here, we assume a more general Gaussian distribution to model current CL parameter distribution, $V(\theta) = \mathcal{N}(\theta|\mu_0, \Sigma_0)$, where $\mu_0$ and $\Sigma_0$ denote the mean and covariance matrix of current CL model parameters, respectively. We can obtain the following update equation:

  $\Sigma^{-1}_{i+1}= (1 - \eta)\Sigma^{-1}_i + \eta \mathbb{E} _{\theta\sim u(\theta)} [-\nabla _{\theta\theta}^2 \mathcal{L} ^{CL}(\theta) + \Sigma_0^{-1}]$

  $\mu_{i+1} = (1 - \eta)\mu_{i} + \eta \Sigma _{i+1} \mathbb{E} _{\theta \sim u(\theta)} [\nabla _{\theta} \mathcal{L}^{CL}(\theta)+ \mu_0 \Sigma _0^{-1}]$

  Compared to standard normal distribution modeling assumption. Here, the above more general Gaussian parameter distribution incorporates one additional term of $\mu_0 \Sigma_0^{-1}$ and $\Sigma_0^{-1}$ for $\mu$ and $\Sigma$ update process, respectively. This allows for greater flexibility and adaptability in the update process.

• Our proposed framework is both flexible and extendible. It can be adapted to handle more general and complex parameter distributions to capture a wider range of parameter variability. This flexibility enhances its applicability to more complex distributions.

Q2 I found it hard to easily understand the relation between the parameter sensitivity and the worst-case CL model.

A2 Parameter sensitivity implies that small changes in parameters can lead to significant drops in performance. By focusing on the worst-case scenario, we are minimizing the potential performance degradation that can occur due to parameter updates. The model is designed to be more stable against the worst possible updates in parameters. If a model can perform well under the worst-case conditions, it will naturally perform well under less extreme conditions.

This approach ensures that the model does not rely on any specific set of parameters to perform well, but rather maintains good performance across a wide range of possible parameter values to minimize performance degradation. By preparing for the worst-case, the model minimizes the variability in its performance. This means that small changes in parameters are less likely to cause significant deviations in performance, thereby reducing parameter sensitivity.

Q3 presentation could be improved.

A3 Thank you for your suggestions! We followed your advice and revised presentation in the revision.

Q4 I would expect to see forgetting reported in Table 3.

A4

Q5 Most baselines are a bit old.

A5 Thank you for pointing out this! We added one SOTA class-incremental learning (CIL) MRFA [1], one SOTA online CL baseline [2] and prompt-based baseline [3]. We invite you to refer to the global response Q1, Q3, Q4 and A1, A3, A4

Q6 motivation for focus on integrating the method with replay-based approaches.

A6 Thank you for your question! This is because memory-replay-based approaches usually achieves better performance and are more widely studied than other continual learning methods.

Q7 generalize to a long sequence of tasks?

A7 Thank you for your question, we invite you to refer to the global response Q2 and A2.

Q8 From Table 1 and Table 2, it seems that the methods improve new task learning more than reducing catastrophic forgetting.

A8 Thank you for your question! We believe that average accuracy is more easily improved because it benefits directly from the effective learning of new tasks and preservation of old task knowledge, which is often the primary focus of many continual learning methods. In contrast, reducing forgetting and improving backward transfer requires maintaining the performance of all previously learned tasks, which is a more complex and challenging problem.

Reference:

[1] Multi-layer Rehearsal Feature Augmentation for Class-Incremental Learning, ICML 2024

[2] Rethinking Momentum Knowledge Distillation in Online Continual Learning, ICML 2024

[3] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning, CVPR 2023

We would like to express our sincere gratitude for your valuable comments.

Q1 treating current continual learning objective as a whole without discussing the interplays among the different tasks or the data scarcity problem of the previous tasks. why CL specifically benefits from this algorithm.

A1 Thanks for your question!

• We would like to clarify that our notation of $L^{CL}(\theta)$ does not explicitly treat the continual leanring process as a whole. Instead, this is an abstract and unified loss notation that can represent the memory-based loss, regularization-based loss, etc. It simplifies the derivation process and guarantees the general applicability of our proposed method. Specifically, $L^{CL}(\theta) = L_{CE}(\theta) + L_{f}(\theta)$. Here, the new task loss is $L_{new} = L_{CE}(\theta)$ and the old task loss approximation is $L_{old} = L_{f}(\theta)$. Thus, it can be further formulated as $L^{CL}(\theta) = L_{new}(\theta) + L_{old}(\theta)$, which describes the interplay between new and old tasks.

• The data scarcity problem of the previous tasks is incorporated into the loss of $L_{old}(\theta)$. For example, if adopting a memory-replay-based method to mitigate forgetting, $L_{old}(\theta) = L_{(x,y) \sim M}(x, y, \theta)$, i.e., the memory-replay loss, where $M$ is a memory buffer that stores a small number of data from previous tasks. If adopting a regularization-based approach, $L_{old}(\theta)$ is a regularization term to mitigate the parameter drift from previous task parameters.

• Our "model-sensitivity-aware" loss operates on both $L_{old}(\theta)$ and $L_{new}(\theta)$ at the same time. Operating on $L_{old}(\theta)$ reduces the parameter sensitivity on old tasks to mitigate forgetting on old tasks. Operating on $L_{new}(\theta)$ reduces the parameter sensitivity on new tasks to reduce overfitting on the new tasks, thus leading to better performance on new tasks. Therefore, our method mitigates the trade-off between old task knowlege retention and new task learning by improving the performance on new and old tasks simultaneously.

Q2 what the generalization bound in theorem 4.3 implies.

A2

• Generalization bound:
  • For the first term on the right-hand side (RHS) of the generalization bound, in traditional CL, the training loss does not bound the generalization error on unseen test data. In contrast, by adopting our model sensitivity-aware CL approach, the model sensitivity-aware training loss (the first term) plus an additional complexity term (the third term) can bound the generalization error on unseen test data. Therefore, this implies that by minimizing the model sensitivity-aware loss, i.e., $\max _{\mathbb{U} \in \mathcal{U}} \mathbb{E} _{\theta \sim \mathbb{U}} L ^{CL} _{T}(\theta)$, we can indeed reduce the generalization error.
  • The third term in the generalization bound is the model complexity term, i.e., $\sqrt{\frac{\tau^2(\sqrt{q} + \sqrt{2\log n})^2 + R + 2\log(\frac{n}{\delta})}{4(n-1)}}$, represented by the KL divergence between the posterior distribution 𝑄 and the prior distribution 𝑃 and simplifid to the current form. This term quantifies the complexity of the posterior distribution relative to the prior. A higher KL divergence indicates that the posterior distribution 𝑄 deviates significantly from the prior 𝑃, implying a more complex model. This complexity term acts as a regularizer. It penalizes models that are too complex and deviate significantly from the prior, thereby encouraging simpler models that are more likely to generalize well, thereby mitigating overfitting issue.

Q3 The organization and presentation.

A3 Thank you for your suggestions! We revised the contents in the revision.

Q4 The experimental results seem not rigorously verified. As far as I know, the CIL results on CIFAR100 dataset should start with 40-50% overall accuracy but not this low in the paper.

A4 Thank you for pointing out this!

• We would like to clarify that as mentioned in lines 243-244 on page 7, the CIFAR100 dataset is divided into 10 tasks, each containing 10 classes, following the same setting as [1]. [1] reports the CIFAR100 accuracy as 37.13 for DER++ and 36.48 for ER-ACE under a memory buffer size of 500 but did not provide an error bar. Our results show DER++ at 36.37 $\pm$ 0.85 and ER-ACE at 37.05 $\pm$ 0.36, which are very similar to [1] with the same settings.

• According to [2], they split CIFAR100 into 5 disjoint tasks with each task having 20 classes, reporting the accuracy of DER++ in class incremental learning as 42.08. The performance difference is due to different task splits: we split CIFAR100 into 10 tasks, while [2] splits it into 5 tasks. We chose the more challenging 10-task scenario over the 5-task scenario.

Reference:

[1] On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning, NeurIPS 2022

[2] Loss Decoupling for Task-Agnostic Continual Learning, NeurIPS 2023

Q5 In Eq. 7, the outer minimization over is not taking any effect in the inner maximization.

A5 Thanks for your question! We would like to clarify that the inner expectation over 𝑈 is achieved by reparameterizing the Gaussian distribution as follows: $\mu + \sigma \zeta$, where $\zeta \sim \mathcal{N}(**0**, **I**)$. Therefore, the expectation for $U$ can be converted to the expectation with respect to the Gaussian distribution $\zeta$. Therefore, the outer minimization over $\theta$ still takes effect on $\theta$ after the inner max expectation. The details are illustrated as the following equation:

$\max _{\mathbb{U}\in \mathcal{U}} \mathbb{E} _{\theta \sim \mathbb{U}} L^{CL} _{T}(\theta) \approx \mathbb{E} _{\zeta \sim \mathcal{N} (**0**, **I**)} L ^{CL} _{T}[(1 - \eta)\theta _{i} + \eta \sigma _{n} ^2 [\nabla _{\theta} L ^{CL}(\theta _i)]+ \sigma_n \zeta]$

Thank you for your response. After the rebuttal, some of my concerns are solved, but not all of them. Here I would like to kindly ask for further clarification.

Q1+Q2 [Oversimplified CL Loss and Generalization Bound] The loss function we use (for optimization and theoretical analysis) always have an underlying assumption, which is the data distribution it is evaluated. CL loss is composed of two parts, as mentioned by the authors, $\mathcal{L}\_{\text{old}}(\theta)$ and $\mathcal{L}\_{\text{new}}(\theta)$, which are typically evaluated under different data distributions: replay buffer populated from the old data distribution $\mathcal{D}\_{\text{old}}$ and the current-task dataset populated from the current data distribution $\mathcal{D}\_{\text{old}}$. Hence it raises a serious question to the main theorem (Theorem 4.3): what is the data distribution defined $\mathcal{D}$ here? A trivial weighted average of $\mathcal{D}\_{\text{old}}$ and $\mathcal{D}\_{\text{new}}$ is not okay as the losses are evaluated separately. Therefore I would question the validity of the theory in this paper as it to me is serious oversimplification.

Q4. [Low Performance of CIL models] Are there any specific reasons for choosing these two papers as the baselines? Specifically, most of the existing work uses ResNet-32 as the backbone for CIFAR100 [1] and why do you use ResNet-18? iCaRL [2], a baseline from 7 years ago, can achieve 40+ accuracy on CIFAR100, with is conducted under the exact setting as you mentioned. The numbers in the Table are not the state-of-the-art.

Q5. [Broken MinMax Problem in Eq. 7] Sorry I still don't get it. In the inner maximization, the expectation over $\theta$ is evaluated under the distribution $U$, it has nothing to do with the $\theta$ you have in the outer minimization. I am okay with the inner maximization being a "sensitivity penalty", can you please elaborate more on the outer $\theta$?

Additional Questions

• Throughout the introduction of the method (Section 3.2), there is no effective reference to existing literature. Where was the concept of "Model Sensitivity" first defined and what are existing techniques? If this concept if completely new and so as the methodology of this paper, please indicate it in the main body.
• To me this method looks alike the idea of SAM (Sharpness-Aware Minimization) applied to CL [3]. Please discuss the relationship between this method and SAM-based CL methods.

References

• [1] Zhou, Da-Wei, et al. "Class-Incremental Learning: A Survey." IEEE Transactions on Pattern Analysis and Machine Intelligence (2024).
• [2] Rebuffi, Sylvestre-Alvise, et al. "icarl: Incremental classifier and representation learning." Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2017.
• [3] Mehta, Sanket Vaibhav, et al. "An empirical investigation of the role of pre-training in lifelong learning." Journal of Machine Learning Research 24.214 (2023): 1-50.

We extend our sincere appreciation for your constructive feedback!

Q1 5-datasets evaluation

A1 Thank you for your suggestions! We perform experiments on 5-datasets (memory buffer size of 500) with our method (MACL) as below.

Q2 analyse the effect of the sequence length or long sequence.

A2 We invite you to refer to the global response Q2 and A2.

Q3 Online Continual Learning setting evaluation.

A3 Thank you for your suggestions! We invite you to refer to the global response Q1 and A1.

Q4 analyse the impact of architecture, and of initialization on the method.

A4 Thank you for your question! We conducted an additional experiment using a pre-trained Vision Transformer (ViT), specifically the vit-base-patch16-224 model pre-trained on ImageNet1K, following existing CL literature settings. The results, shown in the following table for CIFAR100 with DER++ and a memory size of 500, demonstrate that using a pre-trained ViT significantly improves CL performance. In addition, integrating MACL with DER++ further enhances the CL performance with the pre-trained ViT.

Q5 analyse the efficiency of other CL methods

A5 Thank you for pointing this out! We conducted experiments to compare the running efficiency by integrating our proposed method with multiple CL approaches, including DER++, ER-ACE, and LODE. The training time (in seconds) for a single epoch on CIFAR100 is shown below. Our method increases the training time by approximately 55% to 61%.

running time on CIFAR100 (seconds)

Q6 Is modelling the current CL parameters with a standard normal distribution realistic? Isn't it too restrictive? What if we start the CL process with an already pretrained model?

A6 Thank you for your question! We acknowledge that this assumption might be restrictive in certain scenarios. When starting the CL process with an already pretrained model, the parameters may not be initially distributed according to a standard normal distribution. In such cases, our method can be adapted to account for the existing parameter distribution of the pretrained model. We initialize our approach using the pretrained model's parameter statistics to estimate a more general Gaussian distribution, ensuring a more accurate and realistic modeling process. Furthermore, we invite you to the Q7 and A7 for a more general Gaussian distribution derivation. By doing so, our method retains its effectiveness while being flexible enough to accommodate different starting conditions, whether the model begins from scratch or with pretrained parameters. This adaptability helps maintain the generalizability and flexibility of our approach across various settings.

Q7 change assumption with a more general Gaussian distribution?

A7 Thank you for your suggestion! Here, we assume a more general Gaussian distribution to model current CL parameter distribution, $V(\theta) = \mathcal{N}(\theta|\mu_0, \Sigma_0)$, where $\mu_0$ and $\Sigma_0$ denote the mean and covariance matrix of current CL model parameters, respectively. We can obtain the following update equation:

$\Sigma^{-1}_{i+1}= (1 - \eta)\Sigma^{-1}_i + \eta \mathbb{E} _{\theta\sim u(\theta)} [-\nabla _{\theta\theta}^2 \mathcal{L} ^{CL}(\theta) + \Sigma_0^{-1}]$

$\mu_{i+1} = (1 - \eta)\mu_{i} + \eta \Sigma _{i+1} \mathbb{E} _{\theta \sim u(\theta)} [\nabla _{\theta} \mathcal{L}^{CL}(\theta)+ \mu_0 \Sigma _0^{-1}]$

Compared to standard normal distribution modeling assumption. Here, the above more general Gaussian parameter distribution incorporates one additional term of $\mu_0 \Sigma_0^{-1}$ and $\Sigma_0^{-1}$ for $\mu$ and $\Sigma$ update process, respectively. This allows for greater flexibility and adaptability in the update process.

Q8 add to learning to prompt techniques for CL?

A8 Thank you for your suggestions! We conducted an experiment integrating our proposed method (MACL) with the SOTA prompt-based CL method, CODA-Prompt [1]. Our method operates on the parameters of prompt components and corresponding keys/attention vectors. The results on ImageNet-R are shown in the following table. The improvement is due to reducing the sensitivity of these parameters. This way, when the model learns new tasks, parameter changes do not significantly increase the loss. During inference, the weighted prompt remains more stable, mitigating forgetting. Additionally, our method reduces overfitting to new tasks, thus improving overall performance.

CODA Prompt Results on ImageNet-R

Reference:

[1] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning, CVPR 2023

Dear reviewer, authors have provided their rebuttal. Can you please check it and provide your response? Reviewer-author discussion period will end very soon. Thanks.

• AC