Make Continual Learning Stronger via C-Flat

Weakness 1: Improve INTRODUCTION.

A: We have done this for the Introduction section to highlight contributions, including highlighting our CL-friendly optimizers, and also discussing how flatness-related works in current CL (flatness in some data/bases/projection, etc).

Weakness 2: The relationship between low curvature and less forgetting.

A: Exactly, one of our contributions is to prove the positive effect of low curvature on overcoming forgetting. Intuitively, we visualized the change in loss and forgetting of old tasks during CL. Figure R1 in one page shows the lower loss or less forgetting (red line) for old tasks during CL. This is an enlightening finding.

Weakness 3: Discuss the practicality.

A: To discuss practicality better, we provided a tier guideline, which categorizes C-Flat into L1 to L3 levels, as shown below,

As the table above shows, L1 denotes the low-speed version of C-Flat, with a slightly lower speed than SAM and the best performance; L2 follows next; L3 denotes the high-speed version of C-Flat, with a faster speed than SGD and a performance close to L2. We have updated this in the revision.

Weakness 4: Fix misc.

A: We have fixed them in our manuscript, including citations, minor typos, etc.

Question 1: The robustness of C-Flat on larger domain gap.

A: As you mentioned, the performance in the Class IL scenario has already demonstrated the robustness of C-Flat given the fairly complex domain shifts. However, we would still be happy to conduct a further test on C-Flat to ensure its reliability. As shown in Table below, we trained on ImageNet21K and then evaluated CL performance on IN-R/ObjNet. Unlike CIFAR, both datasets are acknowledged to have a large domain gap with ImageNet. The Table below shows that C-Flat maintains stability in more difficult cases.

Thank you for your valuable comments and for acknowledging our work. We commit to releasing the code to the CL community.

We appreciate your time and effort in reviewing our work! Thanks!

Weakness 1: Overly strong claims.

A: We appreciate your suggestion! We have revised the manuscript in line 96 to temper these statements.

Weakness 2: More clear structure for the method.

A: We did shorten that part a lot into one paragraph than a better math environment due to the page limit. We have rewritten that part with highlighted 'assumption' and 'condition' and attached a detailed proof in the appendix. Many thanks for the patient explanation.

Weakness 3: Highlight the context of Eq.10

A: Thanks for pointing it out. We intended to show some mathematical properties, i.e. the connection of C-Flat to Hessian matrix. Indeed, it does look disconnected. We have added a highlighted "Upper Bound" at the beginning of line 141 and some context about this property at the end of line 144 to make it clear.

Question 1: Improve readability and clarity.

A: We have polished the manuscript again to improve readability and clarity, and fixed minor errors, like errors in line 38,40 and 98.

Question 2: Report average boost.

A: We have updated maximum boost/average boost in Table 1 of the main paper as below,

Limitation 1: Discuss limitations.

A: C-Flat potentially has the following limitations. First, all experiments have not yet been validated on pre-training model (PTM)-based CL method. In the era of PTM, exploring the collaborative mechanisms between the proposed method and PTM is essential for advancing CL, which can be a promising future work. Second, C-Flat is under-explored in the transformer-based model, and this architecture is prevalent in the foundation model, which is remaining for future work. As above, we will update this discussion in the revision.

NOTE: Table R1-R4 and Figure R1 are attached to the one-page PDF.

Weakness 1: Clarification of contributions.

A: For the suggested works, they could further validate the importance of C-Flat on a general and stronger flatness-aware CL optimizer, we have cited them and the detailed discussion can be seen in our answer to Weakness 2.

However, all these works are focused on the zeroth order sharpness, i.e. SAM for CL [5], and the improved works are also designed for specific kind of CL approaches like GPM method [2], memory based method [3] or certain CL scenario [4], while wide local minima in [1] is another concept about the model parameter's similarity to the last task.

As a comparison, 1) we proposed a CL optimizer beyond zeroth order sharpness; 2) C-Flat can be generally applied to any CL methods including the aforementioned ones; 3) as can be seen in our reply to Weakness 4, thanks to the faster convergence speed, we still achieve optimal result with a few overhead compared with other SAM optimizer by performing C-Flat in periodical iterations.

Moreover, for clear contributions, you also can refer to the Strengths 1 of Reviewer p3Vh; Strengths 1,2 of Reviewer Tb4q and Strengths 1,2 of Reviewer 475M.

Weakness 2: Improve related work.

A: We have cited and discussed these works in Section 2 as follows:

Wide local minima was considered an important regularization in CL to enforce the similarity of important parameters learned from past tasks [1]. Sharpness-aware seeking for loss landscape flat minima starts to gain more attention in CL, especially SAM based zeroth order sharpness is well discussed. An investigation [5] proves SAM can help with addressing forgetting in CL, and [4] proposed a combined SAM for few shot CL.

SAM is also used for boosting the performance of specific methods like DFGP [2] and FS-DGPM [9] designed for GPM. SAM-CL [3] series with loss term gradient alignment for memory-based CL. These efforts kicked off the study of flat minima in CL, however, zeroth-order sharpness may not be enough for flatter optimal [62]. Thus, flatness with a global optima and universal CL framework is further studied.

As for the connection and difference of C-Flat with various existing SAM methods, PLEASE SEE the beginning of Section 3 Method, where we have a very detailed theoretical explanation and analysis.

Weakness 3: Comparison with related work [1,2,3].

A: The discussion is as follows,

i) Note that C-Flat is an optimizer-level approach, thus, the performance gains on different CL-friendly optimizers should be compared rather than different CL methods. We have already done this in the initial manuscript, please see subsection 4.5/Table 2/Figure 5a/5b. And we further compared C-Flat with more sharpness-aware method, please see Table R4 and our response to Question 4 of Reviewer p3Vh.

ii) Note that the flatness function used in DFGP is rooted in FS-GPM. We have demonstrated in Table 2 that C-Flat outperforms this function of FS-GPM.

Therefore, the comparison with some related work you mentioned is out of the scope of this work. However, we still performed evaluations on CPR even if it was not a baseline against which we should compare. C-Flat still presents tangible gains, the results as below,

Weakness 4: Concern on computational cost.

Actually, C-Flat excels in seeking well-generalized flat minima, which is demonstrated by the outstanding convergence speed (proved this in Figure 5a, and also see comments of Reviewer 475M). Second, C-Flat does not have to perform in every iteration, this means that repeated propagation can be significantly less. This suggests that C-Flat requires only a few epochs or iterations to converge to global optimum, like 50% of iterations or 60% of epochs (best result).

Thus, in practice, we do not need to maintain the same settings as the baseline, which significantly enhances the practicality. Also, note that C-Flat is even faster than SGD in some cases while maintaining high performance, this is surely what the CL community expects. Thus, benefiting from the good optimization nature of C-Flat, this concern is trivial.

Moreover, for easy use of C-Flat, we provided a tier guideline which categorizes C-Flat into L1 to L3 level. PLEASE SEE Weakness 3 of Reviewer 475M and our response to it.

Weakness 5: Slight performance gains in Table 2.

A: i) Since the FS term in FS-GPM already visits the typical zeroth-order sharpness, at this point, C-Flat partly reconfigures FS-GPM via operate the curvature (Eq.3 and Eq.4). This implies that the gains of C-Flat are founded on the top of the typical sharpness optimization used. Such an experimental setup was intended to reaffirm the versatility of C-Flat. Hence, merely noting that the boost in Table 2 is slight, we do not think it is justified. ii) Since we set a fixed random seed during CL, the performance gain of C-Flat is fairly solid. This is rare enough for an optimizer-level effort. PLEASE SEE our response to Weakness 2 of Reviewer p3Vh and Table R1 about stability.

Question 1: About subtitle Gradient-based solution.

A: Gradient-based solutions are a main group in CL, including shaping loss landscape, tempering the tug-of-war of gradient, and other learning dynamics [5, 20]. C-Flat seeks for stable CL by tuning the gradient to flat minima, which is quite different from most gradient-based CL methods. Therefore, we had this paragraph for comparison. We have added more context in line 80 to support the claim that gradient-based solutions focus on dynamic learning to overcome forgetting.

Question 2: Smooth loss curves after 80th epoch.

A: The decay_steps was set to 80 for all methods, which indicates that the learning rate decay was performed after the 80th epoch. This caused a smooth loss curve.

Dear Reviewer PuVp,

Thanks for your careful comments and your time for our work. We have revised our paper and added the discussion and experiments concerning

• better clarification and tempered statements on contributions in the revised manuscript.
• more thorough discussion and analysis with related work in the revised manuscript by citing the work you mentioned.
• more thorough comparisons with the sharpness-aware optimization method in Table R4, with corresponding analysis in the revised manuscript.
• the influence of computational efficiency, with corresponding analysis in the tier guideline section.
• more thorough discussion about the performance gains in Table R1 and the Table from the response to Reviewer p3Vh.

Currently, all of your concerns can be resolved in the revised version of the paper. We want to leave a gentle reminder that the discussion period is closing. We would appreciate your feedback to make sure that our responses and revisions have resolved your concerns, or whether there is a leftover concern that we can address to ensure a quality work.

Yours sincerely,

Authors of Paper 2181

NOTE: Table R1-R4 and Figure R1 are attached to the one-page PDF.

Weakness 1: Fix the omissions.

A: Thanks for your kind reminder. We have fixed the omissions and conducted a thorough proofreading for clarity.

Weakness 2: Significance analysis.

A: i) Note that we set the fixed random seed (seed everything) to ensure that the training results are unique in all experiments during CL [58,59], which shows that our significance is well-supported.

ii) Your nice suggestion inspires us that the class order (or task order) often significantly affects the performance of CL methods, a significance analysis of this is crucial. Hence, we set various seeds (1993, 1995, 1997) to split the dataset to further analyze the significance of C-Flat. In Table R1, we report the results, mean, std of running 3 times on different datasets. Table R1 concludes that the results of C-Flat remains significant across 3 datasets on all case. Finally, we will discuss significance in the revision.

Question 1: Compare with Adam.

A: In Table R2, we further compare C-Flat to Adam. Table R2 shows that C-Flat improves the performance of Adam in most cases.

Moreover, the reasons why we use SGD as a baseline in evaluation instead of Adam are as follows,

• SGD is the mainstream used in the CL community[58,59]. In CL, SGD performs better than Adam in CNN based vision tasks with higher test accuracy. This can also be concluded from the result of SGD and Adam in Table 2. While Adam outperforms SGD in Transformer-based models, like the foundation model, language model.

• Adam is better at saddle-point escaping but worse at flat minima selection than SGD[C2]. We analyze the mean escape time $\tau$ as follows,

$\tau$ of SGD is $\log (\tau)=O\left(\frac{B \Delta L_{a b}}{\eta H_a}\right)$; $\tau$ of Adam is $\log (\tau)=O\left(\frac{\sqrt{B} \Delta L_{a b}}{\eta \sqrt{H_a}}\right)$

$\tau$ of Adam exponentially depends on the square root of the eigenvalues of the Hessian at a minimum[C2], while $\tau$ of SGD exponentially depends on the eigenvalue of the Hessian at minima along the escape direction[C1].

• The saddle-point escape behavior of Adam is approximately independent of saddle-point flatness, whereas SGD is linearly dependent[C2].

Therefore, these observations and conclusions above motivated us to use SGD as a baseline.

[C1] A diffusion theory for deep learning dynamics: Stochastic gradient descent exponentially favors flat minima. ICLR 2021.

[C2] Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum. ICML 2022.

Question 2: The performance in offline-learning.

A: Adhere to the protocol [C3], we evaluate the effectiveness of C-Flat in offline learning scenes. As shown in Table R2, C-Flat still provides stable performance gains. This concludes that C-Flat still converges to flat minima even in offline learning cases, showing C-Flat features a good generalization.

[C3] Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning. CVPR 2021.

Question 3: The performance in domain-incremental settings.

A: Indeed, it should be interesting to see the performance of C-Flat in domain-incremental tasks, and Reviewer 475M is wondering about that as well. We would be happy to conduct a further evaluation on C-Flat.

So we tested the effectiveness of C-Flat according to the protocol [48]. Table R3 shows that C-Flat still provides consistent performance gains in domain-incremental scenarios. This means that C-Flat smoothly migrates to the global optimal of the joint space of the new and old tasks, thus mitigating catastrophic forgetting. This trend also occurs in task-incremental scenarios.

Apart from the typical benchmark, inspired by your suggestion and Reviewer 475M, we further investigated how a larger domain gap affects C-Flat. In this case, the domain shifts much more drastically, thus posing a non-trivial challenge for C-Flat. We trained on ImageNet21K and then evaluated CL performance on IN-R/ObjNet. Please see the Table below, C-Flat surprisingly holds solid performance gains. Such an observation further enhances the technical contribution of C-Flat. You could also refer to Reviewer 475M's question and our response to it.

Question 4: Compare C-Flat with other sharpness-aware optimization.

A: Actually, sharpness-aware method is relatively new in CL, among the limited works, we took the more general approach FS-DGPM for comparison in Table 2. We compared the accuracy, convergence speed and result with different of parameters of C-Flat with other sharpness-aware method like SAM, GAM in Section 4.6 and 4.7 with Figure 5 and 6.

To ensure the reliability, we further expanded the scope of the evaluation. In details, we compare C-Flat with more sharpness-aware methods, e.g, SAM, GSAM (newly added), and GAM. Then, the baseline methods used also increased from 1 of Figure 5 to 3, including Replay, WA, MEMO. Table R4 shows that C-Flat still achieves better accuracy.

Limitation 1: Choice of hyper-parameters.

A: If not specified, the hyper-parameters for tested models are defaulted according to the open-source repo [58,59]. Each task are initialized with the same $\rho$ and $\eta$, which drops with iterations according to the scheduler described in section 4.7.

Limitation 2: Any details on failure modes for C-Flat.

A: As our response to Q1, since the saddle-point escape behavior of Adam is approximate independent of saddle-point flatness, it cannot be as good as SGD at finding flat minima. In this sense, C-Flat is prone to fail on the Adam series, the results in Table R2 reconfirm this. As shown in Table R2, Adam showed a slight drop in some cases. We will discuss these in the revision.

We sincerely apologize for any misunderstanding regarding the significance you mentioned, and we appreciate your feedback on this matter. In this response, we have re-evaluated the uncertainty of our method using different random seeds. The updated results, including the mean and standard deviation over three runs across various benchmarks, are presented in the table below. As shown, our method continues to demonstrate significant and stable performance.

Thank you for your patience and for bringing this to our attention, which is greatly appreciated.

We also summarized the detailed results for each seed in the below table,

• The results with each seed on CIFAR-100 are as follows,

• The results with each seed on Tiny-ImageNet are as follows,