Forgetting Order of Continual Learning: What is Learned First is Forgotten Last

Thank you for your review and positive feedback! We appreciate the recognition of the strengths in our work and the valuable suggestions for improvement. Below, we address your comments and questions:

W1 + Q1: To address this suggestion, we have significantly expanded the section on CIL in the revised manuscript. Specifically, we now include in Appendix A a repetition of all major figures from the main paper under CIL settings. These include:

• Figure 9 (repeating Figure 2)
• Figure 10 (repeating Figure 3)
• Figure 11 (repeating Figure 4)
• Figure 12 (repeating Figure 6)
• Figure 13 (repeating Figure 7)
• Figure 14 (repeating Figure 8)

This expanded analysis shows that, while CIL naturally leads to lower accuracy compared to TIL (as it is inherently more challenging), the qualitative trends and conclusions remain consistent. The results reaffirm that the insights and analysis in the TIL case also hold for CIL.

Thank you for your elaborate review. Below, we address each of your points separately.

Intuition

Simplicity bias research suggests that neural networks learn examples in order of increasing complexity. At the end of a task, the network performs well on examples up to a certain complexity level. For replay buffers, this implies that:

• Excluding slowly learned (high-complexity) examples makes sense because the network struggled to learn them initially and is unlikely to benefit from replaying them.
• Excluding quickly learned (low-complexity) examples is also reasonable because these examples provide limited additional value, given the network’s ability to handle more complex tasks.

We added a discussion of this point to Section 3.2.

Other datasets

While Figs. 2 and 3 focus on 2-task scenarios in CIFAR-10 and CIFAR-100, the paper also includes results from broader settings. For instance:

• Fig. 4c: Tiny-ImageNet with 2 tasks
• Fig. 7: CIFAR-10 with 5 tasks and CIFAR-100 with 20 tasks
• Fig. 8: CIFAR-100 with 5 tasks and Tiny-ImageNet with 10 tasks Although these figures do not explicitly quantify the correlation between learning speed and catastrophic forgetting, the success of Goldilocks in these scenarios implicitly relies on this relationship.

To address your suggestion directly, we have added new results in Appendix B (Fig. 19) and Section 2, validating the correlation from Fig. 2c across diverse datasets and task configurations, including CIFAR-100 with 20 tasks, CIFAR-10 with 5 tasks, and Tiny-ImageNet with 2 tasks finding a stronger correlation.

Task incremental learning

Both class-incremental and task-incremental learning are important and widely accepted paradigms in the community. Our choice to focus on task-incremental learning in the main paper was deliberate, as we believe it provides a "cleaner" view of catastrophic forgetting.

That said, we recognize the value of evaluating both scenarios. To address this, we conducted a parallel analysis in the class-incremental setting and included these results in Appendix A of the original manuscript. In the revised manuscript, we expanded the analysis, repeating all Figs 2, 3, 4, 6, 7, 8 with CIL setting, in Figs. 9-14 respectively (App A). A short discussion was added to Sec 2.2, noting that the CIL results mirror those of TIL.

Marginal benefits and statistical significance We discovered a bug in the code used to generate Fig 24, which contained the standard errors for Fig 4. This was fixed in the revision, showing that all the observed results are statistically significant.

Other than the standard errors themselves, the consistent results across a large range of hyperparameters and experiments also indicate statistical significance. Large standard errors would have resulted in highly erratic behavior, particularly in experiments with closely related hyperparameters. Instead, our findings demonstrate smooth and systematic improvements.

We note that a 2% improvement in final accuracy is non-trivial in the continual learning domain, and many works report much smaller improvements. Moreover, for scenarios with multiple tasks (Fig 8), the gains are more pronounced, reaching up to 5%.

Extracting arguments from the results

While section 3.3 in the main paper focuses on subclasses of CIFAR-100, we address broader task dissimilarities in App E. We include scenarios where the second task is:

• Rotated version of the first task, with a different objective
• Random labels Both setups introduce dissimilarity between tasks (see Fig 28 in App E). We have rephrased the relevant text in Section 3.3 to clarify this point.

c-score:

Thank you for bringing this reference to our attention. The c-score measures the expected accuracy of individual examples and, like other accuracy-based metrics, shows a strong correlation with learning speed. In Appendix G, we discuss similar accuracy-based scores and their relationship to catastrophic forgetting. Among all these scores, learning speed has the strongest correlation to catastrophic forgetting, and is the cheapest to compute, making it well-suited for our analysis. We added a short discussion and a direct comparison of the c-score to App G and Fig 29d.

Class order:

The specific split of classes into tasks did not affect our results. We added to the revised manuscript results with another data-split, in App c, Fig 21c. The results are similar to original ones in Fig.4.

Experience replay strategy:

In our experiments, we observed no significant difference between alternating batches and mixing samples from the current task and the replay buffer in a 50-50 ratio. The choice to alternate batches was made purely for code convenience. This point is noted in the revised manuscript for transparency.

Thank you for your review. Below, we address the different points you raised separately:

Weaknesses — significance and paper's focus

As noted in the general comment, our work adopts an observational approach, focusing on uncovering behavioral patterns in neural networks rather than introducing a new method or theoretical framework. Specifically, we empirically identify a novel and previously unreported connection between simplicity bias and catastrophic forgetting. While simplicity bias — where networks learn simple examples first — is well-known, we demonstrate that catastrophic forgetting exhibits a "reverse simplicity bias," where complex examples are forgotten before simpler ones.

The perceived lack of "surprise" may stem from the robustness of simplicity bias, which intuitively suggests its relevance to forgetting. As almost any neural network learns simple things first, it is intuitive that an equally robust pattern will occur when such a network forgets. We believe that this robustness enhances the significance of our findings: if forgetting patterns mirror simplicity bias, this suggests a foundational phenomenon that can guide future work in continual learning, leveraging established insights and tools from simplicity bias research. While the "surprise" of a result is subjective, we argue that our contribution lies not in the novelty of simplicity bias itself but in its new application and implications for continual learning.

As for Goldilocks, it serves as a proof of concept to demonstrate how accounting for this forgetting pattern can improve continual learning. Although we do not agree that the improvement is not dramatic (2-4% of consistent accuracy gain across different scenarios is not easy to achieve), Goldilocks is mainly designed to show that even a relatively simple sampling function based on the connection between simplicity bias and continual learning can improve a large array of continual learning methods, demonstrating the potential for future works to take into account this connection when devising new methods.

Finally, the title reflects the primary focus of the paper: uncovering and understanding the connection between forgetting and catastrophic forgetting. As Goldilocks serves as a practical example, we do not think it should be featured in the title.

Setting hyperparameters

The wide range of hyperparameters in our experiments aimed to provide a comprehensive view of how buffer compositions impact forgetting. For practical deployment, we added guidance in Section 3.3 of the revised manuscript to simplify hyperparameter selection for new settings.

In summary:

• A heuristic approach is often sufficient. For instance, setting $q=s=20\%$ performed well across all datasets and scenarios tested.

• Adjustments can be made based on prior knowledge of the dataset's complexity:

  • For complex datasets, favor excluding more complex examples $(s>q)$.

  • For simpler datasets, favor excluding simpler examples $(q>s)$.

• For scenarios where computational resources allow, a non-heuristic approach based on auxiliary tasks can further optimize hyperparameters, even with limited data.

We hope these additions address your concerns about practical applicability and encourage you to view this work as a foundation for further exploration of this phenomenon.

Thank you very much for your review!

Addressing the weaknesses and questions you raised:

Completeness:

We agree that extending Table 1 to include larger datasets such as CIFAR-100-5, CIFAR-100-20, and Tiny-ImageNet would strengthen the manuscript and enhance its completeness. However, we were unable to conduct these additional experiments during the rebuttal period due to the computational demands of certain baselines (e.g., GSS and IPM) and the modifications required to adapt their code for multi-task settings. We will incorporate these datasets into Table 1 for the camera-ready version.

Limitation:

Training Goldilocks with a limited number of training epochs presents two potential challenges. First, because the learning speed (Eq. (1)) depends on the number of epochs, its accuracy diminishes as the number of epochs decreases. Second, as you noted, if the networks train for too few epochs, they may not converge, leading to instances where examples are incorrectly classified as "slowly learned" due to insufficient training.

In the revised manuscript, we address these concerns by presenting additional experiments (Appendix C + Figure 22). These experiments show that the same buffer compositions remain effective even with significantly reduced training iterations. Although there is some noise introduced by the less accurate learning speed in these cases, the qualitative results remain consistent. On the other hand, for scenarios like streaming data, where the network may not converge adequately, the learning speed simply can not capture well enough how fast the model learns certain examples, and Goldilocks requires some different complexity measurement to work well (which is beyond the scope of our work).

As recommended, we have incorporated these findings into the limitations section (Section 4), and we discuss the results from Figure 22 in Appendix C and Section 3.2 to provide practical guidance on the appropriate number of epochs needed for the method to perform optimally.

Hyperparameters:

Figs. 4-6 present a wide range of hyperparameters $q$ and $s$ to demonstrate their impact comprehensively. These results indicate that a broad selection of $q$ and $s$ values can effectively enhance learning, highlighting the ease of hyperparameter selection for new datasets. For instance, in all tested settings and datasets, simply setting $s=q=20\%$ significantly improved performance.

These straightforward choices can be further refined based on dataset characteristics: for challenging datasets, setting $q < s$ may be beneficial, as slowly learned examples are less likely to contribute to learning and can be removed more aggressively. Conversely, for simpler datasets, choosing $s < q$ may be preferable. Importantly, these heuristic adjustments require no additional computation and align with the premise of continual learning.

While many continual learning studies provide hyperparameters without justification, we opted to include explanations to guide practitioners. For scenarios with additional computational resources, we also propose a systematic approach for hyperparameter tuning, which we have described in Section 3.3. Additionally, we added to the revised manuscript, at the end of section 3.3, a paragraph explaining these points, and guiding how to pick in practice hyper-parameters when encountering new data, which we hope will help guide future readers.

Question:

In the paper, we present results on multiple datasets, including CIFAR-100-2, CIFAR-100-20, CIFAR-10-2, CIFAR-10-5, TinyImageNet-2, and TinyImageNet-10. Due to the limited time available during the rebuttal period, we were unable to include experiments on datasets significantly different from those already evaluated. However, we added an explanation on how to pick hyper-parameters at the end of Section 3.3. Additionally, we plan to incorporate additional datasets for the camera-ready submission. Moreover, in this revised manuscript, we have included results for an alternative split of CIFAR-100 into two tasks, with classes split into each task randomly. These results are provided in Appendix C, Figure 21c.

Thank you very much for your review! Below, we address separately each point in your review:

Theory

As noted in the general comment, this paper takes an empirical approach, focusing on behavioral observations to reveal novel insights into neural networks. While we acknowledge that theoretical analysis can strengthen the findings, we believe that empirical evidence also plays a crucial role in advancing understanding, especially in areas like continual learning where theoretical tools may be limited.

Correlation under different learning hyper-parameters

Regarding your comments about the empirical analysis, we incorporated your suggestions for additional experiments into the revised manuscript, which we believe improved its quality.

Throughout our empirical study, we explored various datasets and settings, selecting hyperparameters such as optimizers, architectures, and regularization heuristically. Given the extensive hyperparameter space in deep learning, exhaustively testing every result against all possible configurations is infeasible for a single paper. However, we incorporated your suggestions and extended one of our main results (specifically Figure 4) to include the proposed hyperparameter variations. Notably, we found that these variations do not alter the reported correlation: the relationship between learning speed and catastrophic forgetting remains consistent. Moreover, buffer compositions effective in one setting often generalize well to others, further supporting the robustness of our findings.

The new experiments are detailed in Appendix C and referenced in Section 3.2. Specifically, we added:

• Comparisons across different learning rates, including lowering the learning rates for subsequent tasks. These results can be found in Figure 23
• Comparisons across different optimizers, including Adam, SGD, and Adagrad, as suggested. These results can be found in Figure 20.
• Results on a non-ResNet-based architecture, specifically VGG-16, in Figure 21a.
• Comparisons between training with and without regularization. These results can be found in Figure 21b.

Regarding dataset complexity, our results already include experiments on Tiny ImageNet (Figures 4 and 8), which is commonly considered as challenging as ImageNet. Nevertheless, we plan to include experiments on ImageNet subsets in the camera-ready version, as they take too long to include in the limited time of the rebuttal.

---

Thank you again for your constructive feedback, which helped us improve our manuscript. We hope our responses have sufficiently addressed your concerns, and we believe the revisions strengthen the paper. Given the mixed reviews, every point is crucial, and we hope you will consider our clarifications in your final evaluation.