Gradient-Guided Epsilon Constraint Method for Online Continual Learning

We are grateful for your thorough assessment and encouraging feedback. We are delighted that you found our $\epsilon$-constraint perspective "novel and useful," the paper "very well-written," and the empirical results "strong." Your insightful comments will help us improve the quality of our work. We address each of your points below.

W1: Appendix Referencing and Related Work

This is an excellent suggestion to improve the paper's readability and self-containment. We fully agree and will implement the following in the final version:

• We will add explicit pointers (e.g., "(see Appendix A.1 for proof)") throughout the main text to directly link to proofs, algorithms, and other details.
• Also, we will add a new "Related Work" section to the main paper before the Conclusion. It will distill the most critical comparisons and provide context, including the discussion of the works you suggested [1][2].

W2: Comparison with Constrained Proximal Update Methods [1, 2]

We sincerely thank you for bringing these recent papers to our attention. They represent an important and complementary direction in OCL, and a direct comparison helps clarify GEC's contributions. We will add a comparison in our new "Related Work" section:

• First, GEC stems from a MOO perspective, explicitly balancing plasticity and stability via a dynamic $\epsilon$-constraint [3]. In contrast, LPR [1] and OCAR [2] approach the problem from an optimizer design perspective, using proximal methods and second-order information to stabilize the optimization process.

• Second, the GEC's update is dynamic. The coefficient $\lambda_k$ is recalculated at every step based on the current forgetting level $h(\theta_k)$. When forgetting is low ($h(\theta_k) \le 0$), $\lambda_k$ can be zero, focusing entirely on plasticity. When forgetting is high ($h(\theta_k) > 0$), $\lambda_k$ increases to prioritize stability. In contrast, the preconditioning in LPR/OCAR remains fixed between updates.

• As you pointed out, GEC is highly efficient. Our update has a closed-form solution requiring no matrix inversion. This is a significant practical advantage over methods that require inverting matrices.

W3: Missing MER baseline results

The MER baseline was omitted for Seq-CIFAR100 and Seq-TinyImageNet due to its significant computational demands. As a MAML-based method, MER lead to prohibitive training times and instability on these longer benchmarks. We will add a note in the experimental setup to clarify this decision, citing [4].

Nitpicks

• Thank you for this sharp-eyed catch of the use of "posterior". Our use was in the non-Bayesian sense of "after-the-fact." To avoid ambiguity, we will replace all instances with the more precise term "post-hoc".
• We are grateful for your meticulous reading of Figure 3c'scaption. The caption is incorrect and should be "Seq-TinyImageNet - Plasticity (Acc-P)". We will correct this.

Q1: Would the theoretical findings hold when approximated with Monte Carlo?

This is a very insightful question. Our theoretical analysis is presented in a deterministic setting for motivational clarity. In practice, where gradients are Monte Carlo estimates, the key insight holds because the stochastic gradients are unbiased estimates of the true gradients [5,6]. Over iterations, we expect the optimization trajectory to converge in a way that respects the constraints on average. Our empirical results, obtained entirely with mini-batches, serve as a validation that GEC's core idea is effective and resilient to the stochasticity of the OCL setting.

Q2: What were the values of $\alpha$ and $\epsilon$, and how should they be set?

This is a very insightful question about the practical application of GEC. We used $\alpha = 0.5$ and $\epsilon = 0.05$ for all experiments, which proved robust across all benchmarks. Here is a general guide to setting them:

• $\epsilon$ is the principal hyperparameter controlling the stability-plasticity trade-off. A smaller $\epsilon$ enforces stricter forgetting constraints (leading to more stability), while a larger $\epsilon$ allows more flexibility to learn the new task (more plasticity). As illustrated in our Figure 4, varying $\epsilon$ allows a user to effectively trace a Pareto front of high-performing solutions to match their specific needs.
• $\alpha$ controls the strength of the corrective action when the constraint is violated. We found that the performance is quite robust to the choice of $\alpha$. To demonstrate this empirically, we conducted a sensitivity analysis on Seq-CIFAR100.

Table 1: Different $\alpha$ on Seq-CIFAR100 (|M|=5k)

As the table1 shows, performance remains high and very stable across a sensible range of $\alpha$ values. The variation in both AAA and Acc is minimal. We will add this ablation study to the appendix.

Q3: Why does GEC's plasticity differ between Seq-CIFAR100 and Seq-TinyImageNet in Figure 3?

This is a good observation. Our hypothesis centers on the varying forgetting pressure of the datasets[7]:

• On Seq-CIFAR100, with high inter-task confusion, the constraint ($h(\theta) > 0$) is triggered more often, especially early on. GEC correctly prioritizes stability, sometimes at the cost of short-term plasticity. This is GEC working as intended: defending against severe forgetting.
• But on Seq-TinyImageNet, with more distinct classes, the constraint is more easily satisfied ($h(\theta) \le 0$). Here, GEC's advantage shifts to preventing overfitting to the memory buffer. Unlike ER's fixed weighting, GEC can dedicate its full update to the current task once the constraint is met, leading to superior plasticity.

Q4: What do the dashed lines mean in Figure 1a?

Thank you for seeking clarification. The dashed lines represent the iso-objective contours for ER's scalarized loss function, i.e., $\mathcal{L}_c + \lambda \cdot \mathcal{L}_m = c$, where $c$ is a constant. The slope of these lines is $-1/\lambda$. The optimal solution for a given $\lambda$ is found at the point where one of these lines is tangent to the achievable Pareto front. We will add this explanation to the figure caption to make it clear.

[1] Yoo, J., Liu, Y., Wood, F. & Pleiss, G.. (2024). Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:57199-57216.

[2] Urettini, E., & Carta, A. (2025). Online Curvature-Aware Replay: Leveraging 2nd Order Information for Online Continual Learning. arXiv preprint arXiv:2502.01866.

[3] Miettinen K. Nonlinear multiobjective optimization[M]. Springer Science & Business Media, 1999.

[4] Wu Y, Wang H, Zhao P, et al. Mitigating catastrophic forgetting in online continual learning by modeling previous task interrelations via pareto optimization[C]//Forty-first international conference on machine learning. 2024.

[5] Robbins H, Monro S. A stochastic approximation method[J]. The annals of mathematical statistics, 1951: 400-407.

[6] Bottou L. Stochastic gradient descent tricks[M]//Neural networks: tricks of the trade: second edition. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012: 421-436.

[7] Goldfarb D, Evron I, Weinberger N, et al. The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting--An Analytical Model[J]. arXiv preprint arXiv:2401.12617, 2024.

We sincerely thank the reviewer for the insightful and positive feedback. We are encouraged that you recognized our paper's clear organization, the novelty of our $\epsilon$-constraint perspective, and the strength of our proposed method and empirical results. You have raised two crucial points regarding computational overhead and the convexity assumption, which are vital for practical applications. We appreciate this opportunity to provide a detailed clarification.

W1/Q1: QP Computational Overhead and Scalability

This is a critical point regarding the potential computational overhead of the QP formulation. We would like to clarify and emphasize a key aspect of our method that directly addresses this concern: the specific QP problem in GEC, as defined in Eq. (8), admits a closed-form analytical solution, which we present in Eq. (11).

This means we do not need to run an iterative QP solver, and no further approximations that might compromise performance are necessary. The computation of the optimal update direction v_k involves only vector operations that scale linearly with the number of model parameters (d).

To further quantify this and empirically validate your intuition, we conducted a surgical experiment on Seq-CIFAR10 to precisely isolate the costs of GEC's components. We compared standard ER, our full GEC method, and a variant GEC (w/o QP) which performs the dual forward/backward passes but uses a simple gradient addition instead of our dynamic QP logic.

Table 1: Ablation on Computational Cost of GEC Components on Seq-CIFAR10 (|M|=1k)

And we can see that the primary increase in training time arises from the dual forward/backward passes, which are essential to compute the constraint violation $h(\theta_k)$. The additional cost of solving the QP via its closed-form solution is negligible. This efficiency is also reflected in Table 3 of our main paper, where GEC's training time is substantially lower than other advanced OCL methods like POCL and CBA. We will add more complexity analysis in Appendix in the final manuscript.

W2/Q2: Convexity Assumption and Non-Convex Performance

This is another insightful point, and we appreciate the opportunity to clarify the precise and limited role of the convexity assumption in our work.

You are correct that our theoretical analysis uses a convexity assumption. However, it's crucial to distinguish where and why it is used:

• First, the assumption of convexity is used unidirectionally in Proposition 2 to formally establish the equivalence from a standard ε-constraint problem back to ER's weighted-sum objective. This proof relies on the Karush-Kuhn-Tucker (KKT) conditions, where convexity is required for the conditions to be sufficient for global optimality [1]. This step serves to provide a rigorous theoretical grounding for ER's behavior within the established optimization literature.

• The analysis of our proposed GEC method's properties (Section 5.1 and Appendix B.2) does NOT depend on the convexity of the loss functions $\ell_c(\theta)$ and $\ell_m(\theta)$. Our theoretical guarantees (Constraint Adherence, Objective Minimization) are derived from the KKT conditions of the local QP problem defined in Eq. (8). This QP, which seeks the optimal update direction $v_k$ at each step, is always convex by its own construction (a quadratic objective with a single linear constraint), irrespective of the global non-convexity of the underlying deep learning loss landscape. This analytical approach is a standard tool for studying the behavior of optimization algorithms in non-convex settings [2] [3].

Therefore, GEC as presented is designed for and validated in the practical, non-convex setting of deep neural networks, and thus no specific modifications are needed. Moreover, the SOTA performance of GEC across all our experiments using the non-convex ResNet architecture (Tables 1, 2 and results in our response to Reviewer bPNr) serves as the validation of its effectiveness in these non-convex deep learning scenarios.

[1] Boyd S P, Vandenberghe L. Convex optimization[M]. Cambridge university press, 2004.

[2] Su W, Boyd S, Candes E J. A differential equation for modeling Nesterov's accelerated gradient method: Theory and insights[J]. Journal of Machine Learning Research, 2016, 17(153): 1-43.

[3] Miettinen K. Nonlinear multiobjective optimization[M]. Springer Science & Business Media, 1999.

We sincerely thank the reviewer for providing valuable feedback. We are encouraged that the reviewer found our theoretical analysis "original, easy to follow, and provided useful insights" and the GEC algorithm well-motivated. We detail our response below point by point.

W1/Q1: Performance on more challenging benchmarks

We thank the reviewer for this insightful suggestion. To demonstrate GEC's robustness, we conducted extensive new experiments in three challenging settings. Blew are the three experimental settings:

• Following [4], we introduced long-tail distributions to Seq-CIFAR10 and Seq-CIFAR100 with varying imbalance factors ($\gamma=2$ and $\gamma=5$) and distribution orders (Normal, Reversed).
• Following [3], we introduced three types of label noise ('flip1', 'flip2', 'uniform') at different corruption ratios to evaluate robustness.
• Following [5], we re-split Seq-TinyImageNet from 20 to a more challenging 40-task sequence, denoted as Seq-Tiny-l.

We compare GEC against strong SOTA baselines (DER++[1], CLSER[2], POCL[3]).

Table 1: Performance on Long-Tail Distributions

Table 2: Performance under Label Noise

Table 3: Performance on a Longer Task Sequence (Seq-Tiny-l, N=40)

• GEC's constraint is on the memory loss ($\bar{\ell}_m$) with a slack tolerance ($\bar{\varepsilon}$). This design naturally dampens the impact of high-variance gradients from imbalanced or noisy samples in the buffer. Unlike fixed-weighting methods (e.g., ER) which can be easily misguided, GEC's updates are more stable and principled (table1&2).
• The compelling lead in the 40-task scenario, as shown in table3, highlights GEC's key advantage. By explicitly monitoring and correcting for forgetting, GEC prevents the gradual error accumulation that plagues methods with fixed trade-offs over long horizons.

These results confirm that GEC's is also effective for complex, realistic CL settings. We will include these results into the final paper.

W2/Q2: Interaction with BatchNorm (BN)

This is a sharp and important observation. To rigorously isolate GEC's algorithmic contribution from potential BN effects, we conducted two controlled experiments.

• We created a baseline, ER (Dual-Fwd), that mimics GEC’s dual forward pass structure, ensuring identical BN update dynamics for a fair comparison.

• We also replaced BatchNorm with the stateless GroupNorm (GN) for both ER and GEC like [6].

Table 4: Performance comparison isolating the effect of BN dynamics.

Table 5: Performance comparison with Group Normalization (GN) on Seq-CIFAR100 (|M|=5k).

• Table 4 shows that matching BN dynamics (ER (Dual-Fwd)) provides marginal gains over standard ER. This proves GEC's substantial performance leap stems from its core gradient-guided constraint mechanism.
• Table 5 further solidifies this. With GroupNorm, GEC still maintains its significant lead, demonstrating its effectiveness is independent of the normalization layer's statefulness.

We hope these experiments have fully addressed your concerns. Your feedback has been invaluable in strengthening our paper, and we will incorporate these results into the appendix.

[1] Dark experience for general continual learning: a strong, simple baseline. NeurIPS 2020

[2] Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System. ICLR 2022

[3] Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization. ICML 2024

[4] Imbalanced continual learning with partitioning reservoir sampling. ECCV 2020.

[5] Meta Continual Learning Revisited: Implicitly Enhancing Online Hessian Approximation via Variance Reduction. ICLR 2024

[6] Continual normalization: Rethinking batch normalization for online continual learning. ICLR 2022.