Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay

Answer to W4

Regarding task sequences, we evaluate our approach across several stages:

1. Split CIFAR-100: This dataset consists of 100 classes divided into 10 tasks, each containing 10 classes.
2. Split Mini-ImageNet: This dataset divides 100 classes into 5 tasks, each with 20 classes.
3. Split Tiny-ImageNet: This dataset includes 200 classes split into 10 tasks, each containing 20 classes.

Each dataset provides 500 training samples per class. The number of test samples varies: Split CIFAR-100 and Split Mini-ImageNet have 100 test samples per class, while Split Tiny-ImageNet has 50 test samples per class.

Regarding dataset complexity, Split Mini-ImageNet and Split Tiny-ImageNet are recognized as challenging benchmarks in the continual learning literature. While some studies use simpler datasets like MNIST or CIFAR-10, Split Mini-ImageNet, and Split Tiny-ImageNet are considered more complex and are often used to benchmark methods under more demanding conditions.

Answer to W5

We will enhance Figure 2 by incorporating your suggestion and ensuring it is updated for the camera-ready version.

[1] Mohammad Motamedi, Nikolay Sakharnykh, and Tim Kaldewey. A data-centric approach for training deep neural networks with less data. arXiv preprint arXiv:2110.03613, 2021.

[2] Mark Mazumder, Colby Banbury, Xiaozhe Yao, Bojan Karlas, William Gaviria Rojas, Sudnya Diamos, Greg Diamos, Lynn He, Alicia Parrish, Hannah Rose Kirk, et al. Dataperf: Benchmarks for data-centric ai development. arXiv preprint arXiv:2207.10062, 2022.

[3] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159, 2018.

[4] Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. arXiv preprint arXiv:2009.10795, 2020.

Thank you for your insightful comment We appreciate the opportunity to clarify your concerns.

Answer to W1

Buffer-based methods store samples in a buffer, which has been shown in the literature to yield better performance compared to other continual learning (CL) approaches. However, this advantage comes with the drawback of memorizing samples, making these methods more prone to overfitting, as noted in prior work. For this reason, our focus has been primarily on buffer-based methods. Nonetheless, as per your suggestion, we will include results for architecture- and regularization-based methods in the camera-ready version.

Answer to W2

To address the challenge of over-reliance on synthetic OOD corruptions and better emulate real-world distribution shifts, we draw on the methodology proposed by Hendrycks and Dietterich [1]. Their work introduces a standardized benchmark designed to evaluate the robustness of image classifiers against a variety of common corruptions and perturbations, offering a more comprehensive and systematic approach to studying robustness. This benchmark not only identifies classifiers that are better suited for safety-critical applications but also broadens the focus of robustness research beyond adversarial examples. By emphasizing common real-world corruptions, the framework bridges the gap between synthetic benchmarks and practical robustness requirements, enabling a more realistic evaluation of classifier performance under OOD conditions.

To further extend this work, we propose addressing domain shifts and adversarial inputs as part of future research directions.

[1] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019.

Answer to W3

For the temperature $\tau$ hyperparameter, we follow prior studies and select a small value of 0.09, as commonly done in the literature. The role of the temperature $\tau$ in contrastive loss has been extensively analyzed in previous works. However, if further analysis is deemed necessary, we are prepared to conduct a sensitivity study within our framework and include the results in the camera-ready version.

Regarding buffer sizes, we conducted experiments with various configurations, including 500, 1000, and 2000, to evaluate the impact on performance.

Thank you once again for your thoughtful comments and the score you provided. we think we have addressed all your concerns in our previous responses. If we have overlooked anything or if you have additional feedback, please feel free to let us know so we can address them in the camera-ready version. Regarding imbalanced datasets, we will conduct the necessary experiments and include the results in the camera-ready version.

Answer to W6

Thank you for highlighting this important question about the robustness of confidence variance.

To identify the most informative samples of each class, we utilized a data-centric artificial intelligence approach. In data-centric AI, the emphasis is on improving datasets to facilitate effective model training with fewer data requirements [1, 2]. For instance, as cited in our paper, [3] and [4] leverage model predictions during training to refine datasets. These studies demonstrate that early-stage uncertainty is a reliable indicator of a sample’s contribution to the training process. By analyzing network predictions for individual samples during the initial training epochs, they identify noisy labels and outlier samples, aiding dataset optimization.

During the training process, the decision boundary evolves. The key question is: Which boundary leads to better generalization? One shaped by outliers or one influenced by genuinely challenging (most informative) samples?

To address this, we analyzed CIFAR10 in batch training (training all 10 classes at once, resembling one task of Split CIFAR-100 under continual learning). We calculated the target confidence of each sample at every epoch and displayed the mean and variance of sample confidence across these epochs in the Figure linked below. In the left sub-figure, we consider the first $E$ epochs $E = 5$ as in our paper, and in the right sub-figure, we use 50 epochs (matching the epoch count per task in our paper).

Please click to view the Figure

We categorized the samples observed during the first $E = 5$ epochs into three groups:

• Area 1. Simple Samples: High confidence with low variance, consistently well-classified and at the center of the distribution.

• Area 2. Challenging Samples (Most Informative): Moderate mean confidence with high variance, residing near the decision boundary. They are ideal for shaping a robust boundary.

• Area 3. Hard (Outlier) Samples: Low confidence with low variance, far from the decision boundary, offering limited value for boundary refinement.

The bottom sub-figure shows representative images from these areas (Area 1, Area 2, and Area 3) for each CIFAR10 class. Each column represents a class, with rows corresponding to simple, challenging, and hard samples, respectively. Simple samples display clear class attributes (e.g., a horse on grass; grappling with the effects of spurious correlation). Challenging samples exhibit ambiguous traits (e.g., an airplane against a gray backdrop; enhancing OOD generalization), while hard samples feature unusual or rare characteristics (e.g., a horse’s head only), often functioning as outliers.

As training progresses from $E = 5$ to $E = 50$:

• Simple Samples retain high confidence and low variance, remaining at the center of the distribution.

• Challenging Samples show increasing confidence (mean confidence rises) and decreasing variance as the model learns them robustly. Most migrate from the boundary to the center, while a few persist near the boundary.

• Hard Samples exhibit higher confidence but also increased variance, eventually achieving moderate confidence and defining the boundary.

Thus:

• Area 4 ($E = 50$) contains simple samples and most challenging samples.

• Area 5 ($E = 50$) contains hard samples and some challenging samples, shaping the boundary.

Comparing Area 2 ($E = 5$) and Area 5 ($E = 50$), we argue that Area 2, with challenging samples, provides better generalization. By prioritizing these challenging (most informative) samples for buffer selection, the decision boundary is shaped by meaningful boundary samples rather than outlier samples. This strategy "reminds" the model of an ideal boundary in future tasks, promoting generalization and stability-plasticity balance.

In Figure 4 of the Appendix, we analyze the sensitivity of the hyperparameter $E$. The results show stable performance across a range ($E = 2$ to $7$) on all datasets, maintaining consistency in ACC and BWT under i.i.d. and OOD conditions. These findings confirm that identifying boundary samples requires only a few initial training epochs.

Additionally, as shown in the Figure linked below, we conducted another experiment on CIFAR-100, extending the range of $E$ from 2 to 14. The results demonstrate that performance gradually decreases beyond $E = 7$. This is because outliers transition to challenging samples, and challenging samples to simple ones, reducing their impact on refining the decision boundary.

Please click to view the Figure

Thank you for your insightful comment. We appreciate the opportunity to clarify the differences between our approach and the CoPE [1], as well as ER-AML, and ER-ACE [2]. Our proposed method, Adaptive Contrastive Replay (ACR), introduces significant novelties that differentiate it from existing approaches.

Below, we provide a detailed comparison between our approach and CoPE [1]:

Proxy in ACR (Adaptive Contrastive Replay):

1. Definition:

   • In ACR, the weights of the classification layer (linear layer) are used as proxies for each class. Each class $ y $ is represented by a weight vector $ W_y \in \mathbb{R}^d $, which serves as a class-level anchor in the latent space.

2. Formulation:

   • The proxy-based contrastive loss aligns the latent embeddings $ f_\theta(x) $ of inputs with their respective class weights $ W_y $ while pushing them away from the weights of other classes:
     $

     L_{\text{PCL}} = - \frac{1}{N} \sum_{i=1}^N \log \frac{\exp(f_\theta(x_i)^\top W_{y_i} / \tau)}{\sum_{L \in \mathcal{C}} \exp(f_\theta(x_i)^\top W_L / \tau)}.
     $

     • $ W_{y_i} $: Weight vector (proxy) for the target class $ y_i $.
     • $ \mathcal{C} $: Set of class labels in the batch.
     • $ \tau $: Temperature parameter.

3. Role of Weights:

   • The weights ($ W_y $) act as class-level representatives in the latent space.
   • These proxies guide the embeddings $ f_\theta(x) $ toward their respective classes, ensuring better class separation in the latent space.

4. Classifier:

   • Linear Cosine Similarity Classifier: For ACR, the classification is performed using a linear cosine similarity-based classifier. This computes the cosine similarity between normalized feature embeddings and normalized class weights (proxies), scaled by a temperature parameter:

     
     $

     \text{logits}[y] = \frac{\cos\left(f_\theta(x), W_y\right)}{\tau},
     $

     where:

     • $ f_\theta(x) $ is the normalized feature embedding of the input $ x $,
     • $ W_y $ is the normalized weight vector (proxy) for class $ y $,
     • and $ \tau $ is the temperature parameter controlling the sharpness of the logits.

     The class with the highest logit is chosen as the predicted class:

     
     $

     \hat{y} = \arg\max_{y \in \mathcal{Y}} \text{logits}[y].
     $

5. Dynamic Behavior:

   • The proxies ($ W_y $) are tied to the classifier's weight matrix and evolve automatically during training without requiring additional memory structures.

Proxy in CoPE (Continual Prototype Evolution):

1. Definition:

   • CoPE maintains explicit prototypes for each class $ c $, denoted as $ p_c \in \mathbb{R}^d $. These prototypes are stored in a separate memory and represent the center of mass of the latent embeddings for their respective classes.

2. Formulation:

   • Prototypes are updated dynamically using a momentum-based rule:
     $

     p_c \gets \alpha p_c + (1 - \alpha) \bar{p}_c,
     $

     • $ \alpha $: Momentum parameter.
     • $ \bar{p}_c = \frac{1}{|B_c|} \sum_{x_i \in B_c} f_\theta(x_i) $: Mean of latent embeddings for class $ c $ in the current batch $ B_c $.
     • After updating, the prototype $ p_c $ is normalized to unit length:
       $

     p_c \gets \frac{p_c}{||p_c||}.
     $

3. Role of Prototypes:

   • The prototypes explicitly represent the cluster center for each class in the latent space.
   • They are independent of the classifier and stored in a separate memory structure.
   • Prototypes are used in the pseudo-prototypical proxy loss (PPP-loss) to enforce intra-class compactness and inter-class separation.

4. Classifier:

   • Nearest Neighbor Classifier: CoPE uses a nearest neighbor classifier for classification. It compares the feature embedding $ f_\theta(x) $ to stored prototypes $ p_c $ and predicts the class based on the closest prototype:
     $

     c^* = \arg\max_{c \in Y} \cos(f_\theta(x), p_c)
     $

5. Dynamic Behavior:

   • The prototypes are explicitly updated with each batch of data to reflect changes in class distributions over time.

Below, we present a detailed comparison of our approach with ER-AML and ER-ACE [2]:

ACR Loss: Proxy-Based Contrastive Loss

• Formulation: Aligns the latent embedding $ f_\theta(x) $ (where $ f_\theta $ is the encoder with parameters $\theta$, mapping the input $x$ into the latent space) of a sample $x$ with its class proxy $W_y$ (a representative vector for the target class $y$) while separating it from proxies $W_L$ of all other classes in the batch.
  $

  L_{\text{PCL}} = - \frac{1}{N} \sum_{i=1}^N \log \frac{\exp(f_\theta(x) \cdot W_y / \tau)}{\sum_{L \in C} \exp(f_\theta(x) \cdot W_L / \tau)}
  $

  • $N$: The total number of samples in the batch.
  • $\tau$: A temperature parameter controlling the sharpness of similarity distributions.
  • $C$: The set of all class labels in the current batch.

• Objective: Ensures compact clustering of embeddings for the same class and separation between clusters of different classes.

• Mechanism:

  • Optimizes the proxy-to-sample relationship for efficiency, reducing computational overhead by avoiding explicit pairwise comparisons between individual samples.
  • Expands the pool of negatives by considering all non-target class proxies $W_L$ as negatives.

• Key Focus: Efficient and stable feature alignment using proxies.

ER-AML Loss: Asymmetric Metric Learning Loss

• Formulation: Combines two components:

  1. Supervised Contrastive Loss (SupCon) for incoming data ($X_\text{in}$, the set of new samples being learned):
     $

     L_1(X_\text{in}) = - \sum_{x_i \in X_\text{in}} \frac{1}{|P(x_i)|} \sum_{x_p \in P(x_i)} \log \frac{\text{sim}(f_\theta(x_p), f_\theta(x_i))}{\sum_{x_n \in N \cup P(x_i)} \text{sim}(f_\theta(x_n), f_\theta(x_i))}
     $

     • $P(x_i)$: The set of positive samples (other samples of the same class as $x_i$ in the batch or buffer).

     • $N$: The set of negative samples (samples from other classes in the batch or buffer).

     • $\text{sim}(a, b)$: The similarity between embeddings $a$ and $b$, calculated as:
       $

       \text{sim}(a, b) = \exp\left(\frac{a^\top b}{\tau ||a|| ||b||}\right)
       $

  2. Cross-Entropy Loss for replayed data ($X_\text{bf}$, the set of samples replayed from the buffer):
     $

     L_2(X_\text{bf}) = - \sum_{x \in X_\text{bf}} \log \frac{\text{sim}(W_{c(x)}, f_\theta(x))}{\sum_{c \in C_\text{all}} \text{sim}(W_c, f_\theta(x))}
     $

     • $W_{c(x)}$: The class proxy of the label $c(x)$ corresponding to sample $x$.
     • $C_\text{all}$: The set of all class labels observed so far.

• Combined Loss:
  $

  L(X_\text{in} \cup X_\text{bf}) = \gamma L_1(X_\text{in}) + L_2(X_\text{bf})
  $

  • $\gamma$: A weighting factor balancing the two terms.

• Objective: Aligns embeddings of current task data while consolidating knowledge across previous tasks.

• Mechanism:

  • Uses pairwise relationships among samples (via $P(x_i)$ and $N$) for finer control over embeddings.
  • Incorporates class prototypes for replayed data to maintain stability.

ER-ACE Loss: Asymmetric Cross-Entropy Loss

• Formulation: Combines two cross-entropy losses:
  $

  L_{\text{ER-ACE}} = L_{\text{ce}}(X_\text{in}, C_\text{curr}) + L_{\text{ce}}(X_\text{bf}, C_\text{old} \cup C_\text{curr})
  $

  • $C_\text{curr}$: The set of classes in the incoming batch.

  • $C_\text{old}$: The set of previously seen classes stored in the buffer.

  • $L_{\text{ce}}(X, C)$: The cross-entropy loss for samples $X$ over classes $C$:
    $

    L_{\text{ce}}(X, C) = - \sum_{x \in X} \log \frac{\exp(\text{sim}(W_y, f_\theta(x)))}{\sum_{c \in C} \exp(\text{sim}(W_c, f_\theta(x)))}
    $

• Mechanism:

  • For $X_\text{in}$: Computes logits for only current classes $C_\text{curr}$, isolating new class learning.
  • For $X_\text{bf}$: Includes both current and past classes ($C_\text{curr} \cup C_\text{old}$), consolidating previous knowledge.

• Objective: Indirectly enforces contrast by isolating logits for incoming data and leveraging replayed data for broader class alignment.

In response to your question about integrating another method with our proxy-based contrastive learning loss, we have combined SOIF [3] with our proxy-based contrastive loss and present the results in the following table. For comparison, we have also included the results of SOIF from Table 1 and Table 2. SOIF leverages second-order influences to make more informed decisions about which samples to retain in the buffer, using cross-entropy loss. To analyze the performance of our PCL loss, we replaced its loss with our proxy-based contrastive loss. We conducted experiments on the Split CIFAR-100 dataset with a buffer size of 500. The results are shown below:

\begin{array}{c|c|c|c|} \hline Method & & \text{SOIF} & \text{SOIF}_{+\mathbf{\mathcal{L}}_{\mathbf{\text{PCL}}}} \\ \hline ACC & i.i.d. & 18.85 & 21.46 \\ & OOD & 3.18 & 10.69 \\ \hline BWT & i.i.d. & -71.47 & -69.24 \\ & OOD & -19.72 & -43.17 \\ \hline \end{array}

As evident across all metrics and scenarios (except BWT on OOD), we outperform the baseline. The better BWT (OOD) of the baseline, as discussed in our paper, indicates that it has not acquired sufficient knowledge to retain, let alone forget.

References

[1] De Lange, Matthias and Tuytelaars, Tinne. "Continual Prototype Evolution: Learning Online From Non-Stationary Data Streams", ICCV,2021.

[2] Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, Eugene Belilovsky. "New Insights on Reducing Abrupt Representation Change in Online Continual Learning", ICLR,2022.

[3] Zhicheng Sun, Yadong Mu, and Gang Hua. “Regularizing second-order influences for continual learning”, CVPR, 2023.

We appreciate the reviewer’s feedback and agree that additional analysis can clarify the contributions of each component of our method. To isolate the performance gain from our memory management strategy, we analyzed various update policies for our proxy-based contrastive learning method on the Split CIFAR-100 dataset with buffer sizes of 500, 1000, and 2000, as well as their mean performance. The results for each buffer size and the mean are presented in the following table.

Initially, we evaluated the Reservoir update policy, which updates the buffer batch-by-batch using uniform random sampling. As observed in the ER approach (Table 3), which also employs Reservoir sampling, this policy can result in class imbalances due to batch-wise updates and random sampling. Despite the randomness of the Reservoir update, our method achieves a higher average ACC compared to the second-best method, GEM (i.i.d.: 29.86%; OOD: 4.34%; Table 1), by 1.47% in the i.i.d. scenario and 10.2% in the OOD scenario. This indicates that our proxy-based contrastive loss effectively generates distinct representations.

However, the Reservoir update also results in higher forgetting, as observed in both the ER method (Table 2) and the Reservoir policy (in the following table). The batch-by-batch update increases the number of seen samples in the buffer, contributing to class imbalances. As the model encounters numerous samples, including outliers, it tends to focus less on learning informative features and more on memorizing samples due to the frequent buffer updates. This behavior increases the forgetting rate. To address this, we shifted to a task-by-task buffer update strategy and aimed for a balanced class/task distribution in the buffer. The results of this approach, labeled as ”Balanced Class/Task,” are shown in the following table.

In the Random policy scenario, this change leads to significant improvements in both ACC and BWT, surpassing the Reservoir policy by a large margin. This demonstrates that transitioning from batch-by-batch to task-by-task updates and ensuring balanced class/task distributions reduce the forgetting rate and enhance accuracy.

Next, we analyzed the effect of populating the buffer with misclassified samples. As previously mentioned, these samples fall into two categories: outliers (referred to as ”Hard” in the table) and boundary samples (referred to as ”Challenging” or ”ACR”). Populating the buffer with Hard (low-confidence) samples yields the worst performance, whereas using Challenging (high-variability, boundary) samples improves ACC in both i.i.d. and OOD scenarios, and reduces the forgetting rate in both i.i.d. and OOD scenarios.

The table is as follows:

\begin{array}{c|c|c|ccc|c|ccc|} \hline Buffer & & 500 & & 500 & & 1000 & & 1000 & \\ \hline Balanced Class/Task & & - & & \checkmark & & - & & \checkmark & \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 23.34 & \underline{28.12} & 14.23 & 29.27 & 33.03 & \underline{33.97} & 18.49 & 36.06 \\ & OOD & 11.61 & \underline{13.65} & 7.20 & 14.81 & 14.98 & \underline{16.11} & 8.75 & 17.86 \\ \hline BWT & i.i.d. & -58.22 & \underline{-47.61} & -65.47 & -41.78 & -48.70 & \underline{-36.98} & -57.61 & -35.08 \\ & OOD & -34.38 & \underline{-23.86} & -33.04 & -18.38 & -31.35 & \underline{-17.07} & -29.51 & -15.44 \\ \hline \end{array}

\begin{array}{c|c|c|ccc|c|ccc|} \hline Buffer & & 2000 & & 2000 & & Mean & & Mean & \\ \hline Balanced Class/Task & & - & & \checkmark & & - & & \checkmark & \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 37.63 & \underline{40.89} & 23.42 & 42.24 & 31.33 & \underline{34.33} & 18.71 & 35.86 \\ & OOD & 17.02 & \underline{18.95} & 11.01 & 20.57 & 14.54 & \underline{16.24} & 8.99 & 17.75 \\ \hline BWT & i.i.d. & -49.98 & \underline{-27.73} & -53.13 & -24.69 & -52.30 & \underline{-37.44} & -58.74 & -33.85 \\ & OOD & -35.31 & \underline{-12.05} & -26.91 & -7.85 & -33.68 & \underline{-17.66} & -29.82 & -13.89 \\ \hline \end{array}

To address your concern about whether increasing the buffer size from 500 to 2000 affects the performance of our method (‘Challenging’) compared to the ‘Random’ policy, we have repeated the experiments in Table 4 with a buffer size of 2000. The updated results are as follows:

\begin{array}{c|c|c|ccc|c|ccc|} \hline Buffer & & 500 & & 500 & & 2000 & & 2000 & \\ \hline Balanced Class/Task & & - & & \checkmark & & - & & \checkmark & \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 23.34 & \underline{28.12} & 14.23 & 29.27 & 37.63 & \underline{40.89} & 23.42 & 42.24 \\ & OOD & 11.61 & \underline{13.65} & 7.20 & 14.81 & 17.02 & \underline{18.95} & 11.01 & 20.57 \\ \hline BWT & i.i.d. & -58.22 & \underline{-47.61} & -65.47 & -41.78 & -49.98 & \underline{-27.73} & -53.13 & -24.69 \\ & OOD & -34.38 & \underline{-23.86} & -33.04 & -18.38 & -35.31 & \underline{-12.05} & -26.91 & -7.85 \\ \hline \end{array}

From these results, it is evident that our method (‘Challenging’) continues to outperform the ‘Random’ policy across all metrics, even with the increased buffer size. Thank you for pointing out the importance of evaluating the effect of a larger buffer size on the performance comparison between the 'Challenging' and 'Random' policies. This additional analysis strengthens the robustness of our proposed approach and confirms that the superiority of ‘Challenging’ is not limited to smaller buffer sizes. We hope this addresses your concern thoroughly.

Thank you for your insightful comment regarding the need to evaluate the standalone effectiveness of Adaptive Replay Buffer Management without proxy-based contrastive learning.

In response, we have repeated the experiments presented in Table 4 by incorporating our adaptive replay buffer management into ER, excluding proxy-based contrastive learning. The results are as follows:

\begin{array}{c|c|c|ccc} \hline Balanced Class/Task & & - & & \checkmark \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 18.58 & \underline{19.30} & 12.49 & 20.44 \\ & OOD & 3.36 & \underline{3.42} & 2.78 & 3.51 \\ \hline BWT & i.i.d. & -77.19 & \underline{-73.69} & -81.20 & -72.14 \\ & OOD & -21.21 & \underline{-20.20} & -21.22 & -19.99 \\ \hline \end{array}

These experiments were conducted on Split CIFAR-100 with a buffer size of 1000. As shown, our buffer management consistently enhances the performance of ER across all metrics, demonstrating the effectiveness of our adaptive update policy.

References

[1] Online Continual Learning with Maximally Interfered Retrieval, NeurIPS 2019.

[2] iCaRL: Incremental Classifier and Representation Learning, CVPR 2017

[3] GDumb: A Simple Approach that Questions Our Progress in Continual Learning, ECCV 2020

[4] PyCIL: A Python Toolbox for Class-Incremental Learning, Arxiv.

[5] Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning, CVPR 2023

[6] On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486, 2019.

[7] Online Class-Incremental Continual Learning with Adversarial Shapley Value, AAAI 2021.

[8] Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning, CVPR 2021

[9] Exploring example influence in continual learning, NeurIPS 2022.

[10] Regularizing second-order influences for continual learning, CVPR 2023.

[11] Dark experience for general continual learning: a strong, simple baseline, NeurIPS, 2020.

[12] Classincremental continual learning into the extended der-verse, PAMI 2022.

[13] Class-Incremental Learning: A Survey, Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, Ziwei Liu.

Thank you for your insightful comment. We appreciate the opportunity to clarify the differences between our approach and the MIR method [1].

In continual learning, buffer management involves two critical phases: buffer update and buffer retrieval. The buffer update phase determines which samples from previous tasks should be stored in the buffer, while the buffer retrieval phase selects samples from the buffer to train alongside current task data.

Our approach specifically focuses on the buffer update phase: we populate the buffer with the most variable samples from previous classes. For buffer retrieval during training, we use a random sampling policy, akin to Experience Replay (ER) [6]. This means we do not introduce any special selection strategy for retrieving samples during training.

In contrast, the MIR method [1] adopts a different focus. MIR does not apply a specific strategy for buffer updates and uses reservoir sampling to populate the buffer, similar to ER [1] [7]. However, MIR implements a targeted strategy for buffer retrieval by selecting samples that are most interfered with—i.e., those whose predictions are most negatively impacted by expected parameter updates. This retrieval mechanism aims to prioritize replaying samples that are most vulnerable to interference from new tasks. As discussed in [7], selecting samples based on a significant increase in loss can lead to redundancy since similar samples in the latent space may all exhibit increased loss, potentially reducing diversity in the training data.

The following statements from the MIR paper [1] support our claim: “In this work, we consider a controlled sampling of memories for replay.” “We restrict ourselves to the use of reservoir sampling for deciding which samples to store.”

In summary, our approach focuses on buffer update rather than buffer retrieval, which is the opposite of the approach in MIR. Additionally, our method and MIR differ in how they select informative samples: we prioritize high-variance samples, while MIR selects samples based on the increase in loss projected from the anticipated parameter updates with new data. As illustrated in [7], [8], [9], and [10] the performance of MIR on Split Mini-ImageNet and Split CIFAR-100 is nearly close to the ER method, highlighting that their targeted retrieval strategy does not always lead to significant improvements over random sampling.

Thank you for pointing this out. We appreciate the opportunity to clarify the discrepancies you highlighted regarding the results in Table 1 compared to those reported in [4] and [5].

As mentioned in lines 292 and 348, for a fair comparison, we modified the Mammoth framework as outlined in [11, 12], similar to how MetaSP and SOIF [9, 10] modified it. By following the SOIF and MetaSP, our results are directly comparable to theirs. For example, references [9, 10] for Experience Replay (ER) on Split CIFAR-100 with a buffer size of 1000 achieved an ACC of 17.56%, while we achieved 18.58%. Moreover, MetaSP [9] and SOIF [10] for their corresponding methods on Split CIFAR-100 with a buffer size of 1000 reported ACC of 25.72% and 27.99%, respectively. We reported ACC of 25.15% and 26.19% for them, respectively. This indicates that accuracy differs across frameworks based on certain elements, and in the Mammoth framework, our results fall within this expected range. Meanwhile, several factors influence the reported performance, resulting in lower accuracy compared to [4] and [5]. So we would like to elaborate on some of these factors:

Different backbone networks, hyperparameters (e.g., number of epochs, memory batch size), and validation metrics can significantly affect accuracy. Following [9, 10, 11, 12], we used ResNet-18 (line 349), set the number of epochs per task to 50 (line 353), a memory batch size of 32 (line 351), and averaged the ACC over five fixed seeds (line 350). We applied standard data augmentations including random crop and random horizontal flip (mentioned in line 355).

In [4] (as evident in the full version of [4] and their corresponding GitHub code; [13]), they set the number of training epochs to 170, used a memory batch size of 128, and utilized ResNet-32 as the backbone network. Moreover, they reported the Top-1 accuracy and used additional techniques like herding and class balance for the buffer, along with color jitter in addition to random crop and horizontal flip.

In [5], their naive method differs from the standard ER. They updated the buffer after each task with balanced class random sampling, employed normalization techniques, set the number of epochs per task to 160, and used ResNet-32 as the backbone network.

The standard ER uses reservoir sampling, where the buffer is updated as each batch arrives with random samples, leading to potential class imbalances. Also, ER uses random retrieval.

Given these differences, it is understandable that our reported accuracy is lower than those reported in [4] and [5] for both ER and our method compared to DER.

We hope this explanation helps clarify the differences in accuracy between our work and the baselines mentioned. We believe that despite these discrepancies, our modifications to the Mammoth framework provide a fair comparison with related methods, and our results demonstrate competitive performance within the context of our chosen settings. However, if you feel that including results on ResNet-32 is essential, we are open to conducting the necessary experiments and reporting them in the camera-ready version, though this will require significant time.

Regarding your concern about the image sizes, we have conducted experiments using the original 84×84 image size of Split Mini-ImageNet without resizing, in addition to resizing the images to 32×32. Due to computational constraints, we were unable to perform experiments on the full ImageNet dataset or the non-resized ImageNet-100. Consequently, we stay focused on Split Mini-ImageNet to address your concern about image sizes. Additionally, due to the limited time available during the rebuttal phase, these experiments have been conducted with a buffer size of 500 and seed 0. If this is insufficient, we are prepared to conduct experiments across all buffer sizes and average the results over five fixed seeds (0 to 4) as done in the paper and report them in the camera-ready version. The experimental results are summarized below:

\begin{array}{c|c|c|cccccccc} \hline & Method & & GEM & A-GEM & ER & GSS & GDUMB & HAL & MetaSP & SOIF & Ours \\ \hline & ACC & i.i.d. & 17.60 & 14.71 & 16.64 & 18.77 & 7.17 & 4.80 & \underline{19.70} & 18.33 & 26.19 \\ \boldsymbol{32 \times 32}& ACC & OOD & 2.93 & 2.65 & 3.23 & 3.33 & 1.42 & 2.38 & \underline{3.35} & 3.20 & 11.97 \\ & BWT & i.i.d. & -52.49 & -66.79 & -67.04 & -62.00 & - & -38.79 & -65.34 & -65.46 & \underline{-46.30} \\ & BWT & OOD & -9.85 & -12.85 & -12.91 & -11.59 & - & -11.31 & \underline{-11.03} & -12.81 & -23.80 \\ \hline \hline & ACC & i.i.d. & \underline{25.36} & 16.34 & 19.53 & 20.97 & 8.8 & 7.61 & 24.59 & 20.37 & 33.45 \\ \boldsymbol{84 \times 84} & ACC & OOD & 3.92 & 3.41 & 3.74 & 3.43 & 1.73 & 2.97 & \underline{4.16} & 3.37 & 17.38 \\ & BWT & i.i.d. & -64.59 & -73.14 & -76.74 & -71.86 & - & \underline{-45.1} & -69.91 & -75.35 & -34.65 \\ & BWT & OOD & -11.59 & -13.03 & -13.01 & -12.55 & - & \underline{-11.46} & -10.83 & -13.34 & -23.91 \\ \hline \end{array}

The data show that increasing the image size not only does not degrade performance but improves our algorithm's advantage. For the 32×32 images, we outperform the second-best on ACC (i.i.d.) and ACC (OOD) by margins of 6.49% and 8.62%, respectively. Regarding BWT (i.i.d.) (32×32), the HAL method has the lowest ACC (i.i.d.) (4.80%), leading to the best BWT (i.i.d.) due to insufficient learning (i.e., nothing to forget). Our method achieves the second-best BWT (i.i.d.), and for a fair comparison, BWT should be considered alongside ACC (as discussed in Section 4.2 of the paper). For example, compared to GEM, which has a high ACC (i.i.d.) (17.60%) and the third-best BWT (i.i.d.), we outperform by a margin of 6.19%. Also, for MetaSP, which has the second-best ACC (i.i.d.) (19.70%), we outperform it by a margin of 19.16% in BWT (i.i.d.). In the OOD scenario, the ACC of all baselines is too low, resulting in better BWT due to insufficient knowledge capture.

When increasing the image size from 32×32 to 84×84 (original Mini-ImageNet size), our margin of improvement increases further. For ACC (i.i.d.) and ACC (OOD), we outperform the second-best by margins of 8.09% and 13.22%, respectively. Regarding BWT (i.i.d.), we outperform the second-best by 10.45%. Also, for GEM, which has the second-best ACC (i.i.d.) (25.36%) and the third-best BWT (i.i.d.), we outperform it by a margin of 29.94% in BWT (i.i.d.). Again, for the BWT (OOD) case, the ACC for all methods is too low, indicating that they have not acquired sufficient knowledge to retain, let alone forget. This raises concerns about their OOD generalization capabilities.

Regarding your concern about other task scenarios, as mentioned in the full version [13] of the referenced paper [4], there are two different ways to split the classes into incremental stages: 1. Train from Scratch (TFS) and 2. Train from Half (TFH). In [13], the authors state, "Both of these settings are widely adopted in the current CIL community.” As evident in Table 5 and Table 6 in the appendix of the full version of the paper [4], for almost all methods, the ACC for TFS is lower than the ACC for TFH. This is because having more stages of learning introduces greater challenges to retaining knowledge from earlier classes. In contrast, TFH presents a less severe challenge, as the model primarily fine-tunes its knowledge base to accommodate smaller increments of new classes after establishing a large initial set. Furthermore, since we followed the Mammoth framework, which uses the TFS approach, we adopted the same. However, if this remains a concern, we can prepare the results for TFH for the camera-ready version.

We are grateful for your valuable comments and hope that our responses have addressed your concerns. If you have any further concerns, please let us know. We look forward to hearing from you.

We appreciate your reviews. We hope that our responses have adequately addressed your concerns. As the deadline for open discussion nears, we kindly remind you to share any additional feedback you may have. We are keen to engage in further discussion.

Thank you for pointing out the need to clarify the concept of proxies in our method. The explanation on lines 182–184 may not have fully conveyed the rationale behind their use or their specific implementation in our approach. We provide the following clarification:

1. Definition of Proxy:
   In our method, proxies are representative weights that correspond to specific classes. Each proxy acts as a learned anchor point in the embedding space for its associated class, summarizing the class's characteristics. Instead of comparing all individual sample pairs, we use these proxies to establish relationships between classes and samples in the batch, significantly reducing computational complexity.

2. Role in Proxy-Based Contrastive Learning:
   The proxies are utilized in the contrastive learning objective to align each sample's embedding with its class proxy while distancing it from other class proxies. This is achieved through the proxy-based contrastive loss, defined in Eq. (4). This approach enables robust representation learning by focusing on proxy-to-sample relationships rather than exhaustive sample-to-sample comparisons, which can be computationally prohibitive.

3. Advantages of Using Proxies:

   • Efficiency: By reducing the need for pairwise comparisons, proxies enable faster and more stable convergence.
   • Noise Tolerance: Proxies are more robust to noisy or outlier samples, as they summarize the class characteristics rather than relying on individual samples.
   • Generalization: Proxies provide a smoother embedding space by focusing on class-level relationships.

We will revise the explanation in the manuscript (lines 182–184) to explicitly define proxies and their role in our approach. Additionally, we will enhance the connection between proxies and their impact on the computational and generalization benefits in Section 3.2. Thank you for this constructive feedback, which will help us improve the clarity of our manuscript.

References

[1] Mohammad Motamedi, Nikolay Sakharnykh, and Tim Kaldewey. A data-centric approach for training deep neural networks with less data. arXiv preprint arXiv:2110.03613, 2021.

[2] Mark Mazumder, Colby Banbury, Xiaozhe Yao, Bojan Karlas, William Gaviria Rojas, Sudnya Diamos, Greg Diamos, Lynn He, Alicia Parrish, Hannah Rose Kirk, et al. Dataperf: Benchmarks for data-centric ai development. arXiv preprint arXiv:2207.10062, 2022.

[3] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J Gordon. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159, 2018.

[4] Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. arXiv preprint arXiv:2009.10795, 2020.

[5] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15920–15930, 2020.

[6] Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Classincremental continual learning into the extended der-verse. IEEE transactions on pattern analysis and machine intelligence, 45(5):5497–5512, 2022.

[7] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019.

Thank you for bringing up this important point. The mechanism for organizing samples in the buffer without explicitly storing task labels is based on the structure of our proposed buffer. We understand that this may not have been adequately clarified in the manuscript. Allow us to explain:

1. Sequential Task Arrangement:
   The buffer is updated sequentially as tasks are encountered. This means that samples from earlier tasks naturally occupy the lower indices in the buffer, while samples from later tasks occupy higher indices. Thus, the task order is implicitly preserved in the buffer structure without requiring explicit task labels.

2. Class-Specific Organization:
   Within each task, the buffer is further organized by class. Samples belonging to the same class are stored together and ordered by their confidence variance ($ \sigma^2 $). This eliminates the need to store task labels or $ \sigma^2 $ explicitly because the buffer structure inherently reflects the task and class arrangement.

3. Efficient Access Without Task Labels:
   To identify task-specific samples, the buffer leverages its index-based organization. Samples are accessed based on their position in the buffer, which corresponds to their task and class membership due to the sequential and class-based storage scheme.

To improve the clarity of this explanation, we will revise the corresponding lines in the manuscript to explicitly describe how the buffer organization allows task and class separation without storing task labels. Thank you for pointing out this opportunity for enhancement.

Thank you for your comment. We acknowledge that the explanation of the comparison methodology could have been clearer in mentioned lines 89-90. The implementation details for the comparison are already included in the manuscript (Section 4.1 SETUP) but may not have been sufficiently emphasized in mentioned lines 89-90. To clarify:

1. Setup for Comparison:
   All methods were implemented under identical conditions using the Mammoth framework as described in [5] and [6]. The training was conducted under class-incremental settings, as done in previous studies [6], on the Split CIFAR-100, Split Mini-ImageNet, and Split Tiny-ImageNet datasets, using buffer sizes of 500, 1000, and 2000. Detailed training conditions and hyperparameters were outlined in the Datasets and Implementation Details paragraphs. Results were averaged over five independent runs to ensure fairness.

2. OOD Sample Generation (Paragraph: Datasets):
   OOD samples were generated following the methodology of [7]. This involved applying standard corruptions, such as Gaussian Noise, Motion Blur, Elastic Transform, and so on, to the test datasets. These corruptions simulate real-world distributional shifts, providing a robust assessment of OOD generalization.

3. Evaluation Metrics (Paragraph: Metrics):
   Average Accuracy (ACC) and Backward Transfer (BWT), defined in Eq. 9, were used to evaluate performance. ACC measures overall task performance, while BWT quantifies the extent of forgetting. These metrics, crucial for assessing i.i.d. and OOD scenarios, were discussed in detail in the Metrics Paragraph.

4. Findings in Fig. 1:
   Fig. 1 shows the average ACC over three buffer sizes in the OOD scenario for each dataset. The results highlight the substantial performance degradation of existing methods on OOD samples compared to i.i.d. scenarios.

To improve clarity, we will enhance the manuscript by cross-referencing these paragraphs and adding some of the above explanations directly to lines 89–90. We hope this addresses the concern and improves the reader's understanding of the methodology. Thank you again for your valuable feedback.

In Table 4, we initially demonstrated various buffer update policies, including “Reservoir,” “Hard,” “Challenging,” and “Random.” However, a discrepancy in the experimental setup led to the “Random” policy using augmented versions of samples in the buffer, whereas the other policies stored original (non-augmented) samples. This setup inadvertently favored the “Random” policy, resulting in better BWT compared to our proposed “Challenging” policy. We understand how this might raise concerns about the effectiveness of our method in maintaining decision boundaries for old tasks. To address this, we conducted a more equitable comparison by ensuring all policies stored original (non-augmented) samples in the buffer. Furthermore, while the original Table 4 was limited to a buffer size of 500 on the Split CIFAR-100 dataset, we extended the evaluation to include buffer sizes of 1000 and 2000. The updated results, presented in the table below, also include the mean performance across all buffer sizes.

\begin{array}{c|c|c|ccc|c|ccc|} \hline Buffer & & 500 & & 500 & & 1000 & & 1000 & \\ \hline Balanced Class/Task & & - & & \checkmark & & - & & \checkmark & \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 23.34 & \underline{28.12} & 14.23 & 29.27 & 33.03 & \underline{33.97} & 18.49 & 36.06 \\ & OOD & 11.61 & \underline{13.65} & 7.20 & 14.81 & 14.98 & \underline{16.11} & 8.75 & 17.86 \\ \hline BWT & i.i.d. & -58.22 & \underline{-47.61} & -65.47 & -41.78 & -48.70 & \underline{-36.98} & -57.61 & -35.08 \\ & OOD & -34.38 & \underline{-23.86} & -33.04 & -18.38 & -31.35 & \underline{-17.07} & -29.51 & -15.44 \\ \hline \end{array}

\begin{array}{c|c|c|ccc|c|ccc|} \hline Buffer & & 2000 & & 2000 & & Mean & & Mean & \\ \hline Balanced Class/Task & & - & & \checkmark & & - & & \checkmark & \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 37.63 & \underline{40.89} & 23.42 & 42.24 & 31.33 & \underline{34.33} & 18.71 & 35.86 \\ & OOD & 17.02 & \underline{18.95} & 11.01 & 20.57 & 14.54 & \underline{16.24} & 8.99 & 17.75 \\ \hline BWT & i.i.d. & -49.98 & \underline{-27.73} & -53.13 & -24.69 & -52.30 & \underline{-37.44} & -58.74 & -33.85 \\ & OOD & -35.31 & \underline{-12.05} & -26.91 & -7.85 & -33.68 & \underline{-17.66} & -29.82 & -13.89 \\ \hline \end{array}

The updated results demonstrate that across all buffer sizes, metrics (ACC, BWT), and scenarios (i.i.d., OOD), our “Challenging” policy consistently outperforms the “Random” policy, which achieves the second-best performance. This indicates that under fair experimental conditions, our policy is highly effective at capturing better decision boundaries. To further validate the generalizability of our buffer update policy, we integrated it into the ER method, replacing its random sampling strategy. The results, shown below for a buffer size of 1000 on the Split CIFAR-100 dataset, reaffirm that our policy enhances performance regardless of the loss function used.

\begin{array}{c|c|c|ccc} \hline Balanced Class/Task & & - & & \checkmark \\ \hline Memory Update Policy & & Reservoir & Random & Hard & Challenging \\ \hline ACC & i.i.d. & 18.58 & \underline{19.30} & 12.49 & 20.44 \\ & OOD & 3.36 & \underline{3.42} & 2.78 & 3.51 \\ \hline BWT & i.i.d. & -77.19 & \underline{-73.69} & -81.20 & -72.14 \\ & OOD & -21.21 & \underline{-20.20} & -21.22 & -19.99 \\ \hline \end{array}

As shown, our buffer update policy improves ACC and reduces forgetting (BWT) in both i.i.d. and OOD scenarios, even when integrated with the ER method. This demonstrates its effectiveness as a plug-and-play component for buffer-based methods using random update policies, beyond the context of our PCL loss. We hope this additional analysis addresses your concerns and highlights the robustness and utility of our proposed buffer update mechanism.

Thank you for highlighting this important question about the robustness of confidence variance and the implications of limiting its calculation to the first $E$ epochs.

To identify the most informative samples of each class, we utilized a data-centric artificial intelligence approach. In data-centric AI, the emphasis is on improving datasets to facilitate effective model training with fewer data requirements [1, 2]. For instance, as cited in our paper, [3] and [4] leverage model predictions during training to refine datasets. These studies demonstrate that early-stage uncertainty is a reliable indicator of a sample’s contribution to the training process. By analyzing network predictions for individual samples during the initial training epochs, they identify noisy labels and outlier samples, aiding dataset optimization.

As you noted, during the training process, the decision boundary evolves. The key question is: Which boundary leads to better generalization? One shaped by outliers or one influenced by genuinely challenging (most informative) samples?

To address this, we analyzed CIFAR10 in batch training (training all 10 classes at once, resembling one task of Split CIFAR-100 under continual learning). We calculated the target confidence of each sample at every epoch and displayed the mean and variance of sample confidence across these epochs in the Figure linked below. In the left sub-figure, we consider the first $E$ epochs $E = 5$ as in our paper, and in the right sub-figure, we use 50 epochs (matching the epoch count per task in our paper).

Please click to view the Figure

We categorized the samples observed during the first $E = 5$ epochs into three groups:

• Area 1. Simple Samples: High confidence with low variance, consistently well-classified and at the center of the distribution.

• Area 2. Challenging Samples (Most Informative): Moderate mean confidence with high variance, residing near the decision boundary. They are ideal for shaping a robust boundary.

• Area 3. Hard (Outlier) Samples: Low confidence with low variance, far from the decision boundary, offering limited value for boundary refinement.

The bottom sub-figure shows representative images from these areas (Area 1, Area 2, and Area 3) for each CIFAR10 class. Each column represents a class, with rows corresponding to simple, challenging, and hard samples, respectively. Simple samples display clear class attributes (e.g., a horse on grass; grappling with the effects of spurious correlation). Challenging samples exhibit ambiguous traits (e.g., an airplane against a gray backdrop; enhancing OOD generalization), while hard samples feature unusual or rare characteristics (e.g., a horse’s head only), often functioning as outliers.

As training progresses from $E = 5$ to $E = 50$:

• Simple Samples retain high confidence and low variance, remaining at the center of the distribution.

• Challenging Samples show increasing confidence (mean confidence rises) and decreasing variance as the model learns them robustly. Most migrate from the boundary to the center, while a few persist near the boundary.

• Hard Samples exhibit higher confidence but also increased variance, eventually achieving moderate confidence and defining the boundary.

Thus:

• Area 4 ($E = 50$) contains simple samples and most challenging samples.

• Area 5 ($E = 50$) contains hard samples and some challenging samples, shaping the boundary.

Comparing Area 2 ($E = 5$) and Area 5 ($E = 50$), we argue that Area 2, with challenging samples, provides better generalization. By prioritizing these challenging (most informative) samples for buffer selection, the decision boundary is shaped by meaningful boundary samples rather than outlier samples. This strategy "reminds" the model of an ideal boundary in future tasks, promoting generalization and stability-plasticity balance.

In Figure 4 of the Appendix, we analyze the sensitivity of the hyperparameter $E$. The results show stable performance across a range ($E = 2$ to $7$) on all datasets, maintaining consistency in ACC and BWT under i.i.d. and OOD conditions. These findings confirm that identifying boundary samples requires only a few initial training epochs.

Additionally, as shown in the Figure linked below, we conducted another experiment on CIFAR-100, extending the range of $E$ from 2 to 14. The results demonstrate that performance gradually decreases beyond $E = 7$. This is because outliers transition to challenging samples, and challenging samples to simple ones, reducing their impact on refining the decision boundary.

Please click to view the Figure

Thank you for your insightful comment We appreciate the opportunity to clarify your concerns. To address your concern regarding the difference between the proposed Proxy-based Contrastive Loss (PCL) and a standard cross-entropy loss with temperature-scaled softmax, we can focus on the following distinctions:

Key Differences:

1. Proxy-to-Sample Relationships vs. Sample-to-Proxy Relationships:

   • PCL: It explicitly models the relationship between samples and their corresponding proxies (representative class weights). Each sample interacts not only with its own proxy but also with proxies of all other classes, enabling the alignment of samples with their class prototypes while pushing them away from other class prototypes. This proxy-to-sample approach improves generalization by ensuring class separation and robustness to noise.
   • Standard Cross-Entropy with Temperature-Scaled Softmax: It calculates the probability of each class for a sample using the dot product of the sample's embedding and class weights (proxies). The loss only optimizes the alignment of each sample to its true class, without directly incorporating explicit inter-class repulsion.

2. Contrastive Learning Elements:

   • PCL: Inspired by contrastive learning, PCL focuses on expanding the pool of negative samples (all non-class proxies act as negatives) while also ensuring that positive alignments (with the class proxy) are strengthened. This approach is rooted in a proxy-based contrastive objective that draws samples closer to their proxies while distancing them from others.
   • Standard Cross-Entropy: It does not explicitly include a mechanism for ensuring separation from non-class embeddings, which may lead to overlap in feature space, especially in high-dimensional settings.

3. Handling Class Imbalance and Representation:

   • PCL: Designed to handle imbalanced datasets more effectively by operating at the proxy level. This ensures that rare classes or underrepresented tasks are given equal importance in learning robust boundaries.
   • Cross-Entropy: It may suffer from overfitting to dominant classes in imbalanced settings because the optimization focuses directly on sample-level probabilities without leveraging the global class relationships.

4. Computational Benefits:

   • PCL: Proxy-based computation reduces the need for detailed pairwise comparisons across all samples (a common drawback of standard contrastive loss). This makes it computationally efficient compared to full contrastive learning while maintaining the benefits of sample-to-proxy relationships.
   • Cross-Entropy with Temperature Scaling: While computationally efficient, it lacks the ability to encode contrastive relationships or ensure well-separated class boundaries in the latent space.

Summary: The proposed PCL loss fundamentally differs from standard cross-entropy loss with temperature-scaled softmax by integrating proxy-based contrastive learning principles. Specifically, PCL:

• 1. Employs proxy-to-sample relationships to optimize embeddings for both alignment with class prototypes and separation from other class proxies.
• 2. Improves robustness to noise and class imbalance by ensuring a structured representation space with explicit inter-class repulsion.
• 3. Offers computational efficiency compared to full contrastive learning by avoiding pairwise sample comparisons while still leveraging a contrastive mechanism.

These distinctions allow PCL to achieve better Out-of-Distribution (OOD) generalization and stability-plasticity trade-offs compared to standard cross-entropy loss, as evidenced by our experimental results in Section 4.2.