Imbalanced data robust online continual learning based on evolving class aware memory selection and built-in contrastive representation learning

Thank you for your valuable suggestion.

$Q1$: The proposed method appears to be more general rather than specifically tailored to the task... Our method is specifically engineered for the nuanced demands of online continual learning, especially in scenarios with imbalanced data. It addresses critical challenges such as shifting class dynamics and evolving domain characteristics, which are prevalent in such environments. our method incorporates a novel memory selection strategy, named Feature-Distance Based Sample Selection (FDBS), which effectively evaluates and adapts to the changing representational significance of new data in relation to the existing memory set. This approach is crucial for maintaining a relevant and adaptive memory set in the face of continuous data evolution. Additionally, the integration of a unique Contrastive Loss (IWL) aids in refining the feature space, enhancing the model's ability to compact intra-class data while distinctly separating inter-class data. This dual strategy of memory selection and feature space refinement is what sets MSCL apart, making it a tailored solution for the complexities of online continual learning in imbalanced data scenarios. In particular, our proposed memory selection strategy FDBS can be associated with any state of the art rehearsal-based CL methods to improve its performance and robustness to unbalanced data streams.

$Q2$: Computing the distance of all samples at each time step is time-consuming... please see the response $Q1$ to Reviewer b5Cn.

$Q3$: The method focuses on identifying samples with high intra-class... We acknowledge the validity of your comment regarding our method's approach to memory selection. Our strategy focuses on retaining samples that exhibit high intra-class variance and inter-class similarity, effectively capturing the most distinctive features of each class. However, this approach might inadvertently prioritize samples that are outliers or far from the data center, potentially leading to an over-representation of out-of-distribution (OOD) data in the memory set. This scenario could adversely impact the model's performance, especially in datasets with a significant presence of OOD data.

To mitigate this issue, we implemented a comparison with random selection in our methodology, serving as a control to assess the effectiveness of our approach in managing OOD data. We recognize that distinguishing between OOD data and essential new information is a complex yet critical aspect of continual learning. This challenge underscores the need for continual learning models to be adept at discerning and adapting to the evolving nature of data distributions, effectively balancing the retention of valuable information while minimizing the influence of irrelevant data.

$Q4$: The baseline methods used for comparison seem somewhat outdated, with the most recent one dating back to 2022. Our method primarily focuses on the management of the replay buffer which is the cornerstone of rehearsal-based CL methods and the underlying memory selection strategy is fundamental to deal with non stationary data streams. As such, the proposed memory selection strategy FDBS is orthogonal to state of the art rehearsal-based CL methods as it can be combined with any of them to improve their performance and robustness to unbalanced data. This is why we have chosen to compare it mainly with other memory selection methods. Furthermore, in A.7 (p15) of our submission as well as in our reply to Reviewer ddu2, we have further showcased the benefits of the proposed memory selection strategy when combined with two state of the art rehearsal-based CL methods in improving their performance and robustness to unbalanced data. Therefore, we believe the methods selected for comparison are representative and effectively demonstrate the efficacy of our approach.

Thank you for your valuable suggestion.

$Q1$ : There is no discussion of the computational cost of the algorithm. Our approach primarily incurs computational costs in calculating the matrix distance. For each input batch, it's necessary to compute the distance between the batch and the memory set. To optimize this process, our method employs the following strategies:

• Efficient Distance Matrix Computation: We do not recalculate the entire distance matrix $D$ (the distance between memory points) for each input data. Instead, we update only the parts associated with retrieved memory points. This approach significantly reduces the computational overhead.

• Linear Memory Cost: The memory cost scales linearly with the size of the memory set, avoiding exponential increases.

• Computational Cost Evaluation:

To provide a clearer picture of the computational efficiency, we conducted tests on a single RTX3070 using ResNet18 with a memory size of 1000 in a balanced PACS setting. The results are as follows:

As illustrated in the table, our method requires more computation time than ER and OCS, but is less computationally intensive than GSS. This positions our method as a viable option in terms of computational efficiency, particularly when considering the enhanced performance it offers in more complex and realistic learning scenarios.

$Q2$ : The authors mentioned Split ImageNet-1k. Sorry, there was a typo error. It should be the split mini-ImageNet.

$Q3$ : The authors acknowledge the substantial difficulties encountered when dealing with imbalanced data across various classes ... Our method's adaptability to both balanced and imbalanced scenarios, and its scalability in terms of memory size, demonstrate its potential applicability in large-scale scenarios. Although we did not explicitly test on extremely large datasets due to resource constraints, the principles underpinning our method – particularly the efficient memory management and contrastive learning components, are designed to be scalable and applicable in large-scale settings. The scalability of our method is demonstrated by its application from the smaller PACS dataset to the larger DomainNet dataset.

$Q4$: By continuously refining the feature space and adjusting the memory set based on incoming data... In our research, we utilized the PACS dataset, which comprises four distinct domains: Photo, Art-painting, Cartoon, and Sketch. Notably, the Sketch domain presents a significantly different data profile compared to the other three. This disparity allows us to simulate a scenario of rapid distribution change, particularly noticeable during the transition to training on the Sketch domain. We observed that all tested methods, including ours, initially experience a performance drop in this phase. However, our method demonstrates a quicker recovery in accuracy than others, highlighting its adaptability to such abrupt changes in data distribution.

We believe that rapid distribution changes pose a unique challenge: the new data can either be viewed as out-of-distribution (OOD), which may not be relevant and thus could be ignored, or as vital new information that should be incorporated into the model. This dichotomy is crucial in continual learning environments.

Our approach specifically encourages the storage of data near the decision boundary and those instances within a class that exhibit the most significant differences. While this strategy is effective in capturing diverse data representations, it also raises the possibility of incorporating more OOD data into the memory set. This inclusion of OOD data could potentially interfere with the model's ongoing learning process, posing a challenge that we aim to address in future work.

Thank you for your valuable suggestion. We fully understand the importance of including the latest methods in our experiments to enhance the credibility and relevance of our work. However, we would like to highlight the differences in terms of the positioning and focus of our method compared to OnPro[1], GSA[2], and OCM[3], although these works represent important contributions to the field of online class incremental learning.

1. Distinct Scenarios and Setting: OnPro[1], GSA[2], and OCM[3] have brought significant contributions on continual learning but they are primarily proposed under class incremental learning scenario and tested on the Split CIFAR100 and Split mini-Imagenet under data balanced setting. For instance, in section 3.1 on problem definition, OnPro[1] assumes that different tasks D_t are mutually disjoint in terms of classes. Therefore, their method assumes that learning data for a given class can only arrive once through a number of successive batches within a given task. Such an assumption moves away from the original goal of continual learning where the learned knowledge is supposed to be consolidated anytime, including already learned concepts. In contrast, our paper does not make such an assumption and proposes a novel online continual learning method operating under much more diverse and challenging scenarios, including Domain-incremental, Class-incremental, and Domain-Class incremental learning under both balanced and imbalanced settings. This broader scope addresses a wider range of practical applications and challenges in the field.
2. Methodological Differences: our method primarily concentrates on the efficient management and utilization of the memory buffer which is the cornerstone of all rehearsal-based CL methods. This approach is fundamentally orthogonal to the methods mentioned above, emphasizing different aspects of the learning process. Therefore, it is more appropriate for our method to be compared with other memory selection methods, which align more closely with our research's core focus:
   • OnPro[1] does a great work in feature learning to avoid shortcut learning, using in particular the contrastive learning loss on prototypes from the memory set and the current batch and data augmentation. It employs Experience Replay (ER) to select memory samples. In contrats, our method introduces a new, more efficient memory selection technique compared to ER. Their method does a great work on calculating the contrastive prototype learning loss within the memory set and within current batch. In contrast, our method generates the contrastive learning loss between the memory set and current batch without data augmentation.
   • GSA[2] introduces a new training objective with a gradient-based self-adaptive loss to tackle gradient imbalance (GI).
   • OCM[3] maximizes mutual information between input data and holistic feature representations, using local and global rotations for data augmentation.

Experimentally, we have shown in our paper that our memory selection method (FDBS) can be combined with any other rehearsal-based CL methods to obtain improved performance and robustness to unbalanced data. In the following section, we further showcase this property of complementarity of our proposed memory selection method when it is combined with OnPro[1].

3. Comparative Analysis with OnPro[1]:
   • In balanced Class-incremental Learning (CIL) on CIFAR100, OnPro shows superior performance with a 35.98% accuracy compared to our 21.13%. This result suggests the effectiveness of feature learning through online prototype learning in OnPro[1] under data balanced setting;
   • In the more complex and realistic scenario of imbalanced Domain-incremental Learning (DIL) on the PACS dataset, our proposed method (FDBS+IWL) outperforms OnPro, achieving 31.25% accuracy against their 26.54%.
   • If we replace OnPro's memory selection method with our FDBS, we observe that in a balanced scenario, the resultant method increases slightly the performance. However, in the context of imbalanced domain-incremental learning, combining OnPro with our method achieves the best performance, further demonstrating the effectiveness and the versatility of our memory selection approach.

These results along with those in the submission across different scenarios suggest that OnPro and similar methods are highly specialized for balanced CIL. In contrast, our proposed method, focusing on efficient memory management, demonstrates greater versatility and effectiveness in these diverse and more challenging learning environments, and its ability to be combined with any rehearsal-based CL method.