Saliency-driven Experience Replay for Continual Learning

We thank the reviewer for their insight. We first address the major weaknesses (indicated with W1 and W2) identified by the reviewer and then respond, point by point, to their raised questions (indicated with Q1 and Q2). We will also review the whole paper to correct the identified typos.

W1 - Novelty

We acknowledge that prior work, specifically Liu et al. (2024), has introduced the use of saliency for exemplar-free class incremental learning. However, as mentioned in our paper (lines 96-105), Liu et al. (2024) employ a static, pre-trained saliency detector, thus they do not demonstrate (hence do not use) the forgetting-free capabilities of saliency prediction, since it is not continuously trained. In contrast, our method continuously trains the visual saliency network, which reduces forgetting by adapting classification features to new data. Furthermore, SER provides a more flexible and generalizable saliency-classification paradigm that adapts to any dataset without external dependencies, as opposed to Liu et al. (2024), which, instead, requires the use of a pre-trained saliency detector trained on the same data distribution as the target data. We believe these enhancements represent significant contribution to ongoing research in class-incremental learning. Finally, Liu et al. (2024) is not suitable for online continual learning because it is trained for a large number of epochs (100) and uses 50% of the classes in the first task, distributing the remaining classes across subsequent tasks. In our online continual learning setting, where 100 classes are divided into 20 tasks, we found that it achieved very low performance ( around 6%) as reported in Tab. R1 in the pdf attached to the global rebuttal.

W2 - Model agnostic

It is correct that SER requires the saliency encoder to share the same architecture as the classifier. However, we want to clarify that this requirement does not compromise the model-agnostic nature of our approach. Unlike methods that depend on pre-existing saliency predictors (such as the work by Liu mentioned by the reviewer), SER does not necessitate frozen saliency models as it trains saliency prediction continuously alongside the classifier. This integrated learning strategy involves instantiating an additional instance of the classifier, attaching a decoder, and training the entire system (classifier and saliency predictor) jointly. This can be done given that forgetting-free nature of saliency prediction. This learning design ensures that SER remains adaptable to various model architectures, preserving its model-agnostic characteristic while providing an effective solution for simultaneous saliency prediction and classification.

As for the compatibility of SER with various continual learning methods, the reviewer correctly points out that SER may not align well with some methods, like L2P, and this is a crucial distinction to address. Prompt learning approaches, such as L2P, operate under a different paradigm where the encoder remains static during continual learning. These methods leverage a pre-trained encoder's global knowledge of the visual world, using learnable prompts to identify the relevant sections of the huge latent space that address specific tasks. This is fundamentally different from the approach taken by SER. SER is tailored for scenarios where models undergo continuous training and learning from scratch. SER design ensures that classification features are dynamically modulated by saliency features throughout the learning process. Such dynamic modulation is essential for the model to adapt continually to new tasks and visual stimuli, which is critical for maintaining high performance and robustness in ever-changing environments. Training from scratch allows the model to develop hierarchical feature representations from the ground up, closely mimicking the visual cortex's processing mechanisms.

Thus, while the reviewer is correct that SER cannot be applied to every continual learning method, it is indeed well-suited for models which are trained from scratch. Accordingly, we will reformulate the claims about the applicability of SER to every continual learning method.

Response to Questions:

Q1 - What is the novelty?

Please refer to our response to W1.

Q2 - Generalization of SER to architectures other than ResNet-18

We extended our experiments to include additional backbones such as ResNet50, MobileNet V2, and DenseNet-121 (see Table R2 of the attached pdf). In all cases, SER improves the performance of the baseline, whether the classifier C is trained from scratch or with the same weights as the encoder of the saliency predictor S (we replicate the same setting as in Table 1 of our original manuscript). This demonstrates that SER remains effective across different architectures.

We thank the reviewer for the provided comments. In the following we respond to the raised questions (indicated with Q1, Q2, and Q3).

Q1 - Class-Incremental with Repetition

To clarify, in our approach, the tasks $D_i$ and $D_j$ are indeed considered disjoint. We acknowledge that our current description could be interpreted as suggesting otherwise, which is not our intention. We will amend the paper to explicitly state that tasks $D_i$ and $D_j$ are mutually exclusive within the same domain $Y$. The ‘Class-Incremental with Repetition’ setting represents a different scenario and could potentially simplify the problem. We recognize this as an interesting direction for future testing of our SER method (and we will include it in our paper), but our current focus remains on the disjoint class-incremental setting, which poses a more challenging problem.

Q2 - Clarification to Figure 4

We applied the Projected Gradient Descent attack ( [38] of the manuscript) to the ER-ACE + SER model, referred to as SER-ER-ACE and represented by the orange line (the label SAM-ER-ACE was a typo and will be corrected in the final manuscript) on the Split Mini-ImageNet dataset. We compared its performance against the baseline ER-ACE (green line). The figure shows the accuracy drop (in %) compared to the standard training performance (values reported in Table 1 of the manuscript) as the attack intensity $\epsilon$ increases. It is evident that the model equipped with SER experiences a significantly smaller drop and is better able to tolerate this attack.

Q3 - Clarification to Table 3

The values presented in Table 3 of the manuscript correspond to our custom benchmark using MiniImageNet. Specifically, we crafted this benchmark to evaluate the robustness of the SER strategy against spurious features and adversarial attacks. In this benchmark, we selected the first ten classes from MiniImageNet, organized into 5 tasks with 2 classes per task, and introduced spurious features by modifying the brightness of the training images with a class-dependent offset. The test images remained unaltered.

Regarding the reviewer’s concern about the requirement for the saliency encoder and classifier to have the same architecture, we would like to clarify that this does not compromise the model-agnostic nature of our approach. The saliency predictor in the SER strategy does not need to be an external model, but it can be built by creating an additional instance of the classifier, attaching a decoder, and jointly training it with the classifier. Thus, the architecture alignment is not a limitation, as the saliency predictor can be directly constructed using the continually trained classifier.

We thank the reviewer for their insight. We first address the weaknesses (indicated with W1, W2, and W3) identified by the reviewer and then respond, point by point, to their raised questions (indicated with Q1, Q2, Q3, and Q4).

W1 - Generalization to other models: To address this concern, we have extended our experiments to include additional backbones beyond ResNet-18. Specifically, we have tested our SER strategy with ResNet-50, MobileNet V2, and DenseNet-121. As reported in Table R2 in the PDF attached to the global rebuttal, in all cases, our SER approach leads to improved performance, thereby demonstrating its effectiveness across various architectures. Regarding the ViT backbone, please refer to our next response (to W2) where we provide both conceptual and practical reasons for excluding ViT from our analysis.

W2 - Generalization to transformer models: SER focuses on emulating visual cortex mechanisms, particularly object recognition via selective attention, which aligns more closely with Convolutional Neural Networks (CNNs) due to their similarity to the visual cortex’s hierarchical and localized processing. CNNs have local receptive fields and hierarchical stages similar to those in the primate visual cortex, as detailed by Hubel et al. (1962) and DiCarlo et al. (2007). Empirical studies show CNNs’ effectiveness in predicting neural responses and modeling visual encoding in primates (Yamins et al., 2016; Cadena et al., 2019), reinforcing their suitability for selective attention and low-level visual processing.

While Vision Transformer (ViT) excels in capturing global context and dynamic attention, it does not align with the hierarchical, localized processing of CNNs. Additionally, ViT requires large amounts of data and extensive training. Our experiments showed that ViT achieved only 1% accuracy on the Split-MiniImageNet benchmark, regardless of whether trained for 1 epoch or 50 epochs, underscoring its limitations in the context of online continual learning.

Hybrid solutions that combine CNN-based and Transformer-based models pose significant challenges due to the need for aligned semantic representation of features. SER relies on modulating classification features with saliency features. This modulation requires that the semantic representation of these features be coherent and aligned. Using hybrid solutions can disrupt this alignment, as CNNs and Transformers process and represent features differently.

Thus, our preference for CNNs over ViT and hybrid models is driven by the need to closely mimic visual cortex processing and by empirical evidence of low performance in OCL by ViT. We will revise our claims on SER’s applicability to continual learning methods.

• Hubel et al., 1962. “Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex,” Journal of Physiology.
• DiCarlo et al., 2007. “Untangling invariant object recognition,” Trends in Cognitive Sciences.
• Yamins et al., 2016. “Using goal-driven deep learning models to understand sensory cortex,” Nature Neuroscience.
• Cadena et al., 2019. “Deep convolutional models improve predictions of macaque v1 responses to natural images,” PLoS Computational Biology.

W3 - More detailed description of the method: We acknowledge the request for a more detailed description and will provide additional details in the revised manuscript.

Responses to Questions:

Q1 - SAM: It was a typo that will be corrected in the final version of the paper.

Q2 - Sensitivity of SER to hyperparameters: The SER hyperparameters include: 1) the $\lambda$ term that weights the loss component in Equation 3 of the manuscript, 2) the various saliency modulation strategies (ranging from saliency provided as input to the model, to layerwise summation, to layerwise multiplication), and 3) the specific layers to which SER modulation is applied. The latter two aspects have been thoroughly evaluated in the paper (see Fig. 3 and Table 2). Regarding the $\lambda$ term, we tested several values ($\lambda = 0.1, 0.5, 1, 1.5, 2$). Although the model's performance was relatively stable across these values, the best results were achieved with $\lambda = 1$. Additionally, the PDF attached to the global rebuttal details the various tested hyperparameters (Table R3) for the methods reported in Table 1 of the manuscript.

Q3 - Generalization to transformer models: Please refer to our response to W2.

Q4 - Limited scope to image classification: Our approach specifically addresses the problem of class-incremental learning (CIL) that targets image classification within continual learning settings. This particular focus aligns with a substantial body of existing literature and remains an unsolved challenge in the field. Indeed, for class-incremental continual learning, there are well-established frameworks such as Mammoth and Avalanche and widely recognized benchmarks, which we have employed in our work.

While dealing with continuous training for tasks like semantic segmentation and object detection is an important and related area of research, it falls under other broader categories (e.g., incremental object detection - Liu et al. 2023 - and incremental semantic segmentation - Michieli et al. 2019). These tasks have different baselines, methods, and benchmarks. Expanding our approach to include these other tasks would require a separate and comprehensive investigation.

However, class-incremental learning can be adapted to object detection, for instance, in the region proposal step, while it is less trivial to use in semantic segmentation. Nonetheless, we acknowledge the importance of extending continual learning methodologies to other tasks and consider it a valuable direction for future research.

• Liu et al., 2023. Continual Detection Transformer for Incremental Object Detection. CVPR 2023.
• Michieli et al., 2019. Incremental learning techniques for semantic segmentation. ICCV 2019.

We thank the reviewer for the provided comments. We here address point by point the weaknesses (indicated with W1 and W2) identified by the reviewer.

W1 - Activation maps per layer

We have tested GradCAM at different layers of the network, as well as LayerIntegratedGradients, as suggested. Our results, shown in Figures R1 and R2 of the attached PDF to the global rebuttal, reveal that activation maps from deeper classification layers, as well as intermediate and lower layers, degrade as the model is trained continuously. This indicates that catastrophic forgetting affects not just the classifier, but multiple layers of the network. Using LayerIntegratedGradients, we observed that the features learned by the network tend to change from task to task. This change leads to the forgetting of initially learned classes and concepts, demonstrating that this method also depends on the classifier and is susceptible to forgetting.

As for the significance of layer attributions in continual learning, we would like to clarify the differences between our method and the one suggested (and indicated with [2]) by the reviewer. SER is specifically designed for class-incremental learning in image classification. It employs saliency prediction techniques to modulate classification features during training. We have demonstrated that saliency prediction remains robust during training and is less prone to degradation, ensuring better performance and stability over time.

In contrast, the method described in [2] addresses class-incremental semantic segmentation (CISS) and focuses on forgetting in semantic background shift. Their approach introduces a novel classifier initialization technique that uses gradient-based attributions to identify and transfer relevant weights for new classes, specifically to address background shift in segmentation tasks.

The key difference lies in our use of saliency prediction techniques versus [2]’s use of gradient-based attribution for weight transfer. Our approach does not rely on attribution techniques, which we have shown to degrade during training, but rather on saliency prediction, which is resilient to forgetting.

However, we appreciate the opportunity to clarify these differences and we will include them in our revised manuscript, acknowledging [2]'s contribution to the field and discussing its relevance to our work.

W2 - Comparison with more recent OCL methods

We have extended our comparison to include more recent methods (see Table R1 of the pdf attached to the global rebuttal). Specifically, we have included several existing methods already integrated within the Mammoth continual learning framework (Boschini et al, 2022). This framework offers a common and consolidated benchmark for online and offline continual learning methods, which simplifies the comparison of performance and the assessment of each method’s contribution.

Among the tested methods, there are very recent approaches specifically designed for Online Continual Learning (OCL), such as PEC ([R1] in the global rebuttal), and OnPro ([R13] in the global rebuttal). Results show that methods trained with the SER strategy outperform both their counterparts without SER and existing methods by several percentage points, demonstrating the effectiveness of our SER strategy.

• Boschini et al, 2022. “Class-Incremental Continual Learning into the eXtended DER-verse,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.