YoooP: You Only Optimize One Prototype per Class for Non-Exemplar Incremental Learning

Thank you for your constructive comments and suggestions! We have addressed all the comments and suggestions from the Reviewer and accordingly updated our manuscript highlighted in BLUE. We hope our responses below address your concerns. Please let us know if you have any additional concerns.

Q1: The title may be unsuitable.

A1: Thanks for your suggestion! We will change the title to "OPT: Optimizing Prototypes for Non-Exemplar Class-Incremental Learning" if the reviewer thinks it is appropriate.

Q2: Some illustrations are unclear.

A2: We have added the X coordinate for Figure 1 and also explained the "memory footprint" in the caption in our revised manuscript.

Additionally, we have explained the memory gap between our method and existing studies in the revised version. As described in the caption of Figure 1 and detailed in Appendix D, the term "memory size" denotes the memory footprint of our method and other approaches when computing prototypes for each task during training. Unlike other prototype-based methods that necessitate saving the embeddings of the entire class to generate the class mean prototype, YoooP generates optimized prototypes with a smaller memory footprint based on a mini-batch attentional mean-shift algorithm. That is why our approach requires a smaller memory footprint than the existing methods.

Q3: The experimental results should be reorganized.

A3: Per your suggestion, we have moved the experimental results under half-base 5 phases and half-base 10 phases settings to the main paper.

Q4: Explain the significance and advantage of sampling in Section 3.2.1.

A4: Per your suggestion, we have added some explanations about sampling in Section 3.2.1. The main advantage of our method is that we store the distribution of real angular using a histogram while existing works did not. The existing approaches like PASS involve adding high-dimensional noise into prototypes, causing a significant divergence in the angular distributions between the class prototypes and synthetic data from the actual angular distribution. Moreover, Figure 5 shows the angular distribution comparison of synthetic data augmented from stored prototypes in PASS and the ground-truth angular distribution. We can observe that the angular distributions of PASS differ from the original distribution. On the other hand, our method samples the cosine similarity in the original angular distribution. Hence, our method can generate higher-quality synthetic data than the baselines like PASS and FeTrIL, contributing to a more effective mitigation of forgetting. For more details, please refer to Sections 3.2.1 and 4.2 in our revised version.

Q5: Clarifying the memory budget.

A5: Per your suggestion, we have presented the additional memory requirements of various methods for computing prototypes in Table 1. Here, the term 'extra memory budget' specifically denotes the number of saved embeddings utilized for creating class prototypes. In our methods, this corresponds to the number of embeddings in a mini-batch. In contrast, the other methods necessitate storing the entire set of embeddings for each task to calculate class mean prototypes.

Table 1: Extra memory budget: # of saved embeddings

Q6: The influence of hyper-parameter $\gamma$ in Eq. 11.

A6: We have explored the impact of hyper-parameter $\gamma$ on CIFAR-100 with zero-base 10 phases setting, as illustrated in the following table. It can be observed that the hyper-parameter $\gamma$ is not very sensitive, and increasing the weight of knowledge distillation within a certain range can be beneficial for YoooP/YoooP+. We can see that weight values between 25 to 40 result in excellent performance.

Table 2: Influence of hyper-parameter $\gamma$ on model performance

Q4: Report the last accuracy and compare with iCaRL.

A4: Thanks for your suggestion! Our average accuracy means the average incremental accuracy (AIA) in our submission! Per your suggestion, we already updated the name of this metric in the revised manuscript.

Note that our proposed models are non-exemplar CIL methods that do not store real samples for replay but only store prototypes, while the iCaRL is an exemplar-based method that needs to store and replay real samples. Thus, iCaRL requires a much larger memory budget than our methods. Moreover, prior studies [3][4][5] have pointed out that exemplar-based CIL may suffer from extra risk of privacy leakage and large memory costs. In reality, non-exemplar CIL is much more challenging than exemplar CIL as mentioned by the Reviewer gcwm. Therefore, a direct comparison between our method and iCaRL seems unfair.

Nevertheless, we already compared our methods with iCaRL [1] on the TinyImageNet dataset under different settings in Figure 10 in Appendix F. We report the last accuracy in the following table. It can be observed that our methods still outperform the iCaRL in terms of average incremental accuracy.

Table 1: Evaluation of TinyImageNet.

References

[1] Incremental Classifier and Representation Learning. (CVPR2017)

[2] Feature Translation for Exemplar-Free Class-Incremental Learning. (WACV2023)

[4] Prototype Augmentation and Self-Supervision for Incremental Learning. (CVPR 2021)

[5] Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning. (CVPR 2022)

[6] Feature Translation for Exemplar-Free Class-Incremental Learning. (WACV 2023)

Q5: More datasets and baselines with or without using exemplars should be compared as saving some data is not an issue.

A5: Per your suggestion, we added a new experiment evaluating our proposed methods on another large ImageNet-100 dataset. The comparison results are illustrated in the table below. Our methods (YoooP and YoooP+) still significantly outperform the baselines in terms of average incremental accuracy on the new ImageNet-100 dataset.

Table 2: Evaluation of ImageNet-100.

Regarding more baselines, the proposed models are non-exemplar CIL methods that do not store real samples for replay but only store prototypes, while the baselines you suggested are exemplar-based methods that need to store and replay real samples. Thus, comparing non-exemplar methods with the latest exemplar methods could be deemed unfair. Moreover, prior studies [1][2][3] have pointed out that exemplar-based CIL may suffer from extra risk of privacy leakage and large memory costs. Nonetheless, we still compared our method with a few exemplar-based methods like BiC [4], iCaRL [5], and WA [6] in Figure 11 in Appendix G. We can see that the proposed YoooP and YoooP+ can achieve comparable results under the zero-base settings, and even outperform some exemplar-based methods under the half-base settings.

Concerning the previous study [7], they utilize task ID information during training, which deviates from the established definition of Class-incremental learning (CIL). As outlined in [8] and [9], CIL, true to its name, operates without predefined task boundaries. Consequently, CIL methods typically lack access to any task ID information during both training and inference phases. This implies that it's impractical to train a model to predict the task ID of input samples during testing or inference in CIL scenarios. Unlike prior work [7], we only access the class label rather than task ID information during model training, which aligns with the methodology of prior works [1],[2],[3],[4],[5],[6]. Therefore, we have opted to exclude that particular work [7] from our comparative analysis.

References (continue)

[1] Prototype Augmentation and Self-Supervision for Incremental Learning. (CVPR 2021)

[2] Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning. (CVPR 2022)

[3] Feature Translation for Exemplar-Free Class-Incremental Learning. (WACV 2023)

Thank you for your constructive comments and suggestions! We have addressed all the comments and suggestions from the Reviewer and accordingly updated our manuscript highlighted in BLUE. We hope our responses below address your concerns. Please let us know if you have any additional concerns.

Q1: Idea is similar to contrastive learning.

A1: Our approaches bring substantial advancements over existing prototype-based methods. Unlike existing prototype-based approaches that only calculate the mean prototype for each class, we propose a novel attentional mean-shift algorithm for optimizing the prototype per class. The proposed method helps to learn a more compact and informative prototype. To our best knowledge, we are the first to study prototype optimization in CIL. Moreover, we develop a novel prototype augmentation technique based on a high-dimensional rotation matrix to generate high-quality synthetic data. In contrast, existing studies only add Gaussian noise into saved prototypes for data generation. The evaluation results demonstrated our methods can significantly outperform the baselines on multiple datasets under different settings. Thus, other reviewers, such as Reviewer #ZsAp, gcwm, and EwLK, recognize the novelty of our proposed methods.

In addition, we would like to highlight the distinction between our method and contrastive learning. Contrastive learning is known as self-supervised learning for pre-training. It aims to learn the representations of input data through the comparison of sample pairs, rather than learning a signal from individual data samples sequentially. This comparison involves both positive pairs (comprising "similar" inputs) and negative pairs (involving "dissimilar" inputs) [1]. In contrast, our methods introduce a novel attentional mean-shift algorithm for optimizing the prototype for each class. We then apply contrastive loss to further enhance the representativeness of each class's prototype. These two components ensure that the learned prototypes are more effectively representative. Note that the optimized prototypes can not be directly obtained through contrastive learning. Rather, the proposed attentional mean-shift algorithm plays a critical role here, though the contrastive loss is helpful.

References

[1] Contrastive representation learning: A framework and review. (Ieee Access 2020)

Q2: Where the forgetting occurs?

A2: As mentioned in our manuscript, catastrophic forgetting [1] refers to deep neural networks forgetting the acquired knowledge from the previous tasks disastrously while learning the current task. In other words, forgetting will occur when neural networks learn new tasks in sequence. In the context of Class-Incremental Learning (CIL), each task is learned independently and sequentially using a single shared neural network. Since all the tasks share the same network, learning a new task can overwrite the network weights learned for the previous tasks [2]. This leads to forgetting, i.e., accuracy drop on previous tasks.

References

[1] Catastrophic interference in connectionist networks: The sequential learning problem. (Psychology of learning and motivation 1989, p109-165)

[2] Prototype Augmentation and Self-Supervision for Incremental Learning. (CVPR2021)

Q3: The inference procedure is not discussed.

A3: In Class-Incremental Learning (CIL), the inference for each task is performed similarly to a standard classification task. Specifically, as depicted in Figure 2, we use the backbone, denoted as $F(;\theta)$, to extract embeddings from the test samples. These embeddings are then fed into the classifier $G(;w)$ to estimate the probability distribution over all classes. Conversely, the saved prototypes are merely utilized during the model training phase to avoid forgetting.

Q6: More accurate notation.

A6: Thanks for your suggestion! We have already updated the notation with $softmax(G(F(x; \theta); w))$ in Section 3 highlighted in blue.

Q7: The function c(,) in Eq.2, Eq.4.

A7: Thanks for your correction! After fixing Eq.4 in our revised version, $c(\cdot,\cdot)$ in both Eq. 2 and Eq.4 denotes the cosine similarity function. In Eq. 4, it is used to compute the logits for all the classes $j\in C_{1:t}$ as the cosine similarities between each vector in the weight matrix of the final-layer classifier and (1) a sample's representation $z_i$ (the first term in Eq. 4); or (2) each class's prototype $p_k$ (the second term in Eq. 4).

$L_{t,C}(\theta,w)\triangleq\frac{1}{|C_t|}\frac{1}{n_t}\sum_{k\in C_t}\sum_{i\in [n_t]:y_i=k}\ell([c(z_i, w_j)]\_{j\in C_{1:t}, k)} + \frac{1}{|C_{1:{t-1}}|}\sum_{k\in C_{1:{t-1}}}\ell([c(p_k, w_j)]\_{j\in C_{1:t}}, k)$

Q8: The initial class-wise prototype $p_k$.

A8: We randomly choose one sample from each class-$k$ and use its embedding as the initial prototype $p_k$. Our prototype update method is an extension of the classical mean-shift algorithm [1] that replaces the pre-defined kernel similarity with a learnable attention score. As stated in [1], random initialization of $p_k$ can guarantee the independence of the solution to the initialization problem.

References

[1] Mean shift, mode seeking, and clustering, TPAMI 1995.

Thank you for your constructive comments and suggestions! We have addressed all the comments and suggestions from the Reviewer and accordingly updated our manuscript highlighted in BLUE. We hope our responses below address your concerns. Please let us know if you have any additional concerns.

Q1: Whether the method works with bigger datasets

A1: Thanks for your suggestion! We have evaluated the proposed two approaches on ImageNet-100. The results are reported in the following table, where "*" indicates non-exemplar methods, "-last" means the average accuracy of the last task. IL2A is not included in the comparison because we do not have enough GPU memory to train their model on ImageNet-100. Our methods (YoooP and YoooP+) still significantly outperform the baselines in terms of average accuracy on the large ImageNet-100 dataset.

Table 1: Evaluation on ImageNet-100.

Q2: About the result with 20 phases

A2: We have evaluated the proposed methods on TinyImageNet under zero-base 20 phases and half-base 20 phases settings. The results are reported in the following table, where "*" indicates non-exemplar methods, "-last" means the average accuracy of the last task. We can observe that our methods (YoooP and YoooP+) still significantly outperform the baselines in terms of average accuracy on the 20 phases settings.

Table 2: Evaluation on TinyImageNet.

Q3: Optimized prototype seems limited in situations.

A3: Our prototype optimization method is versatile and can be applied not only to prototype-based methods but also to various scenarios in class incremental learning, as demonstrated in [1][2][3][4].

References

[1] Incremental Classifier and Representation Learning. (CVPR 2017)

[2] Large Scale Incremental Learning. (CVPR 2019)

[3] BEEF: Bi-Compatible Class-Incremental Learning via Energy-Based Expansion and Fusion. (ICLR 2023)

[4] Feature Boosting and Compression for Class-incremental Learning. (ECCV 2022)

Q4: Apply the new augmentation strategy to PASS.

A4: We have applied our new prototype augmentation (PA) strategy to the PASS, and then evaluated its performance on CIFAR-100 under the zero-base 10 phases setting. The results are reported in the following table. We can observe that our prototype augmentation method can significantly improve the performance of PASS, even without prototype optimization.

Table 3: Comparison of adding Gaussian noise and our strategy on PASS.

Q5: Show the classification confusion matrix.

A5: Thanks for your suggestion! We randomly choose 10 classes to show a part of the classification confusion matrix of YoooP+, and the entire confusion matrix of 100 classes is presented in Appendix E of the revised version.

Table 4: Confusion Matrix (randomly sampled 10x10 submatrix) for CIFAR-100 with zero-base 10 phases setting

Q6: Evaluation on large ImageNet-100 dataset.

A6: Per your suggestion, we have evaluated our methods on ImageNet-100. The results are reported in the following table, where "*" denotes non-exemplar methods, "-last" means the average accuracy of the last task. IL2A is not included in the comparison because we do not have enough GPU memory to run their model on ImageNet-100. Our methods (YoooP and YoooP+) still significantly outperform the baselines in terms of average accuracy on the new dataset.

Table 2: Evaluation on ImageNet-100.

Q7: Logic in the reasoning.

A7: We already updated the logic meaning on Page 7 as follows.

According to Table 1 and Figure 4, both SSRE and FeTrIL have lower forgetting while the prediction accuracy drops rapidly in the initial tasks. A lower forgetting in this case (with lower accuracy) indicates that the model is not improving or learning so such performance is not preferred. In contrast, the proposed YoooP and YoooP+ reach higher average incremental accuracy while achieving slightly higher or comparable forgetting compared to other non-exemplar baselines. This indicates a better stability-plasticity trade-off, implying that the model learns quickly on new tasks while incurring a minimal cost in forgetting. Therefore, it can be concluded that the proposed methods outperform the non-exemplar baselines.

Q8: Synthetic data constrain the boundary of old tasks.

A8: Our experiments in Figure 6 verify that our method can achieve better classification boundaries for old tasks. As shown in Figure 6, the light gray points represent the distribution of data from old tasks. A comparison between Figure 6(a) and Figure 6(b) reveals that the data from previous tasks are still clustered into several groups using our method, exhibiting a clear boundary with the clusters of current task data. This pattern is also evident in Figure 6$c$. The high-quality synthetic data effectively constrains the boundary of old tasks when training the current tasks, enabling the current tasks to establish a clear boundary with previous tasks. In contrast, the baselines like PASS (illustrated in Figure 6(d), (e), (f)), without our proposed prototype augmentation in YoooP+, will make the boundary between the current task data and previous task data gradually become less clear and their classes may even overlap with each other.

Q9: References in the Introducion.

A9: We have cited some earlier works on class-incremental learning in the Introduction in our revised version.

Q10: Different starting points in Fig.8(a).

A10: According to Equation 12, we update the parameters $\theta$ based on $\beta$. The different $\beta$ in our ablation study result in different starting points, $\theta_1$, for each case.

Q11: Different start points for YoooP and YoooP+ in Fig.4.

A11: It is caused by the random error (random seeds) in the experiments. Below, we present the exact accuracy of the first task in Figure 4. We can see that their accuracy is very close to each other.

Table 3: First Task Accuracy

Thank you for your constructive comments and suggestions! We have addressed all the comments and suggestions from the Reviewer and accordingly updated our manuscript highlighted in BLUE. We hope our responses below address your concerns. Please let us know if you have any additional concerns.

Q1: Hard to reproduce with hyperparameters.

A1: To reproduce our experimental results, we have uploaded the source code in the Supplementary Material. Additionally, we will make the source code publically available on GitHub upon publication. By the way, our source code is not based on an existing framework like PyCIL.

Regarding parameters $S$ steps and $R$ iterations in Algorithm 1, they are determined by the ratio of Dataset Size to Mini-batch size. More specifically, $S$ is the number of total steps per epoch for each task. For the number of steps $R$ used to update the prototypes in each task, we simply set $R=S$ in our experiments. We have added the explanations of these two parameters in Appendix A in our revised version.

For the Arcface loss hyperparameter $\delta$, we already conducted an ablation study in Appendix C.5 and consistently used $\delta=0.25$ across all experiments with various datasets. Per your suggestion, we have introduced the Arcface loss in the main paper.

Q2: Eq.4 has the misunderstanding.

A2: Per your suggestion, we added some explanations of Eq.4 in the updated version as follows.

where $\ell(\cdot,\cdot)$ is a classification loss, and $i\in [n_t]:y_i=k$ is replaced by $i\in B$ when training by mini-batch.

Q3: Some crucial hyper-parameters are not provided in main paper, or not provided at all - number of S steps, R iterations.

A3: Per your suggestion, we have introduced the important hyper-parameters in the main paper. Regarding parameters $S$ steps and $R$ iterations, they are actually determined by the ratio of Dataset Size to Mini-batch size. Please refer to A1 above.

Q4: The ablation is not clear enough.

A4: The classification loss $L_{t,C}$ and the knowledge distillation loss $L_{t,KD}$ in YoooP are commonly used in recent Non-exemplar CIL methods[1],[2],[3]. These two components are also adopted by the baselines. In addition, the partial freezing of the classifier is also used in existing work LwF[1]. So any improvement over the baselines is a result of the proposed $L_{t,P}$ loss.

Accordingly, we do an ablation study on the newly proposed components, including model interpolation (MI), prototype optimization $P$, and prototype augmentation (PA) in YoooP+. As illustrated in Figure 7 in Appendix C.1, we can see that YoooP(-P), which means without prototype optimization ($L_{t,P}$), will result in much lower accuracy, 51.50%, than that of YoooP(-MI), 54.82%. It thus suggests that prototype optimization is more important than MI. However, MI is also important since the accuracy of YoooP(-MI) will be lower compared to YoooP, 57.66%.

Additionally, we conducted an additional ablation study about MI on YoooP+. The evaluation results on CIFAR-100 with zeros-base 10 phases setting are reported in Table 1. Removing MI from YoooP+ reduces the accuracy by 3.91% while removing PA from YoooP+ (i.e. YoooP) reduces the accuracy more, 4.27%. Thus, prototype augmentation (PA) in YoooP+ is more important than MI.

Table 1: Additional Ablation Study

References

[1] Learning without Forgetting. (ECCV2016)

[2] Prototype Augmentation and Self-Supervision for Incremental Learning. (CVPR2021)

[3] Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning. (CVPR2022)

Q5: The first figure misses units for the memory size as well as SSRE to my knowledge has the growing part and then compression.

A5: Thanks for your suggestion! We have updated the units for the memory size in Figure 1.

As described in the caption of Figure 1 and detailed in Appendix D, the term 'memory size' denotes the memory footprint of our method and other approaches when computing prototypes for each task during training.

For SSRE, our reported memory size does not refer to the GPU memory during model training. SSRE, IL2A, and PASS compute the mean value of all embeddings per class. Thus, they have the same memory footprint (same line in Figure 1) and incur an expensive memory footprint for storing their embeddings. In contrast, our proposed mini-batch attentional mean-shift method computes prototypes in a mini-batch manner. This significantly reduces the memory footprint required for prototype generation.