Coreset Selection via Reducible Loss in Continual Learning

Thank you for your detailed review and thoughtful feedback. We appreciate your recognition of the effort put into our work and your constructive comments on the methodology and experimental setup. Your insights have been valuable in identifying areas for improvement, and we have addressed your concerns in our responses.

Weakness

1. This is an excellent question. The issue of selection order is a common challenge in greedy methods, which are employed due to the NP-hard nature of the coreset selection problem. Our method selects samples for the coreset using a greedy incremental framework, where once a sample is selected, it is excluded from consideration in subsequent selections. We have addressed this problem as follows:
   • We previously considered the possibility that the performance gain of selected samples might decrease or that some selected samples could be detrimental, and conducted experiments on MNIST, CIFAR-10 and CIFAR-100 by iteratively removing $n$ samples from the selected coreset with the lowest performance gain, then reselecting new samples from the remaining candidates. The results, presented in Table below, show that performance remains stable after removal and reselection. Additionally, the reselection process produces coresets different from the original ones.
   • One possible explanation is the existence of multiple subsets containing a similar amount of knowledge.
   • Our method selects samples that are both representative and complementary to the current coreset. This implies that selecting different initial samples results in similar final performance because our method inherently identifies subsets that complement the initial selection.
   • Further evidence for this comes from our evaluation of the coreset selection method under different random seeds in Section 5.1. Despite the variation in selected subsets, the performance remains consistent.
   • Therefore, the order of selection is not a significant concern in real-world data scenarios. We have also added this discussion in Appendix Q. in our updated submission.

2. We acknowledge that difficult samples and noisy samples cannot be distinguished solely based on Equations (4) and (6), as both types of samples may exhibit high loss values on the holdout model. Our method could differentiate difficult and noisy samples in the following aspects.
   • Our method selects coresets iteratively, difficult and noisy samples may have different impacts during the selection process.
   • Difficult samples could be informative and not harm the learning of other samples, selecting difficult samples does not preclude the selection of other difficult samples. As demonstrated in Figure 8(a) in Appendix H. For the relatively simple MNIST dataset, difficult samples tend to be more informative and are therefore selected.
   • While if noisy or ambiguous samples are included in the coreset, they may hinder the coreset model $\boldsymbol{\theta}_{\mathcal{S}}$ from effectively learning other clean samples, leading to increased loss for the clean candidates on the coreset model. As a result, the CSRL of clean candidates may increase, making them more likely to be selected in subsequent iterations.
   • We additionally analyzed the difficulty levels of selected samples in the noisy MNIST experiment discussed in the Ablation Study (Section 5.3). The results show that more easier samples are selected from the noisy MNIST dataset. This indicates that the model prioritizes easier samples when the learning of these samples is disrupted by the presence of selected noisy samples.
   • In low noise-ratio cases, which are common in real world applications, we observe that our method can distinguish difficult samples from noisy ones, as demonstrated in the noisy MNIST experiments (noise-ratio = 0.1). This is because the holdout model can effectively learn difficult samples, whereas noisy samples cannot be learned by either the holdout model or the coreset model. In high noise-ratio scenarios, the holdout model is significantly affected by noisy samples and may struggle to effectively learn difficult samples. Distinguishing noisy samples from difficult ones in noisy datasets remains a common challenge, and we plan to address this issue in our future work.

Thank you for your thoughtful follow-up and for acknowledging the aspects of your concerns that have been addressed, such as the effects of selection order and subset division methods. Regarding your remaining concerns:

1. We have demonstrated in Appendix R that our method can distinguish between difficult and noisy samples under a low noise ratio. We apologize for not emphasizing this in our rebuttal. Selecting difficult samples versus noisy samples leads to different outcomes, as shown in our analysis in Appendix R. However, we also acknowledge that our method struggles to differentiate between difficult and noisy samples under high noise ratios, which remains a common challenge in coreset selection tasks.
2. In Appendix H, we evenly split the samples into eight groups based on the holdout model loss, and there is no majority group in our split. To reflect the difficulty distribution, we provide the average loss of each difficulty group in the table below. Our method selects samples that are representative and contribute new knowledge relative to the currently selected coreset. The selected samples from each difficulty group in Appendix H. represent the outcome of our selection process, not the reasoning behind it.

3. In Table 11, we compared our method with BCSR and Greedy Coreset, which show similar performance on the CIFAR-100 dataset as reported in Table 1. While GCR is not included in Table 11. We believe the results in Table 1 (Greedy Coreset: 15.04%, BCSR: 15.11%) and Table 11 (Greedy Coreset: 28.17%, BCSR: 27.32%) are consistent.
4. Regarding the time cost of BCSR, this method selects candidate samples from each batch and then selects the coreset for memory from these candidates. Since Tiny-ImageNet contains 100,000 samples—twice as many as CIFAR-100, and the number of batches also doubles. Consequently, the selection time doubles as selection step is doubled, which explains why BCSR incurs a higher time cost compared to the other two methods.

We kindly ask that you reconsider the soundness of our work. If there are any remaining clarifications or follow-up questions, we would be happy to address them promptly.

Dear reviewer Ungg,

Thank you for your thoughtful comments, which have significantly improved our work, and for your time and effort in reviewing our paper. We would like to provide a final clarification regarding your concern about distinguishing between noisy and difficult samples.

Our method can distinguish these two types of samples in low noise ratios. The holdout model can learn difficult samples, while the model trained on the currently selected subset may exhibit high loss on these samples, resulting in a large CSRL score for difficult samples. In contrast, noisy samples cannot be effectively learned by either model, leading to a low CSRL score and making them less likely to be selected.

Additionally, our method selects samples iteratively, and the inclusion of noisy samples would lead to different outcomes. Noisy samples can negatively impact the learning of other clean samples, resulting in higher loss for those clean samples on the model trained with the current subset. This, in turn, increases the CSRL score of this clean samples, making them more likely to be selected, as discussed in Appendix R. Therefore, our method demonstrates robustness to noisy conditions, as shown in the ablation study (Section 5.3). We apologize for not emphasizing this point sufficiently in our rebuttal.

Separating noisy samples from difficult samples remains a common challenge in coreset selection tasks. As shown in Figure 8(a) (Appendix H), our method does not simply select easy samples. Furthermore, we have demonstrated that our method is more robust under noisy conditions compared to other bi-level coreset selection methods, enhancing both the soundness and effectiveness of our approach.

We hope this response adequately addresses your concern and remain open to any further questions.

Best regards,
Authors of submission 1231.

Thank you for your thoughtful and detailed feedback. We appreciate your recognition of the potential impact of our approach and your constructive suggestions for improving our work.

Weakness

1. We have identified key hyperparameters in our ablation study (Section 5.3) and evaluated their impact. Additionally, we have addressed the design choice of removing coreset samples with low performance gain and included this discussion in Appendix Q. of our updated submission.
2. For a comprehensive comparison, we have included recent coreset selection methods for continual learning, such as GCR, OCS, and BCSR, in Table 1 of our updated submission. These results highlights the timeliness and effectiveness of our approach.
3. Furthermore, we have compared our method with Greedy Coreset and BCSR using more complex backbone models, specifically ResNet-50 for CIFAR-100 and VIT-Tiny for Tiny-ImageNet. The memory size was set to 1000 for CIFAR-100 and 2000 for Tiny-ImageNet respectively. The average accuracy and time cost are presented in Tables below. The results demonstrate both the effectiveness and efficiency of our method.

Average accuracy and time cost on CIFAR-100

Average accuracy and time cost on Tiny-ImageNet

If you have any further concerns, please feel free to let us know.

1. Some coreset selection methods such as [1,2,3] are all without bi-level strategy, are they also just heuristic? Also, they are SOTAs of coreset selection methods, maybe more comparison experiments are supposed to be conducted with them.
2. You claim that “samples with high performance gain are both representative and informative” in the abstruct, why works like [4,5] can not be considered as comparisons? They also make sample selections or updates based on the performance gain.

[1] Yang S, Cao Z, Guo S, et al. Mind the Boundary: Coreset Selection via Reconstructing the Decision Boundary. Forty-first International Conference on Machine Learning. 2024.

[2] Wan Z, Wang Z, Wang Y, et al. Contributing Dimension Structure of Deep Feature for Coreset Selection. Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(8): 9080-9088.

[3] Xu X, Zhang J, Liu F, et al. Efficient adversarial contrastive learning via robustness-aware coreset selection. Advances in Neural Information Processing Systems, 2024, 36.

[4] Aljundi R, Belilovsky E, Tuytelaars T, et al. Online continual learning with maximal interfered retrieval. Advances in neural information processing systems, 2019, 32.

[5] Jin X, Sadhu A, Du J, et al. Gradient-based editing of memory examples for online task-free continual learning. Advances in Neural Information Processing Systems, 2021, 34: 29193-29205.

Thank you for your detailed review, thoughtful questions, and constructive suggestions. We appreciate your recognition of the strengths of our work and the valuable feedback regarding clarity and the holdout model's adaptability. Your insights have helped us identify areas for improvement, and we’ve addressed your comments thoroughly in our responses. We are grateful for your time and effort in evaluating our submission.

weakness

We have included an overview figure of our CSRL-CL method in Appendix P. in the updated version of our paper for a clear understanding. If you have any further concerns, please feel free to let us know.

Questions

1. In the coreset selection task, we train a holdout model on the entire dataset until convergence. For the continual learning task, we treat the training dataset of the current task as the entire dataset and select the coreset from it. After training the continual learning (CL) model on each task, the holdout model is then trained on the dataset of the current task. In CSRL-CL-prv (described in Section 4.2), holdout model is trained on current task dataset and memory. The holdout model is updated after training on each task, rather than being trained only at the initial step. As a result, holdout model incorporates updated information, enabling it to provide valuable insights for the selection process. Since the future data distribution in continual learning is unknown, our goal is to extract an informative subset from the seen datasets. This subset is replayed during training on future tasks to mitigate forgetting of previous tasks. Therefore, the holdout model only needs to provide information relevant to the seen tasks and does not need to account for unseen tasks. Figure 14 in our updated submission illustrates the coreset selection process for continual learning.
2. Equation (4) and Equation (7) represent the same equation applied to different tasks. Equation (4) pertains to coreset selection tasks, where $\mathcal{S}$ represents the currently selected subset and $(\mathbf{x},\mathbf{y})$ denotes the full dataset. While Equation (7) is used in the context of continual learning, where the subscript $t$ indicates the task ID. $\mathcal{S_t}$ represents the subset selected from the training dataset of task $t$, and $(\mathbf{x_{1:t}},\mathbf{y_{1:t}})$ denotes the data available from all seen tasks, specifically the current task's training dataset $\mathcal{D_t}$ and the memory $\mathcal{M}$.    In coreset selection task, $\mathcal{S}$ denotes the subset, while in continual learning, $\mathcal{M}$ refers to the memory. In continual learning, coresets are selected from each task, and the memory $\mathcal{M}$ contains coresets from all previously seen tasks. Therefore, $\mathcal{M}=\cup_{i}^{t}\mathcal{S}_i$.

Dear Reviewers,

We hope this message finds you well. We are writing to kindly follow up on the rebuttal we submitted a few days ago. We have provided detailed responses to your comments and would be happy to clarify or expand on any points if needed.

We understand that everyone is busy, and we greatly appreciate the time and effort you are dedicating to reviewing our submission. Please let us know if there are additional questions or feedback we can address during this discussion phase.

Thank you again for your thoughtful evaluations and guidance in improving our work.

Best regards,
Authors of submission 1231.