Continual Learning via Continual Weighted Sparsity and Meta-Plasticity Scheduling

Limited Discussion of Related Literature (Why I gave 1 to Presentation)

While the paper presents interesting ideas, there are several important concerns regarding the positioning and novelty of the work:

• The proposed "meta-plasticity" mechanism is similar to existing regularization methods (e.g., elastic weight consolidation, synaptic intelligence, and memory aware synapses). These methods similarly modulate learning rates across different connections. A more thorough discussion differentiating the proposed approach from these established methods would be valuable, particularly given that these regularization approaches are explicitly categorized as "meta-plasticity" in the "Biological underpinnings for lifelong learning machines" paper (page 204) they cite to motivate the idea.

• The paper's claimed novelty in iterative pruning appears to overlap significantly with existing work:

  • NISPA (https://arxiv.org/abs/2206.09117, ICML 2022), which presents similar ideas about iterative connection rewiring while maintaining constant sparsity Also see, Space-Net (https://arxiv.org/abs/2007.07617) and AFAF (https://arxiv.org/abs/2110.05329) - this is not an exhaustive list.

• While the current work introduces more sophisticated rewiring strategies and meta-plasticity mechanisms compared to NISPA's random selection approach (or other mentioned works), the fundamental principles appear derivative. The paper would benefit from a clearer justification for why these more complex mechanisms are necessary and advantageous.

Experimental Design and Methodology Concerns (Why I gave 1 to Soundness)

Results appear to be identical to the values in the TriRE paper, which would be acceptable with proper attribution (but I do not see any attribution). However, a more serious concern is the architectural discrepancy: while TriRE uses ResNet-18, this work claims to use ResNet-50, potentially creating an unfair comparison due to significantly different parameter counts.

While the paper includes ablation studies comparing different parameter settings for individual components, it lacks a comprehensive analysis justifying the necessity of combining all three approaches (replay, parameter isolation, and meta-plasticity). The ablations focus on tuning parameters within each component rather than demonstrating why the full combination of components is required to achieve the reported performance gains.

Critical details about buffer sizes in experiments are missing, making it difficult to properly evaluate the replay component

1.Network Saturation Challenges: The proposed framework may face limitations as the number of tasks increases, leading to network saturation. This saturation can result in reduced adaptability and performance on new tasks, as the model may struggle to allocate sufficient resources to accommodate additional knowledge without interference from previously learned tasks.

2.Complexity of Implementation: The integration of both the continual weighted sparsity scheduler and the meta-plasticity scheduler adds complexity to the implementation. This complexity may pose challenges in terms of computational resources and tuning hyperparameters, making it less accessible for practical applications compared to simpler continual learning methods. Therefore, this paper should add a hyper-parameter analysis.

3.Limited Experiments: The experimental evaluation in this paper appears to be insufficient. The authors should extend their experiments to include more diverse datasets to better demonstrate the robustness and generalizability of their proposed method.

4.Limited Improvement over SPARCL (NeurIPS 2022): The enhancements made in this framework compared to SPARCL (Sparse Continual Learning on the Edge) seem relatively incremental. A more detailed comparison could clarify the specific contributions and advantages of the proposed method over SPARCL, especially in terms of practical improvements and scalability.

5.Lack of Open-Source Code: The absence of code or implementation details in the supplementary material limits the reproducibility and practical applicability of the proposed framework.

• What is the primary contribution of the proposed $*continual weighted sparsity scheduler*$ compared to [1,2]? Is it simply a novel combination applied to CL? Why is it necessary to gradually reduce $S_{t,n}$ during epoch training? During successive epochs, are previously selected neurons fixed, with only remaining neurons being considered for selection?
• In Eqn (6), what does $\delta$ represent? I checked paper [1], and it seems this should be $\partial$, please clarify.
• Based on steps 1 and 2, the algorithm obtains a set of groups of neurons and connections $\mathcal{G}=\{g_1,...,g_T\}$. I am curious about the extent of overlap among these groups across different layers- does this provide any insight? During testing, is the task ID required to be predicted and then select the corresponding $g_i$? If not, does the algorithm need to store all groups $\mathcal{G}=\{g_1,...,g_T\}$, or would it be sufficient to store only $g_T$?
• In Eqn (7), $\omega_t^e$ is determined only after completing task t. Is the adjusted learning rate used for training task t+1, or is there a retraining step required here?
• The ablation study on the continual weighted sparsity scheduler is insufficient, and the description of the static condition is ambiguous. Does "fixed sparsity from scratch" mean the sparsity remains fixed throughout each training epoch or that the same fixed sparsity is applied across all layers? The authors should clarify these differences and interpret the performance under each condition in the ablation study.

[1] To prune, or not to prune: exploring the efficacy of pruning for model compression, 2017.

[2] Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science, 2018.

1. The proposed algorithm emphasizes the importance of carefully setting the hyperparameters, such as pre-defined sparsity, $\lambda$, $\alpha_1$, $\alpha_2$, $\beta_1$, and $\beta_2$, for effective utilization. However, this seems to involve a considerable number of hyperparameters.

1-1) Although $\alpha_1$, $\alpha_2$, $\beta_1$, $\beta_2$, and pre-defined sparsity utilize already reported values, and experiments have been conducted on various values for pre-defined sparsity and $\lambda$, how can these parameters be configured in actual continual learning situations? This hyperparameter issue has been discussed as a significant concern in continual learning research, as highlighted in studies [1, 2, 3], so I believe a discussion on this aspect is essential.

1-2) How sensitive is the proposed algorithm to changes in the values of $\alpha_1$, $\alpha_2$, $\beta_1$, and $\beta_2$?

1-3) In studies such as [1,2], scenarios where many classes are learned in the first task and the remaining classes are learned evenly across multiple tasks are considered. In such scenarios, can the proposed algorithm still achieve superior performance using the same hyperparameters, or would it require finding the new best hyperparameter values?

2. All experiments appear to be conducted based on ResNet-50. Recently, research using Vision Transformers in the computer vision domain has been actively used, reporting superior performance [4]. In this context, I wonder if the proposed algorithm would function effectively with Vision Transformers, which have entirely different characteristics. I think additional experiments are needed to verify this aspect.

3. The proposed algorithm is structured in two stages, and particularly, I believe that calculating Equation (6) incurs additional computational costs. From this perspective, how does the computation cost of the proposed algorithm (e.g., FLOPs or training time) compare to other algorithms in Table 1? As shown in studies [3, 5], it is crucial to compare not only the performance of each continual learning algorithm but also their practical computation costs.

4. All experiments were conducted using small image datasets with sizes of 32x32 or 64x64. As noted in studies [3, 6], each algorithm exhibits different trends depending on the dataset. In this light, I believe the authors should conduct experiments on at least the ImageNet-100 dataset (image size of 224x224) and compare the results with baselines to validate additional effectiveness.

5. What was the memory buffer size used in the experiments? When varying this size, can the proposed algorithm consistently achieve superior performance?

6. Considering the results and experiments in [6] (refer to GitHub), does the proposed algorithm outperform well-established algorithms in class-incremental learning, such as WA, BiC, and FOSTER? To concretely validate the superiority of the proposed algorithm, a comparison of performance and costs with these methods is necessary.

[1] Online hyperparameter optimization for class-incremental learning, AAAI 2024.

[2] Hyperparameter Selection in Continual Learning, CoLLAs 2024 Workshop.

[3] Hyperparameters in Continual Learning: A Reality Check, CoLLAs 2024 Workshop.

[4] A Comprehensive Survey of Continual Learning: Theory, Method and Application, TPAMI 2024.

[5] Computationally Budgeted Continual Learning: What Does Matter?, CVPR 2023.

[6] PyCIL: A Python Toolbox for Class-Incremental Learning, SCIENCE CHINA Information Sciences.