Divide and not forget: Ensemble of selectively trained experts in Continual Learning

Can you be a bit more elaborate on Weakness 1 to give the authors an opportunity to answer and the other reviewers to verify for themselves? In particular, I'd like to see references to the existing work. Thanks!

We appreciate the feedback provided by the Reviewer. We will now address the specific weaknesses indicated:

Novelty concern related to the next task selection

Selecting the next task based on largest KL divergence in continual learning is not a novel strategy. So is the expandable MoE architecture and the inference method.

We thank the reviewer for that comment, yet we believe that there may be some misunderstanding of our method. Please note that we do not select tasks. We use KL-divergency to assign a single, existing expert to be trained on a given task. However, during the inference time, we use an ensemble of all available $K$ experts whose predictions are aggregated, and this approach is different from conventional MoE architectures. As far as the novelty of our approach is concerned, we believe that our work is the first to show how allocating experts to be trained on tasks can alleviate catastrophic forgetting in continual learning. We would appreciate if the Reviewer could link references to the mentioned relevant existing continual learning strategies so we can refer to them in our work.

Concern of the overhead growth

As stated in Method section 3 Our approach, presented in Fig.2, consists of $K$ deep network experts. Therefore, K is the number of experts, which is equal to 3 or 5 in our experimental section regardless of the number of tasks |T|. We do not grow the network architecture above that number, e.g., in Fig. 4 and Tab. 1 we present experiments with |T|=50, where K=5 experts are used.

We hope our explanation alleviates any concerns the Reviewer may have. However, if there are any remaining uncertainties, kindly specify references and additional questions, and we can further discuss them. Otherwise, we'd appreciate if the reviewer reconsidered the final score of our submission.

Dear Reviewer rq4G,

The discussion period ends in 24 hours.

We appreciate the Reviewer's contributions to the reviewing process. We have put much effort into our work and would appreciate the Reviewer's constructive feedback.

We are also dedicated to the review process and ensuring that our work adheres to the highest standards. We kindly ask the Reviewer to respond to our previous comment and identify specific areas in our submission where further improvement would allow the Reviewer to confidently recommend our paper for acceptance (more than above-borderline acceptance). We highly value and appreciate your efforts.

We thank the Reviewer for the constructive feedback, insightful comments and providing a reference. Below we respond to the weaknesses mentioned:

Do the gains come from the proposed method or from the additional complexities?

We agree that the question whether the gains come from the proposed method or the additional complexities is important. Therefore, we run the experiments with results presented in Tab. 3 and Tab. 4 of our submission. More specifically, in Tab. 4, we show that our method of ensembling experts in CIL outperforms existing SoTA ensembling methods; all require the same amount of parameters. In Tab. 3, our method achieves better results than competitors while using fewer parameters. We have also performed another experiment presented in Tab. 5 (and below) where we evaluate methods with regard to the number of theirs parameters during inference. We have added it to the revised version. The above observations lead us to the conclusion that the gain of our method does not result only from the additional complexities.

We affirm that various CL methods present different memory/computational footprints. Especially for ensemble methods, it is inherent that few experts have to be stored in memory. Evaluating the complexity of some methods from Tab. 1 & 2 is not trivial, e.g., SSRE uses a complex few-phase training with expansion and pruning phases. However, pruning can also be applied to every other method from these tables. We try to be clear about SEED, and therefore we present and discuss: (1) the parameter efficiency in Tab. 3 compared to other MoE and ensemble architectures, (2) memory footprint in Appendix A.3, (3) we have also added a tradeoff between number of models' parameters vs accuracy in Appendix A.4 (revision version) to better address this concern.

Performance comparison under a limited number of parameters.

To address this concern, we have added Tab. 5 to the appendix (and below). SEED achieves better accuracy than the best competitor (FeTriL) from Tables 1&2 while using less parameters. Here we consider number of parameters required by the method to perform the inference. For SEED these paramters come from feature extractors of experts (here they share first 17 layers) and Gaussian distributions.

Avoiding catastrophic forgetting in SEED

One of the main goals of our method is to limit catastrophic forgetting. For that reason, we proposed our expert selection strategy to alleviate forgetting. We demonstrate the improvements in Tab. 4 and Fig. 6. Please note that we train only one expert in each task and that knowledge distillation regularization is applied only to it. If an expert is inactive (not trained on tasks), the parameters of its feature extractor do not change. Thus, it's not prone to catastrophic forgetting.

How could SEED avoid forgetting when the number of tasks is significantly larger than the number of experts?

In that case SEED avoids forgetting in two ways. First, in each task SEED trains only one expert, other experts (K-1) remain unchanged so that they avoid forgetting. Please note, that layers shared by experts are frozen after the first task. Second, we utilize feature distillation for trained expert to alleviate its forgetting - we made it more clear in Section 3 in the revised version.

Comparing VariGrow to SEED

The method cited by the Reviewer performs experiments with exemplars and auxilary datasets (4.1 Implementation details: We also use these exemplars along with OoD datasets), while we propose an exemplar-free method for more challenging scenarios and we do not use exemplars or auxilary data in our experiments.

Two different references for CoSCL in the Introduction.

We have fixed this in the revised version. Thank you.

We trust that our explanation has addressed the Reviewer's concerns. Should there be any additional queries, we are more than willing to provide further details. If no further clarification is needed, we kindly ask the Reviewer to reconsider the final score.

I appreciate the authors efforts in addressing my comments.

However, my concerns regarding the drawback of SEED remains. SEED alleviates forgetting by freezing the feature extractor and activating only 1 expert per task. However, this strategy strongly assumes that the first task can provide a good feature representation for all future tasks and there are no tasks that require a completely different representation, which are quite restrictive. For example, consider the complex transfer settings in the CTrL benchmark [A] where the data stream can have unrelated tasks ($\mathcal{S}^{\text{plastic}}$), or the first task only have limited training data ($\mathcal{S}^+$), then SEED may not perform well on such scenarios.

It may not be feasible to conduct comprehensive experiments on the CTrL benchmark during this rebuttal period, but the authors should clearly state the assumption of SEED, indicating which settings it might work well on, and acknowledging the conceptual drawbacks.

[A] Veniat, Tom, Ludovic Denoyer, and Marc'Aurelio Ranzato. "Efficient continual learning with modular networks and task-driven priors." ICLR 2021.

Although the rebuttal period is almost over, we are still eager to answer the Reviewer's questions and continue the discussion. Insights provided by the Reviewer were helpful and allowed to increase the quality of our work. If we have addressed uncertainties regarding our work, we would appreciate adjusting the final score.

We express our gratitude to the reviewer for the constructive feedback and insightful remarks. We shall commence by addressing the specific weaknesses pointed out:

The tradeoff between number of parameters and accuracy

We have run additional experiments and presented them in Appendix A.4, where we use ResNet32 and ResNet18 architectures and provide accuracy depending on the number of experts and shared layers. The tradeoff between an overall number of model parameters and accuracy can be observed. We can use that analysis to limit the parameters and still reach a satisfactory accuracy. However, we agree that number of parameters can be a limiting factor our method if the models are extremely large, as few experts have to be stored in memory.

The impact of a pretrained model on SEED

We have performed experiments to examine the impact of using a pretrained model vs training from scratch. The results for DomainNet datasets are presented below (we also added them in A.5):

We use DomainNet here as we consider it the most interesting scenario to use ImageNet pretrained network. SEED can benefit from starting each expert from the pretrained network and reach better accuracies with a slightly higher forgetting. However, here, we use the same setting as from the scratch (hyperparameters), which can be suboptimal for this scenario.

Differences between finetuning and one expert setting

There are two significant differences between using SEED with one expert and finetuning. First, we perform Bayes classification during the inference. Second, we use a feature distillation to increase the stability of the expert. The method is doing well in this scenario as in our ensemble, few experts are not finetuned and increase stability of the method, while the one being finetuned allows for good plasticity.

If we have adequately addressed the Reviewer's concerns, we kindly ask for your support. If you have any further concerns or additional points to raise, we are eager to address them. Your insights are valuable in enhancing the quality and impact of our research.

We thank the Reviewer for the positive feedback and insightful comments. We begin by responding to the weakness pointed out in the review:

The need to train and store a number of experts

As the Reviewer stated, this is inherent in ensemble methods. However, we discuss the number of parameters in Tab.3 and Appendix A.3. We have additionally run experiments to examine the trade-off between the number of parameters and accuracy of SEED in Appendix 4. We show that the total number of parameters can be lowered, and SEED can still achieve satisfactory results.

Improvements in the paper

In Figure 3, it's hard to see that task 3 overlap least with the second expert. Perhaps make this a bit clearer.

We appreciate that suggestion. We updated the figure in the new revision.

In Page 4, perhaps the inference and training algorithms would be easier to read in an "Algorithm" structure rather than simple paragraphs.

_In Page 5, just before Section 4.1., the task incremental scenario is mentioned but is not really defined properly in my opinion.

We have addressed both comments in the revised version. Thank you.

Questions

In Page 8, it's written that "SEED uses a regularization method known from LwF". Are you actually using LwF for each expert? I don't recall reading it in other places in the paper. Please clarify.

SEED uses feature distillation for each expert, similar to LwF (logit distillation) but performed in the latent space. It's mentioned in Sec. 3, Training (page 4/5). We clarified this part of our method description in the new revision.

How are the hyperparameters of EWC and LwF set in the experiments ? For instance, Figure 7 shows without any justification.

We set default parameters of the methods as reported in the original articles. For avoidance of doubt, we also stated it in Appendix A.1: For baseline methods, we set default hyperparameters. For Fig. 7 hyperparameters were manually chosen to exhibit the tradeoff between intransigence and forgetting, and we list all these parameters in Appendix A.1 as well.

In Page 1, the authors mention "The trend is evident... results steadily improve over time". What time?

In this case, we meant the dates of publication of the corresponding articles. The methods presented in Fig. 1 are positioned according to the order of their publication dates. We have labeled the publication year axis x in Fig. 1 to better visualize that in the revised version.

In Table 1, how many repetitions are performed per experiment?

We perform three repetitions per experiment, as stated in Appendix A.1: All experiments are the average over three runs.

In Table 1, why aren't there results for ImageNet-Subset for two algorithms?

For these two methods, we performed experiments using their implementations in PyCIL benchmark and default hyperparameters. However, on ImageNet-Subset IL2A performance was really low, while PASS experiments did not converge, probably getting stuck, and we could not finish them.

In Figure 5, the presented metric is "relative accuracy". Relative to what?

We present the metric relative to the mean achieved by all experts. We have clarified that in the caption of Fig. 5 in the updated version: It is calculated by subtracting the accuracy of each expert from the averaged accuracy of all experts.

If the Reviewer's concerns have been sufficiently addressed in our responses, we humbly seek their support for the paper.