HyperMask: Adaptive Hypernetwork-based Masks for Continual Learning

We thank the reviewer for the comments related to our manuscript. We refer to your remarks in the text below.

Q1: Baselines suggested in this paper are relatively outdated. I understand that incremental architecture methods are no longer dominating this field of research but it does not mean that they can ignore regularization or representation based methods. For example, FeCAM [1] does not even need to know the task index and it still outperforms the author's method.

R1:

• HyperMask lies in a completely different category of continual learning methods. Therefore, its comparison with FeCAM is not trivial. FeCAM is dedicated to a different setting where we have a pre-trained backbone, which does not change during the next training iterations. Authors use pre-trained architectures, like ResNet-18 and ViT-B/16, to generate class prototypes. They model the feature covariance relations using Mahalanobis distance to determine decision boundaries. FeCAM is not an architecture-based method since the architecture is not changed during training. We want to highlight that a setting with a frozen backbone and prototypes is important but is not dedicated to architecture-based approaches. Nevertheless, FeCAM is a good benchmark for the prototypical methods.

• Consequently, no experiment from our paper matches any experiment in FeCAM. By comparing the results from HyperMask and FeCAM, it is impossible to say which model works better because the evaluation strategies of the methods are different. Authors of FeCAM state, as follows:

We experiment with three different incremental settings for CIFAR100 and ImageNet-Subset: 1) 50 initial classes and 5 incremental learning (IL) tasks of 10 classes; 2) 50 initial classes and 10 IL tasks of 5 classes; 3) 40 initial classes and 20 IL tasks of 3 classes. For TinyImageNet, we use 100 initial classes and distribute the remaining classes into three incremental settings: 1) 5 IL tasks of 20 classes; 2) 10 IL tasks of 10 classes; 3) 20 IL tasks of 5 classes.

• In HyperMask, we followed the experimental scenarios from architecture-based methods like HNET [1] and WSN [2]. In the case of CIFAR-100, the dataset was divided into 10 tasks with 10 classes per task, as in [2]. Similarly, we added experiments for TinyImageNet according to the strategy described in [2] or [3], i.e. we created 40 tasks with 5 classes in a single task. It means that the evaluation criteria are different and there is no possibility of comparing the results for the experiments mentioned above.

• We also think that models that do not use a pre-train backbone are still very important. We compare our model with WSN (published in ICML 2022) in a similar setting to the depicted one. We also compare our results with two strong algorithms: La-MaML [3] from 2020 and FS-DGPM [4] from 2021.

• In the paper, in Table 2, we described results for a continual learning setting in which the task identity has to be recognized by the model itself. The results are presented for Permuted MNIST and Split MNIST datasets. The method applied was very simple because it was based on the entropy calculation, and still, the results were competitive. It suggests that the application of the more advanced approach (for instance, based on clustering methods used for the target network classification layer) should lead to high performance. It is part of our future work.

[1] J. von Oswald et al., Continual learning with hypernetworks, ICLR 2020.

[2] H. Kang et al., Forget-free Continual Learning with Winning Subnetworks, ICML 2022.

[3] G. Gupta et al., La-MAML: Look-ahead Meta Learning for Continual Learning, Advances in NeurIPS, 2020.

[4] D. Deng et al., Flattening sharpness for dynamic gradient projection memory benefits continual learning, In Advances in NeurIPS, 2021.

Q2: The amount of experimental result is severely insufficient. According to the paper, HyperMask is superior to other methods only in Split-Cifar100 and in my knowledge this is not the best one in continual learning. Please refer to

Goswami, Dipam, et al. "FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning." arXiv preprint arXiv:2309.14062 (2023).

R2:

• We performed experiments on TinyImageNet, a demanding dataset consisting of 40 continual learning tasks with 5 classes. A single image has a shape of 64 x 64 x 3. We prepared experiments using ResNet-20 architecture with an embedding vector of 96, β and λ equalling 1 and 0.1, respectively. Furthermore, p was set to 0, batch size to 16 and learning rate to 0.0001. Similar to WSN, the number of training epochs for each task was 10, and 50 validation samples from each class were selected in each CL task.

• We achieved state-of-the-art results, outperforming all reference methods. The mean classification accuracy for 5 runs of HyperMask was equal to 76.22%, while the best baseline, WSN, reached 71.96% using the architecture with 4 convolutional and 3 fully connected layers. Full results for TinyImageNet are depicted in the following table:

Results for TinyImageNet

Thanks for the prompt response of the authors. Here are several issues I still have.

• FeCAM can be used upon a pre-trained network but the main method described in the paper is training on the first task and keep it frozen during the remaining training phases. Therefore it's not a method that makes use of something more than this paper.
• I do know the difference of experimental settings but the one from FeCAM is far more difficult because the task index is not given during the evaluation. Furthermore, FeCAM has demonstrated on a smaller architecture and it's hard to find the benefit of this method then.
• But as the authors say, considering the difference on settings and setting limits on the range of baselines, the performance seems to be reasonable

• I think it's a difference of a perspective but in my opinion, HAT trains individual embedding vectors for each task through a learnable module and I don't see the difference from the hypernetwork in terms of controlling the main network from outside. Accordingly, the contibutions the authors are claiming does not seem to be valid.

We thank the reviewer for the feedback regarding our paper and constructive remarks. We are glad that you appreciate our approach. We respond to your questions in the points below.

Q1: Can the authors provide a more detailed discussion of the novelty of their proposed method compared to prior work in the field of continual learning and hypernetwork-based approaches?

R1: The model combines two continual learning (CL) methodologies and provides a new concept using a meta-model to produce subnetworks. The masks produced by the hypernetwork modulate the weights of the main network and act like dynamic filters, enhancing the target weights important for a given task and decreasing the importance of the remaining weights.

Architecture-based CL methods

• To the best of our knowledge, our model is the first architecture-based CL model that uses hypernetwork, or, in general, any meta-model, for producing masks for other networks.
• Hypernetwork is a meta-model that builds masks by sharing information across subsequent tasks in the meta-model’s weights. Therefore, knowledge from previous tasks is stored in the hypernetwork weights and the masks produced are not completely independent.
• In consequence, we do not freeze any target model weights like in many architecture-based models.

Hypernetwork-based methods

• A single hypernetwork model produces sets of weights dedicated to each CL task. Updates of hypernetworks are prepared not directly for the weights of the main model, like in HNET [1], but for masks modulating the target model.
• HyperMask, unlike most of the methods used, produces trainable updates for a trainable model without freezing in any part.
• We show that hypernetworks may be used completely differently than classical hypernetwork CL models like HNET [1].

[1] J. von Oswald et al., Continual learning with hypernetworks, ICLR 2020.

Q2: The paper mentions that HyperMask has some limitations in terms of memory consumption due to the requirement for the hypernetwork's output layer to match the number of parameters in the target network. Are there any potential strategies or approaches to mitigate this memory consumption issue?

R2:

• We currently require that the hypernetwork output layer should match the number of parameters in the target network. Applying chunked hypernetworks similarly to [1] may mitigate this issue. In this architecture, the weights of the target model are created partially through so-called chunks; the number of chunks is another hyperparameter. Furthermore, other embedding vectors related to the chunk position must be created because the hypernetwork has to get information on which part of the target model weights will be produced. Similarly to the task embedding vectors, they are learned through back-propagation. Chunked hypernetworks will contribute to memory saving due to a lower number of hypernetwork weights (the smaller number of output neurons than in the case of the full hypernetwork architectures). Our initial experiments with chunked hypernetworks did not produce sufficient enough results in terms of classification accuracy but a more thorough analysis may solve this problem.

• We need a similar calculation time as in HNET [1] and more detailed results for calculations performed on a single GPU card are presented in the following table:

Dear Reviewer,

Thank you for your time and effort in reading our response! We hope our response has addressed your concerns. If you still feel unclear or worried, please let us know; we would be happy to clarify further and discuss any additional questions. If you feel your concerns have been addressed, please kindly consider if it is possible to update your score.

Thank you!

We thank the reviewer for the remarks regarding our paper, which will improve its quality. We answer all of your questions as follows.

Q1: There are some works mentioned in the related work using masks as an extension of the whole network, it is unclear what benefits hypernetwork can bring.

R1: The model combines two continual learning (CL) methodologies and provides a new concept using a meta-model to produce subnetworks. The masks produced by the hypernetwork modulate the weights of the main network and act like dynamic filters, enhancing the target weights important for a given task and decreasing the importance of the remaining weights.

Architecture-based CL methods

• To the best of our knowledge, our model is the first architecture-based CL model that uses hypernetwork, or, in general, any meta-model, for producing masks for other networks.
• Hypernetwork is a meta-model that builds masks by sharing information across subsequent tasks in the meta-model’s weights. Therefore, knowledge from previous tasks is stored in the hypernetwork weights and the masks produced are not completely independent.
• In consequence, we do not freeze any target model weights like in many architecture-based models.

Hypernetwork-based methods

• A single hypernetwork model produces sets of weights dedicated to each CL task. Updates of hypernetworks are prepared not directly for the weights of the main model, like in HNET [1], but for masks modulating the target model.
• HyperMask, unlike most of the methods used, produces trainable updates for a trainable model without freezing in any part.
• We show that hypernetworks may be used completely differently than classical hypernetwork CL models like HNET [1].

[1] J. von Oswald et al., Continual learning with hypernetworks, ICLR 2020.

Q2: There are several common loss functions are used in the method, and it is unclear if the improvements are from the proposed hypernetworks or the additional regularizations. There is no ablation study on these components.

R2:

• Hypernetwork regularization using previously trained embeddings is a classical approach in hypernetwork models, while L1 regularization of the target model is an often used approach in CL. These two components are combined with the model and together form the presented method consisting of two trainable networks. The first component is responsible for the regularization of the hypernetwork weights producing masks, while the second one accounts for the target network weights. Therefore, both parts are necessary for HyperMask because it consists of two trainable networks and we have to prevent radical changes in their weights. In HNET [1], hypernetworks directly produce the target model’s weights and additional regularization is unnecessary because only one model is trained.

• When we do not regularize the hypernetwork or main model, the weights are changed too drastically and catastrophic forgetting occurs. We present an ablation study for different hyperparameter settings responsible for the regularization strength (3 runs per one setup) on the Permuted MNIST dataset with 10 tasks using the same architecture, for which we presented results in Table 1 in our paper.

Q3: The experimental evaluation is very limited. It only compares with other methods on very tiny benchmarks. The performance gain from Table 1 seems not very significant. And it is hard to know how much more computation and complexity the method needs.

R3:

• We followed the evaluation protocols from HNET [1] and WSN [2]. Also, we performed experiments on TinyImageNet, a demanding dataset consisting of 40 continual learning tasks with 5 classes. A single image has a shape of 64 x 64 x 3. We prepared experiments using ResNet-20 architecture with an embedding vector of 96, β and λ equalling 1 and 0.1, respectively. Furthermore, p was set to 0, batch size to 16 and learning rate to 0.0001. Similar to WSN, the number of training epochs for each task was 10, and 50 validation samples from each class were selected in each CL task.

• We achieved state-of-the-art results, outperforming all reference methods. The mean classification accuracy for 5 runs of HyperMask was equal to 76.22%, while the best baseline, WSN, reached 71.96% using the architecture with 4 convolutional and 3 fully connected layers. Full results for TinyImageNet are depicted in the following table:

Results for TinyImageNet

• In addition to the results for Split CIFAR-100 and Tiny ImageNet, we show in the paper results for Permuted MNIST in 100 tasks with 10 classes and they are presented in Figure 2. It confirms that our model may be trained on continual learning datasets with many tasks. Especially interesting is the high result for the first task despite learning 99 of the subsequent ones. We additionally analyze this phenomenon with the t-SNE method for Permuted MNIST with 10 tasks (Figure 3) and Split MNIST with 5 tasks (Figure 6).

• The classical continual learning benchmarks are quite old and we think that state-of-the-art CL models are close to optimal results in such tasks. Our results are strong because we have two best results and two the second ones. We also achieved competitive results in a larger benchmark, i.e. TinyImageNet.

• We need a similar calculation time as in HNET [1] and more detailed results for calculations performed on a single GPU card are presented in the following table:

• According to the memory consumption, currently, we require that the hypernetwork output layer should match the number of parameters in the target network. Applying chunked hypernetworks similarly to [1] may mitigate this issue. In this architecture, the weights of the target model are created partially through so-called chunks; the number of chunks is another hyperparameter. Furthermore, other embedding vectors related to the chunk position must be created because the hypernetwork has to get information on which part of the target model weights will be produced. Similarly to the task embedding vectors, they are learned through back-propagation. Chunked hypernetworks will contribute to memory saving due to a lower number of hypernetwork weights (the smaller number of output neurons than in the case of the full hypernetwork architectures). Our initial experiments with chunked hypernetworks did not produce sufficient enough results in terms of classification accuracy but a more thorough analysis may solve this problem.

[2] H. Kang et al., Forget-free Continual Learning with Winning Subnetworks, ICML 2022.

[3] G. Gupta et al., La-MaML: Looking ahead meta learning for continual learning, In Advances in NeurIPS, 2020.

[4] D. Deng et al., Flattening sharpness for dynamic gradient projection memory benefits continual learning, In Advances in NeurIPS, 2021.

Dear Reviewer,

Thank you for your time and effort in reading our response! We hope our response has addressed your concerns. If you still feel unclear or worried, please let us know; we would be happy to clarify further and discuss any additional questions. If you feel your concerns have been addressed, please kindly consider if it is possible to update your score.

Thank you!