Knowledge Accumulating Contrastive Prompt for Continual Learning

Q1. The reviewer is concerned about the further improvement of the core insight of this paper, i.e., learning all tasks in a single prompt.

As the reviewer comments, we suggest the view that the single prompt has enough capacity to capture the knowledge of all tasks, which is verified by the upper-bound prompt $p^*$.

According to the reviewer's concern, we highlight the accuracy gap between the upper-bound model and the continual learning methods, as shown in the Table 1 of main paper. We believe that the single prompt can further tighten the gap, as the existence of the optimal point within the parameter space is already provided. Moreover, the accumulation of the task knowledge improves the generalizability of the model, as suggested in [1].

Moreover, our work is beneficial for task-agnostic setting, since using the single prompt allows the overlap between the tasks. In contrast, prompt pool stores the task-specific prompts, which requires the explicit task boundary.

[1]Wang, Qin, et al. "Continual test-time domain adaptation." in CVPR 2022.

Q2. Comparison for the training cost is required, as the method utilizes contrastive learning which may demand significant computation.

Following to the Reviewer's comment, we compare the training time per iteration for the validation sets of all tasks, as shown in Table 9.

For the training, our method has the similar cost with the prompt selection methods. The previous methods computes the token feature from the seperate frozen model, which is used to train the prompt key. Similarly, we compute the features from the previous model which is frozen during the current task. Moreover, our method utilizes the positive-only contrastive loss, which avoids the additional computational burden by the negative pairs.

Table 9. Comparison of training time for Split ImageNet-R.

Q3. While not required, I encourage the authors to make comparisons (at least conceptually) with HiDe-Prompt.

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work(11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper(28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research.

We summarize the differences between HiDe-prompt and our method, as follows:

• HiDe-prompt is based on the prompt selection process and prompt pool.
• Contrastive loss with negative pairs is suggested to remove the instance-wise and task-wise overlaps, which is different motivation with our method.
• Additional process and buffer memory are required to obtain old-task statistics.
• Not applicable for smooth transition of data distribution.

We provide the detailed explanation for each element. First, HiDe-prompt suggest to improve the prompt selection process for the prompt pool, using the representative power of the uninstructed features. Second, the method proposes the contrastive learning to reduce the overlap between the task, which requires negative pairs. Positive pairs $(h, \mu_c)$ are presented, where $h$ is feature extracted by the prompt and $\mu_c$ is corresponding class prototype of the current task. Also, negative pairs $(h,h')$ and $\lbrace$h,\mu_{c,i} $\rbrace_{i=1}^{t-1}$are defined to minimize the overlaps between the instances and tasks.

Third, the method utilizes the old-task statistics, which should be calculated for each training stage. It requires additional process and the memory. Lastly, the method requires the explicit task boundary which is not provided for the continual learning with smooth transition of the data distribution(i.e. G-CIFAR100 dataset). In contrast, our method is applicable for the task-agnostic dataset, as shown in Table 1 in the main paper.

Q1. Clarify the main advantage of single prompt compared to the prompt pool

We appreciate the constructive discussion suggested by the reviewer. According to the reviewer's concern, we highlight two main points of this paper as follows:

• The proposed method allows the overlaps between the tasks.
• We present the novel research direction for the prompt-based method, as the single prompt already has enough capacity.

We provide the detailed explanation for each element. First, we argue that our method allows the overlap, which enables our work to be applicable for the task-agnostic setting. Specifically, for the G-CIFAR100 dataset, we introduce 200 different tasks to the model, as the data distribution is smoothly changed. Then, the prompt-pool based methods requires 200 different task-specific prompts. Moreover, the overlap between the tasks induces the redundancy of the prompt, which also arises the difficulty in prompt selection process. In contrast, our method is free from the limitations as the prompt pool and the prompt selection process are not required. To support the claim, we provide the result for G-CIFAR100 by dual prompt with 200 task-specific prompts.

Table7. Comparison with task-specific prompts for task-agnostic setting (G-CIFAR100)

Second, we suggest that the single prompt already has an enough capacity to instruct the model for all tasks, which is verified by the upper-bound prompt $p^*$. Hence, we claim that the primary concern is the optimization process, as the existence of the optimal point is provided. Accordingly, we propose the novel research direction of the prompt-based continual learning methods to approach the optimal point by the single prompt, which removes the need of the prompt pool and the selection process.

Q2.More discussions and experiments for each loss term, $L_{ctr}$ and $L_{prev}$.

As introduced in the main paper, $L_{ctr}$ captures the embedding space to pull the defined positive pairs, and $L_{prev}$ aligns the feature space between the old model and the current model. Specifically for $L_{prev}$, we kindly remind the plasticity-stability tradeoff suggested in the section 6 of the main paper. $L_{prev}$ enhances the alignment of the feature space, which contributes to the stability of the model for the continual learning.

Following to the reviewer's comment, we present the additional ablation study for the loss term, as follows:

Table8. Additional ablation study for loss terms

The result show that increasing $L_{prev}$ supports that the loss term alleviates the forgetting by enhancing the stability of the model. In case of $L_{ctr}$, the loss term contributes to the alleviated forgetting, however, increased $\lambda_{ctr}$ lowers the accuracy, suggesting degraded stability-plasticity tradeoff.

Comment4. Detailed explanation about the upper-bound prompt and its performance.

The reviewer is kindly reminded that we have already provided the performance of the upper-bound prompt in the Table 1 in the main paper. In section 3.1 of main paper, we specified the upper-bound prompt $p^*$ which is optimized by the merged dataset of all tasks (i.e. $\mathcal{D}_{all}=\mathcal{D}_1 \cup \dots \cup \mathcal{D}_T$). Hence, the $p^*$ instructs the model to perform well for all tasks. In Table 1 in the main paper, we present the accuracy by the $p^*$ denoted as `Upper bound'.

Comment5. The reviewer commented to present more results using different pretrained models

We provide the ablation study for different pretrained models in Table 3. The result shows that our method consistently outperforms DualPrompt and L2P for all models. The comparison with CODA-prompt shows dependency for model architecture.

Table4. Ablation study for pretrained model with related methods by 10-split-CIFAR100. We set learning rate and the scheduler identical to the main paper, for fair comparison

Comment1. Provide more evidence for supporting the advantage of single prompt compared to task-specific prompt.

We appreciate for the constructive comments. We highlight that the task-specific prompts require the explicit task boundary, which is not provided for the task-agnostic setting. Specifically, G-CIFAR100 dataset introduces 200 different tasks, allowing the overlaps between tasks. According to reviewer's concern, we present the result of DualPrompt with 200 task-specific prompts.

Table1. Comparison with task-specific prompts for task-agnostic setting (G-CIFAR100)

As shown in the table, the methods with task-specific prompt require more parameters, which becomes redundant by the overlaps between the tasks.

Comment2. The comparisons with recent related works, including CODA-Prompt are required.

In response to the reviewer's comment we present the result of the CODA-prompt by our implementation, as follows:

Table2. Comparison with related approaches by 10-split-CIFAR100. ViT-B/16 model pre-trained on ImageNet-21k is used. We set learning rate and the scheduler identical to the main paper, for fair comparison.

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work(11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper(28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research.

We provide the conceptual comparison with HiDe-prompt, which is summarized as follows:

• HiDe-prompt is based on the prompt selection process and prompt pool.
• Contrastive loss with negative pairs is suggested to remove the instance-wise and task-wise overlaps, which is different motivation with our method.
• Additional process and buffer memory are required to obtain old-task statistics.
• Not applicable for smooth transition of data distribution.

First, HiDe-prompt propose to improve the prompt selection process for the prompt pool, by the uninstructed features. Second, the method proposes the contrastive learning to reduce the task-wise overlap, which requires negative pairs. Positive pairs $(h, \mu_c)$ are presented, where $h$ is feature extracted by the prompt and $\mu_c$ is corresponding class prototype of the current task. Also, negative pairs $(h,h')$ and $\lbrace$h,\mu_{c,i} $\rbrace_{i=1}^{t-1}$are defined to minimize the overlaps between the instances and tasks.

Third, the method utilizes the old-task statistics, which should be calculated for each training stage. It requires additional process and the memory. Lastly, the method requires the explicit task boundary which is not provided for the continual learning with smooth transition of the data distribution(i.e. G-CIFAR100 dataset). In contrast, our method is applicable for the task-agnostic dataset, as shown in Table 1 in the main paper.

Comment3. More discussion for previous regularization, including LwF is required

We appreciate for the comments. The reviewer is kindly reminded that we have present the comparison with the regularization methods, in Table 7 of supplementary material. We compare our method with the previous related methods, which are AFC, PODNet(POD-flat loss), and VID. According to the comment, we provide the result for KD loss which is similar loss with LwF.

Table3. Result by Split-CIFAR100 using pretrained ViT by ImageNet-21k.

Our method regularize the model in the feature space, differently to the KD loss and the LwF. We compare the KD and LwF in response to the reviewer’s comment. KD loss regulate the current model’s logit by old model’s logit, which requires both the current prompt $p_t$ and the old prompt $p_{t-1}$. In constrast, LwF utilize the prompt $p_t$ and the memorized last linear layer $\theta_o$.

The Table 3 suggests that the regularization by old models helps to alleviate the forgetting, rather than the additional gradient by memorized $\theta_o$. Moreover, in our framework, we can easily get the old model by using the old prompt $p_{t-1}$. The results supports our approach, increasing the mutual information between the $p_t$ and $p_{t-1}$, to accumulate the knowledge.

Comment 1. The reviewer suggested the detailed comparison with the recent related works, including S-Prompt and CODA-Prompt.

Thanks to the reviewer's fruitful comment, we recognize the HiDe-prompt which is a very recent work (11 October, 2023 Arxiv and accepted for NeurIPS 2023) publicly released after the submission of our paper (28 September, 2023). We clarify that we could not recognize HiDe-prompt during our research.

In response to the reviewer’s comment, We provide the comparison with the related work, as follows:

Table 5. Comparison with related works.

Comment 2. How the performance changes if the parameters of the model are not frozen?

According to the reviewer's comment, we changed the parameters of the model to be learnable. Then, the result shows a severe forgetting. We present the task accuracy after the task 3, for the brevity of the response.

Table 6. Task accuracy after training task 3 by unfrozen model.

Comment 3. The reviewer commented the change of performance by the prompt length

The reviewer is kindly reminded that we have already provided the ablation study for prompt length, in the Table 8 in the supplementary material. We enlarged the prompt parameters by increasing the prompt length. The result shows the insensitivity of the performance to the length of the prompt.

Comment 4. The reviewer commented the change of performance by the prompt length

We appreciate for the constructive discussion. The reviewer is kindly reminded that we have presented the effect of misalignment of the prompt in the Table 3 in the main paper. As DualPrompt predicts the task identity to select the task-specific prompt, we calculate the accuracy of the task prediction, denoted as `Task ID Accuracy'. The accuracy is below 60$\%$, and the accuracy degrades as the number of the tasks increases. In case of L2P, the prompt index is trained by the diversity regularization, without the ground-truth index. Accordingly, the accuracy for task prediction cannot be defined. Instead, we suggest the result of task-agnostic setting(e.g. G-CIFAR100) in Table 1 of the main paper. The lower performance of L2P compared to our method indirectly indicates the misaligned prompt problem of L2P.