Mixture of Noise for Pre-Trained Model-Based Class-Incremental Learning

We sincerely thank you for the positive feedback on our novel recasting of forgetting, the comprehensive experiments, and the parameter efficiency. Please find our responses to the comments as follows:

Reply to W1 & Q1 & Q2

W1 & Q1 & Q2: Unclear statements in the introduction:

Reply: Thanks for pointing out some unclear expressions in Introduction. Due to limitations of rebuttal rules, we have to exclude the detail response of this part. We promise that we will polish the inexact statements in the revision. We can also reply these questions during the discussion phase.

Reply to W2

W2: Methodological complexity and clarity issues.

Reply: Thanks for your suggestions about improving the clarity of Sec. 4.2. We provide the pseudo code of training pipeline in Sec. 5 of supplementary.

According to your suggestion, we intend to revise the Sec. 4.2 to extend more details about the optimization process, such as visualizations of learned weight $\omega$ across the incremental phase.

Reply to W3 & Q4

W3 & Q4: Experimental details

Reply:

1. (Different task orderings) In Sec. 5.1, line 205-206, we state that the learning order is determined by the random seed 1993 and all methods are run 3 times to report the average results. This setting is the same as most of previous works (MOS, EASE, APER, and RanPAC).

2. (STD) We run baselines 3 times and report the average results. The results with stds are shown in Tab. 1, 2 and 3. We did not report the stds in the main paper limited by the table width. And most previous works also did not report the stds (MOS, EASE, ...).

Tab. 1

Tab. 2

Tab. 3

3. (Different learning orders) We perform the experiments of different random seeds {1993, 1994, 1995, 1996, 1997, 1998} in ImageNet-R 10 steps setting, and report the results in Sec. 2 of the supplementary. The results are shown in Tab. 4.

Tab. 4

4. (Conditions) We ensure all baselines are compared under fair conditions. First, all baselines adopt the same backbones (ViT-B/16-IN21K from timm library and ViT-B/16-IN1K downloading from Huggingface) and the exemplar-free setting. Second, they are trained with the same learning rate, weight decay, optimizer and scheduler. Third, their own hyper-parameters are kept default. Therefore, their performance may have slight differences from the original paper.

5. (MOS for CUB) MOS provides no results under 10 tasks setting. It just reports the results of CUB B0-Inc10, equivalent to the 20 steps setting in our paper since CUB contains 200 categories. Their reported result (93.49) is closed to the one we reported (92.62). In addition, most of reported results are close with the original papers, and in some datasets (Omni.) even higher than the original papers.

Reply to W4 & Q3 & Q5

W4 & Q3 & Q5: More ablation studies and sensitivity:

Reply:

1. (STD) Limited by the table width, we did not report the standard deviations in the main text. The results of the ablation study with the standard deviations are reported in the following table (Tab. 5). Compare row 3, 4 and 9, the results show that the contributions of each component are stable and effective.

Tab. 5

2. (Sensitivity of $d_2$) We have performed the experiment in sensitivity of the dimensionality $d_2$, and provide the results in supplementary materials (Sec. 4 in the supplementary). The results show that it achieves the best trade off between performance and the number of parameters when setting the ${d_2}=192$.
3. (Sensitivity of $\tau$) We keep the temperature ${\tau}=2$ in all experiments in the main text. According to your suggestion, we perform the sensitivity experiment on the temperature $\tau$. The results are shown in the following table (Tab. 6). The results indicate that the model performance is not sensitive to the hyper-parameter $\tau$.

Tab. 6

4. (More detailed ablation studies) We have explored more ablation studies with the Noise Mixture in Tab. 5.

   In Tab. 2, AvgN denotes the method with a simple average of noises, which is the same as the row 4 of Tab. 4 in the main text, LastN denotes the method using only the noise from last noise and RandomSN denotes the method with generating noise by selecting a task randomly.

   A comparison of experimental results for AvgN, LastN, and RandomSN demonstrates that integrating noise injection across multiple tasks is superior to single-task noise injection alone.

We sincerely thank you for highlighting the novelty of our noise perspective on parameter drift, the effectiveness of our designed filters, the framework's generalizability to other tasks, the consistent improvements and the sufficient ablation study, and the clear writing. Please find our responses to the comments as follows:

W1: Add some visualization or other high-level intuition explanations to make readers accept the noise perspective more easily and naturally.

Reply to W1: Thanks for your insightful suggestion. Due to the limitation of rebuttal rules, we cannot upload any figures. We intend to add the visualization of intermediate features before and after noise injection to intuitively demonstrate the impact of the beneficial noise on these features. Furthermore, heat-map visualizations of learnable weights $w$ will be added to intuitively illustrate the dynamics of noise learning.

Q1: Can your method be extended to the incremental learning in a generative model, for instance, to learn some new concepts to generate, but without destroying the previous concept generation？

Reply to Q1: It is interesting and promising to extend the MIN to generative model. We find highly related paper [1]. After reading the paper, we believe that MIN could potentially extend to generative incremental learning as its core mechanism.

By adding Pi-Noise layers to U-Net/DiT, it avoids destructive parameter drift in learning the new concepts to generate. This aligns with generative models' need to preserve core distributional features.

Its ability to suppress irrelevant activations indicates that it could also help generative models maintain fidelity to prior concepts. By masking inefficient patterns in intermediate features, it may prevent new concepts from overwriting existing latent representations.

[1] Continual learning of diffusion models with generative distillation, 3rd Conference on Lifelong Learning Agents (CoLLAs), 2024.

We sincerely thank you for recognizing the novelty of our beneficial noise perspective, the completeness of experiments with recent baselines and ablation studies, and the provision of source code for reproducibility. Please find our responses to the comments as follow:

Reply to W1 & Q4

W1 & Q4: Strengthen the distinctions and connection with existing theories, specifically Bayesian networks and uncertainty modeling.

Reply: Thank you for your insightful observation regarding connections to Bayesian Neural Networks (BNNs) and uncertainty modeling. We acknowledge that the introduction of noise and its generation might partly resemble some aspects of BNNs.

However, it is crucial to clarify that our MIN framework fundamentally differs in its objective and mechanism from traditional BNNs or uncertainty modeling. MIN adopts noise layer to construct a buffer of confusing feature rather than quantifying prediction uncertainty. The noise generators learn to mask cross-task interference patterns, analogous to targeting "confusion uncertainty" in feature space.

Specifically, although they are similar with each other (from the aspect of terminology and mechanism), the main differences are:

1. The primary function of this noise in MIN is direct feature manipulation for CIL, not uncertainty estimation.
2. While we sample noise from a standard normal distribution, this serves primarily as a reparameterization trick to enable backpropagation for training the noise generators. This approach is distinct from Bayesian methods, which aim to model posterior distributions to quantify epistemic and aleatoric uncertainties in parameters or predictions.

Reply to Q1

Q1: Does the inference require multiple samplings? Explain the reason.

Reply: A Pi-Noise layer is integrated into each layer of the model. During inference, each layer samples $\epsilon$ once and injects it into the features of that layer. Therefore, the model actually samples multiple $\epsilon$ during each inference pass, yet only needs to operate on a single $\epsilon$ per layer.

The forward propagation during training:

During the initial stages of training, models may indeed produce uncertain outcomes due to the randomness of injected noise. This uncertainty is precisely what building a robust classifier requires.

Specifically, a zero-mean noise sampled from $P_t$ parameterized by $\sigma$ is injected into input features. After supervised training, representations highly relevant to the current task acquire smaller $\sigma$ values to mitigate predictive uncertainty, whereas less relevant representations adopt larger $\sigma$ values.

Building upon this, ${\cal F}$ in Eq. (1) exhibits higher uncertainty toward low-correlation representations. Consequently, when calculating weights via Eq. (3), it reduces the weighting of representations with low correlations, thereby decreasing reliance on unrelated features. Therefore, the updated classifier exhibits robustness to low-correlation representations, and its output remains stable even without multiple sampling.

The forward propagation after training (during inference):

The main reasons why we just sample once are:

1. After well training MIN, the output remains stable across multiple independent noise sampling. And additional overload is introduced by multiple sampling but it does not improve the performance.
2. From the additional results in Tab. 1, the std of performance is rather small, which shows the stability without multiple sampling.

Reply to W2 & Q2

W2 & Q2: How about directly adopting a single parameter $\mu$ to make it a deterministic adapter? Clarify the connection with the design.

Reply: Thanks for your suggestion. We have explored more ablation studies in the following table (Tab. 1).

Tab. 1 Results of ablation study.

OnlyMu denotes the method adopting a single parameter $\mu$. However, the strategy of solely using $\mu$ demonstrates adverse effects in the most of datasets. Furthermore, we explore to adopt a single parameter $\sigma$ to generate a series of zero-mean noise. The experimental results (OnlySigmma) shows excellent performance.

We speculate that MIN mainly works by the uncertainties introduced by the $\sigma$. And the noise center $\mu$ provides a simple bias to increase the difference between tasks.

In the absence of $\sigma$, the classifier is influenced by numerous low-correlation representations. Directly adjusting the feature center using $\mu$ induces alterations in certain low-correlation representations that previous tasks rely on, resulting in performance degradation.

Conversely, after simultaneous application of $\mu$ and $\sigma$, the classifier becomes insensitive to low-correlation representations. Under these conditions, feature center adjustment with $\mu$ can enhance inter-task differences.

Reply to W3 & Q3

W3 & Q3: Clarify the motivation behind the auxiliary classifier.

Reply to W3 & Q3: The analytic classifier employed in our study determines the weights through an iterative linear regression approach, as detailed in Sec. 3 of the main text, which means this classifier does not employ back-propagation for gradient updates.

Consequently, an additional FC layer must be utilized as an auxiliary classifier to train the Pi-Noise layer, enabling proper gradient back-propagation.

We intend to incorporate the explanation for employing auxiliary classifiers within the main text. Appreciate your valuable suggestion.

We sincerely thank you for the positive feedback highlighting the clarity and ease of following our methodology, the high performance demonstrated in our experiments, and the novel insights presented in our work. Please find our responses to the comments as follow:

Reply to W1

W1: If F(·) is modified, does the solution presented in Eq. (3) still hold?.

Reply: The methods such as finetune and PEFT indeed update the ${\cal F}$ and learn new representations for a new task, so Eq. (3) does not hold. Therefore, as shown in the row "Finetune" of the following table (Tab. 1), even with an analytic classifier, fine-tuning the backbone network on each task still leads to severe catastrophic forgetting.

Tab. 1 Incremental trends of replacing the Pi-Noise with the different PEFT approaches under the setting of ImageNet-R 10 steps.

Instead of learning new representation like finetune and PEFT methods, the proposed MIN introduces additional noise layer to capture the confusing features while holding the original representations. The noise layer can be regarded as a kind of buffer of these confusing features, so that the representations of current task maintain more stable. Formally, the noise layer can be viewed as converting $z$ to $z+\epsilon$.

It essentially still leverages the representational power of the pretrained model. This noise layer does not disrupt the representation of the previous task, so Eq. (3) still holds.

Reply to W2

W2: More insightful discussion to further elaborate on the advantages of MIN over previous adapter-based mechanisms, specifically for CIL.

Reply : Current PEFT methods primarily focus on performance in new tasks by learning new representations with minimal parameters. However, the knowledge acquired from new tasks is destructive to previous tasks in CIL.

As shown in Tab. 1, after fine-tuning with three PEFT schemes separately for each task, all these methods (VPT-Shallow, VPT-Deep and Adapter) result in catastrophic forgetting. Furthermore, methods with greater learning capabilities (Adapter) exhibit a more rapid decline in performance.

In contrast, MIN does not learn new representation, so it can avoid drifting. The main differences and advantages are:

1. Different Motivation: MIN aims to adapt to downstream tasks through suppressing irrelevant feature representations by leveraging the powerful representational ability of PTMs rather than significantly updating the representations. The core premise is that the visual patterns captured by existing pre-trained models sufficiently encompass most downstream tasks.
2. Stability caused by noise layer: The core of MIN is the noise layer and it can be regarded as a kind of buffer of these confusing features, so that the representations of current task maintain more stable. Formally, the noise layer can be viewed as converting $z$ to $z+\epsilon$.

Reply to W3.1

W3.1: More ablation studies such as replacing the weighted summation in Eq. (13) with a simple average of the noises.

Reply: We appreciate your suggestion and have included additional ablation study results in Tab. 2.

In addition to the existing NE and NM components, we implement other five distinct methods as the alternatives for NM (Noise Mixture). In Tab. 2, AvgN denotes the method with a simple average of noises, which is the same as the row 4 of Tab. 4 in the main text, OnlyMu denotes the method with generating the deterministic noise only by vector $\mu$, OnlySigmma denotes the method with generating the zero-mean noise only by vector $\sigma$, LastN denotes the method using only the noise from last noise and RandomSN denotes the method with generating noise by selecting a task randomly.

A comparison of experimental results for AvgN, LastN, and RandomSN demonstrates that integrating noise injection across multiple tasks is superior to single-task noise injection alone.

In addition, we also add the ablation study suggested by Reviewer Cck3. Comparison of the experimental results between OnlyMu and OnlySigma demonstrates that the uncertainty introduced by the vector $\sigma$ is crucial for operation of MIN. In contrast, the deterministic adapter constructed solely from the $\mu$ vector negatively impacts the original baseline.

Tab. 2 Results of more ablation studies.

Reply to W3.2

W3.2: It would be insightful to analyze or visualize the final learned weights $w$ after incremental tasks are completed.

Reply: Thanks for your valuable advice. Due to the limitation of rebuttal rules, we can not upload any figures. We intend to add the visualizations of the learned weights $w$ across 10 tasks in the format of the heat map to illustrate the dynamics of weight $w$.

Dear Reviewer,

Thank you sincerely for your previous comments.

I wanted to follow up to ensure that we have adequately addressed all of your concerns from the initial review. Could you please let us know if our explanations and revisions have satisfactorily resolved the issues you raised? If there are any remaining concerns or if you need further clarification on any points, we would be happy to provide additional information.

We appreciate your time and valuable feedback throughout this review process.

Best regards,

All of Authors.