Test Time Adaptation with Auxiliary Tasks

Regarding the effect of the size of unlabeled source data stored on performance: We thank the reviewer for suggesting this experimental ablation. Based on the reviewer’s suggestion, we conducted experiments where we reduced the size of source data used for DISTA. The Table below shows the results on episodic evaluation on ImageNet-C where DISTA leverages different fractions of the validation set of ImageNet sampled uniformly at random (without labels).

We observe that DISTA is very robots against the size of $\mathcal D_s$. In particular, we find that even with 10% of the validation set (i.e. storing 5000 unlabeled images), DISTA improves over EATA by 1.4% on average across all corruptions. Furthermore, with even (1%) of source data, DISTA improves over EATA by 1% on shot and impulse noise. This table along with its discussion is now at Appendix B.4.

Regarding the trainable parameters: We thank the reviewer for this comment. As mentioned in appendix B.1, DISTA only updates the learnable parameters of normalization layers. This choice is common practice in the TTA literature [A, B, C]. We agree this is an important detail and decided to move this experimental detail to the main paper for better clarity.

Regarding the importance of $E_0$: We appreciate the reviewer’s comment. E_0 plays a vital role in filtering unreliable samples (samples with high entropy). We conducted experiments with two values of E_0 (0.4log(1000), 0.5log(1000)) and report the results in the table below.

We observe that varying E_0 results in performance variation due to filtering more/less samples from the update step. This result is in Table 11 in the Appendix.

We hope that our answers addressed the reviewers weaknesses and questions. We are happy to engage in further discussion to clarify any confusion or misunderstanding.

We thank the reviewer for their constructive comments. Next, we address the questions and weaknesses raised by the reviewer.

Regarding the use of unlabeled data instead of storing labeled examples in a replay buffer: We thank the reviewer for this remark. While we agree that in some applications, storing labeled training examples is feasible, in other scenarios (such as CLIP [A]), labeled training data can be a private property that cannot be accessed. This was the main rationale behind using unlabeled data for our DISTA. Nevertheless, and as per the reviewers request, we conducted experiments on DISTA with replacing the distillation loss with a cross entropy loss with the ground truth labels. The overall objective function for DISTA-Labeled is:

$\min_{\theta} \underset{x_t\sim \mathcal S}{\mathbb E} \lambda_t(x_t) E\left(f_\theta(x_t)\right) + \underset{(x_s, y_s) \sim \mathcal D_s}{\mathbb E} \lambda_s(x_s) CE\left(f_\theta(x_s),y_s\right)$

Our results on ImageNet-C benchmark are in the table below where we report error rates.

We observe that incorporating labeled data in DISTA instead of our distillation loss does not improve its results over using an unsupervised objective (while still providing state-of-the-art results compared to other baselines in our paper). We included this experiment in Appendix B.5 and referenced it in the main paper.

Regarding the justification for the federated evaluation: We thank the reviewer for this remark. We see three main federated adaptation scenarios: (1) All clients experience the same distribution shift (e.g. fog). This setup would amount to a straight-forward application of federated learning methods to TTA, and thus is not the most interesting. (2) Different clients observe completely different distribution shifts, which is the most challenging setting; (3) Different clients experience different domain shifts that share some similarities or, as in our experiments, come from the same category (“setup” in our paper). We are tackling this latter scenario as a first step towards the most general case in (2). Besides being a relevant stepping stone towards more challenging federated settings, scenario (3) is also interesting in and of itself and allowed us to demonstrate that federated averaging still accelerates adaptation even when clients experience different domain shifts.

Regarding the lookahead analysis: We appreciate the reviewer’s comment on the lookahead analysis. While indeed, one should quantify the lookahead based on performance measures (e.g. accuracy), the labels at test time are not provided by the stream. We emphasize that this is an inherent challenge in the test-time-adaptation problem where the model is required to predict and leverage unlabeled data for the adaptation strategy. Hence, and following the TTA setup, we extended the lookahead analysis to this unsupervised setup by defining it in terms of entropy as a proxy metric, that is known to be highly correlated to test accuracy. We believe that providing the lookahead in terms of accuracy might make the analysis less practical for the TTA setup where labels are never available.

Regarding experiments with batch size smaller than 8: We thank the reviewer for this comment. We note that we included experiments under batch size 1 in Table 9 in the appendix, where we compare DISTA against the state-of-the-art SAR under that setup. DISTA provided significant performance gains (around 5% reduction in error rate).

Regarding the effect of source data batch size: We thank the reviewer for this comment. Figure 2 (a) in the main paper shows exactly this effect varying the source data batch size (curve in orange color). We note that DISTA is robust to such variations as we see that, for example, even with 50% less source data (50% smaller batch size or 50% less frequent updates on source data), DISTA achieves an average error rate on ImageNet-C that is 1.4% percentage lower than that of EATA.

We thank the reviewer for the additional feedback.

Regarding the performance of DISTA+Labels: We note first, that DISTA + Labels is better than solely conducting entropy minimization (EATA). That is, including the additional supervised auxiliary task of leveraging source data is beneficial to the adaptation task (reduced error rate from 51.9 to 51.0 on average on ImageNet-C). Thus, we suspect that the supervised loss will have a positive lookahead. Due to the very narrow time limit until the end of the discussion period, we are unable to provide this lookahead analysis in our response. We commit to include the lookahead plots for the DISTA+labels baseline in the final version of our work.

Note that, However, DISTA+Labels performs worse than DISTA. We believe that the supervised loss that includes labels adds an additional complexity to the adaptation due to the imperfect performance of the trained models. That is, if a pretrained model is having a perfect performance with perfect calibration (i.e. predicting all samples correctly, and the correct class is assigned a probability of 1), then DISTA+Labels is reduced to DISTA. We hypothesize that the benefit of using source data in DISTA is that it regularizes the test-time adapted model to remain close to the original one in functional space. This probably eases the TTA optimization problem, since in the absence of labels for the test data, TTA methods have to resort to noisy proxies to the underlying classification objective, which might lead the model away from the original classifier and into poor local minima. In that light, it is reasonable to expect that the output of the original model output makes for a better regularization than the labels themselves.

Regarding the improvement that entropy minimization on source data can provide (Equation 2): We hypothesize that during the adaptation process, the adapted model overfits to the presented distribution from the stream while increasing its uncertainty on the clean data. However, including a simple entropy minimization on source data might mitigate partially this effect by providing a more stable adaptation process. We note that we confirmed the usefulness of this approach in Table 6 in the main paper where Aux-Tent improved the error rate over Tent by more than 1% on Gaussian, Shot, and Impulse noises, and by 0.6% on average on ImageNet-C confirming the earlier lookahead analysis in Figure 1 (a).

Regarding the effectiveness of using small portions of source data as $\mathcal D_s$: Coming back to the idea of regularization mentioned above, the intuition behind the usage of source data is not necessarily to maintain the best performance possible on the original dataset but to prevent the test-time-adapted model to diverge too much from the original classifier. Arguably, the latter is a much simpler task, and it seems that 1% of the data (500 images in this case) already includes enough variability to regularize the model effectively. Adding more source data improves such a regularization, but the regularization alone cannot drive the model’s performance, which explains why we quickly reach a regime of diminishing returns in terms of the availability of source data.

We hope that our answers addressed the reviewers questions. We are happy to engage in further discussion to clarify any confusion or misunderstanding.

Regarding the stochasticity of DISTA: We have repeated the experiments with DISTA (episodic evaluation on Gaussian Noise in ImageNet-C) 10 times, where we found that the maximum difference between two runs (in terms of error rate) is 0.1%. This shows that our method provides consistent results with very negligible stochasticity. We note that this stochasticity is due to either different order of data revealed from the stream (as pointed out in our second distinction between UDA and TTA) and the random sampling from $\mathcal D_s$.

Regarding including experiments on different models: We would like to thank the reviewer for this comment. In our experiments, we included 4 different architectures (ResNet50, ResNet18, ResNet50-GN, ViT) which exceeds the experimental complexity of several published works (e.g. [A, B, C]). Further, we considered a total of 8 different TTA baselines.

Regarding rerunning baselines or copying numbers from original papers: We re-evaluated the previous method and reproduced their reported results in the original paper. As mentioned in Appendix B.1, we used the original publicly available code for each method.

Regarding the effect of $\epsilon$ and $E_0$: We appreciate the reviewer’s comment. We note here that Both $E_0$ and $\epsilon$ are set following EATA with their recommended hyperparameters. Further, we conducted experiments with two values of E_0 and report the results in the table below.

We observe that varying E_0 results in performance variation due to filtering more/less samples from the update step. This result is in Table 11 in the Appendix.

Regarding ‘limited data’ in our abstract: We thank the reviewer for this comment. We would like to clarify here that we meant by limited data is the amount of data the stream reveals at a given time step $t$. TTA methods can only leverage this amount of data (which is limited by construction) for adaptation (our first distinction between TTA and UDA). We agree that a wealth of data exists, but it is made available in a sequential manner in test-time adaptation.

Regarding "conducting several adaptation steps ... will lead to overfitting": We appreciate this comment. We note here that since the stream $\mathcal S$ reveals data in batches for the adaptation method, conducting several adaptation steps will make the model overfit to the given batch as all the adaptation steps are performed on the same batch. To confirm this claim, we conducted an experiment with the previous state-of-the-art method EATA where we allow the model to adapt with 1, 2, 3, 4, 5 steps for each received batch from the stream. Results are reported in the table below:

We observe that the more adaptation steps EATA conducts on each revealed batch, the worse its performance becomes. This experiment confirms our claims and demonstrates the effectiveness of our auxiliary task in improving the performance with the additional adaptation step.

We hope that our answers addressed the reviewers weaknesses and questions. We are happy to engage in further discussion to clarify any confusion or misunderstanding.

We thank the reviewer for their valuable comments. We are glad that the reviewer recognizes the clarity and significance of our results. Next, we address the questions and weaknesses raised by the reviewer.

Regarding the relation between our work and unsupervised domain adaptation: We appreciate the reviewer’s comment and the interesting link to unsupervised domain adaptation, however, we disagree about casting our work as a reinvention of domain adaptation. We would like to clarify first the following: Differences between TTA and Unsupervised Domain Adaptation: The reviewer pointed out that the difference between test-time adaptatio and unsupervised domain adaptation is the access to source data during adaptation; specifically that in TTA source data is assumed to be unavailable. We respectfully disagree with that view, and in fact we are not the first to propose a TTA method that leverages source data. RMT [A], EATA [B] and ActMAD [C] were developed under the premise that source data is available and still are considered integral parts of the TTA literature. More recently, Kang et.al. [D] distilled source data for the sake of adaptation, as mentioned in our related work. All these examples demonstrate that the use of source data is accepted and not uncommon in the Test-Time Adaptation literature. While we do agree that there are similarities between TTA and UDA, we believe there are key differences in (1) the way the pretrained model is accessing the target distribution(s), and (2) the way an adaptation method is evaluated, as we elaborate below. (1) In Domain adaptation, the pretrained model can access the target domain all at once for adaptation. However, in Test-Time Adaptation, the pretrained model accesses the target domain as a stream of data. The model observes each revealed batch of data only once unlike domain adaptation where one could potentially revisit examples from the target domain multiple times. (2) The performance in domain adaptation is calculated after adaptation. That is, after a pretrained model adapts on a target domain, its performance on novel samples from the target domain is calculated. In Test-Time Adaptation, the performance of an adaptation method is calculated in an online manner where the model needs to provide the prediction of a given batch before receiving the next batch from the stream (please refer to the preliminary paragraph in our method section where we formalize the interaction between a TTA method and the stream of unlabeled data). Given these differences, we believe that our work, similar to [A, B, C, D], falls into the test time adaptation category where we followed the standard evaluation setups in the literature. While we do believe that cross-talk between the two fields is very interesting and important, we think that this deserves a work on its own and it falls outside the scope of our submission.Nonetheless, we are happy to include a discussion and comparison (if possible) on related unsupervised domain adaptation methods that the reviewer thinks are most related.

[A] Robust Mean Teacher for Continual and Gradual Test-Time Adaptation, CVPR 2023

[B] Efficient Test-Time Model Adaptation without Forgetting, ICML 2022

[C] ActMAD: Activation Matching To Align Distributions for Test-Time-Training, CVPR2023

[D] Leveraging proxy of training data for test-time adaptation, ICML 2023.

Regarding the confusion due to the use of “auxiliary tasks”: Response: In our work, we analyzed the effectiveness of 3 different auxiliary tasks. Equation (2) shows a simple entropy minimization on source data as auxiliary task while Equation (4) shows our proposed distillation-based auxiliary task. In Table 6, we reported the performance of Aux-SHOT which employs an information maximization on source data as an auxiliary task. Moreover, auxiliary tasks have proven useful in numerous machine learning applications, and we believe formalizing our contributions as auxiliary tasks might facilitate and encourage the study of new auxiliary tasks in TTA

Dear Reviewer

As the discussion period is about to end (22nd of November), we ask the reviewer to take a look at our response and modifications to the paper to address their questions and weaknesses. We are happy to engage further in discussions to clarify any confusion or misunderstanding.

Best,

We thank the reviewer for their constructive comments. We are glad that the reviewer recognizes the novelty of our auxiliary task for test-time adaptation and the effectiveness of our method. Next, we address the questions and weaknesses raised by the reviewer.

Regarding the intuition behind Equation (3): The main intuition behind Equation (3) is to solve the optimization problem in Equation (2) in an alternating optimization manner. This is because we want to analyze the impact of the auxiliary task (second term in Equation 2), and conducting this alternating optimization approach grant us an access to the intermediate model parameters $\theta_t^c$ and allows us to compare the performances of $\theta_{t+1}$ and $\theta_t^c$. We edited the paragraph before equation (3) to clarify that. Note that the expectations in equation (3) are taken over different data distributions (resp. corrupted and clean data distributions), and while the gradients w.r.t. $\theta$ are used to update $\theta^c_t$ to get $\theta_{t+1}$, the gradient is evaluated at $\theta_t^c$. That is,

$\theta_t^c = \theta_t - \gamma \nabla_\theta [E(f_\theta(x_t)]|_{\theta = \theta_t}$

$\theta_{t+1} = \theta_t^c - \gamma \nabla_\theta [E(f_\theta(x_t)]|_{\theta = \theta_t^c}$

We hope that this has clarified the intuition behind Equation (3).

Regarding the effect of the size of unlabeled source data stored on performance: We thank the reviewer for suggesting this experimental ablation. Based on the reviewer’s suggestion, we conducted experiments where we reduced the size of source data used for DISTA. The Table below shows the results on episodic evaluation on ImageNet-C where DISTA leverages different fractions of the validation set of ImageNet sampled uniformly at random (without labels).

We observe that DISTA is very robots against the size of $\mathcal D_s$. In particular, we find that even with 10% of the validation set (i.e. storing 5000 unlabeled images), DISTA improves over EATA by 1.4% on average across all corruptions. Furthermore, with even (1%) of source data, DISTA improves over EATA by 1% on shot and impulse noise. This table along with its discussion is now at Appendix B.4.

Regarding the use of unseen unlabeled data from the source distribution: We thank the reviewer for this great comment. Note that as mentioned in the footnote in page 6 that we also experimented with $\mathcal D_s$ being the validation set of ImageNet and did not notice any changes in our results. For completeness, we are attaching the results below. We report the results on ImageNet-C under episodic evaluation.

Regarding the federated evaluation: We thank the reviewer for this remark. We see three main federated adaptation scenarios: (1) All clients experience the same distribution shift (e.g. fog). This setup would amount to a straight-forward application of federated learning methods to TTA, and thus is not the most interesting. (2) Different clients observe completely different distribution shifts, which is the most challenging setting; (3) Different clients experience different domain shifts that share some similarities or, as in our experiments, come from the same category (“setup” in our paper). We are tackling this latter scenario as a first step towards the most general case in (2). Besides being a relevant stepping stone towards more challenging federated settings, scenario (3) is also interesting in and of itself and allowed us to demonstrate that federated averaging is still useful even when clients experience different domain shifts. Since in TTA performance is measured in an online manner, the information shared via federated updates can significantly speed up adaptation.

Is $f_theta$ in eq. (3) defaults to $f_theta$ at time $t$? In the first part of Equation 3, f_{\theta_t} is used to estimate the entropy, while in the right hand side, $f_{\theta_t^c}$ is used. We added a clarification description about Equation 3 in page 3.

Regarding the use of labeled data instead of distillation loss: as per the reviewers request, we conducted experiments on DISTA with replacing the distillation loss with a cross entropy loss with the ground truth labels. The overall objective function for DISTA-Labeled is:

$\min_{\theta} \underset{x_t\sim \mathcal S}{\mathbb E} \lambda_t(x_t) E\left(f_\theta(x_t)\right) + \underset{(x_s, y_s) \sim \mathcal D_s}{\mathbb E} \lambda_s(x_s) CE\left(f_\theta(x_s),y_s\right)$

Our results on ImageNet-C benchmark are in the table below.

We observe that incorporating labeled data in DISTA instead of our distillation loss does not improve its results over using an unsupervised objective (while still providing state-of-the-art results compared to other baselines in our paper). We included this experiment in Appendix B.5 and referenced it in the main paper.

Regarding mentioning the network architecture in captions: as per the reviewers request, we updated the captions of all tables in the main paper and appendix to mention the network architecture used.

We hope that our answers addressed the reviewers weaknesses and questions. We are happy to engage in further discussion to clarify any confusion or misunderstanding.

Dear Reviewer

As the discussion period is about to end (22nd of November), we ask the reviewer to take a look at our response and modifications to the paper to address their questions and weaknesses. We are happy to engage further in discussions to clarify any confusion or misunderstanding.

Best,