Binary-Feedback Active Test-Time Adaptation

Dear Reviewer ZvDw,

We sincerely appreciate your constructive feedback on our work. We have substantially revised the paper to address all raised concerns and strengthen our contribution. We provide detailed responses to each point:

W1. Considerations behind reward function design for BFA and ABA.

• We thank the reviewer for this insightful question about our reward function design. The different reward structures for BFA and ABA were carefully chosen based on the reliability of each signal. In BFA, we have explicit binary feedback from the oracle, so we can confidently assign a negative reward (-1) to incorrect predictions to actively discourage them. However, in ABA, prediction disagreement doesn't necessarily indicate incorrect predictions - as demonstrated in Figure 4(b), disagreement samples show mixed accuracy rather than consistent incorrectness. Therefore, rather than actively penalizing these potentially noisy signals with negative rewards, we simply excluded them from the adaptation process by setting their reward to 0. This design choice allows our method to leverage confident predictions while gracefully handling uncertain cases without introducing potentially harmful adaptation signals. We added this clarification to the paper to better explain the rationale behind our reward function design.

W2. How the baselines of previous TTA and ATTA are adapted to fit into the proposed setting to ensure fairness.

• We appreciate this important point about experimental transparency. We have substantially expanded the implementation details in Appendix D.2 to explain how we adapted existing TTA and ATTA methods to incorporate binary feedback. For TTA baselines, we modified their objectives to: $L = L_{TTA} + L_{CE} + L_{CCE}$, where L_TTA is the original TTA loss, L_CE is the cross-entropy loss on correct samples, and L_CCE is the complementary cross-entropy loss on incorrect samples following Kim et al. [1]: $L_{CCE} = -∑^{num\_class}_{k=1} y_k \log (1 - f_θ ( k | x ) )$. For the active TTA baseline (SimATTA), we adapted its clustering-based sample selection while modifying its supervision signal to use both correct and incorrect binary feedback using the same loss formulation above. The complete implementation details, including hyperparameter settings, are now documented in Appendix D.2.

[1] Youngdong Kim, Junho Yim, Juseung Yun, and Junmo Kim. Nlnl: Negative learning for noisy labels. In Proceedings of the IEEE/CVF international conference on computer vision, 2019.

W3. Frequency of Human Intervention: The requirement of annotating 3 samples per batch of 64 suggests a substantial annotation budget, even with binary feedback. The need for human intervention in every test batch implies that annotators must be continuously available while the system operates. Reducing the annotation budget and frequency could make the system more practical.

We thank the reviewer for raising this important practical consideration about annotation frequency. Our experiments demonstrate BATTA-RL's robustness across various annotation frequencies, maintaining strong performance with as few as 1-2 binary feedbacks per batch (85.02-86.23% accuracy) and during intermittent labeling scenarios where annotations are only available for a fraction of batches (84.56-85.83% accuracy).

• First, our setting of 3 binary-feedback samples per batch of 64 (less than 5%) was chosen to match the sample budget used in previous active TTA work [1] for a fair comparison. Importantly, binary feedback is significantly less intensive than full-label annotation - requiring only a yes/no response versus selecting from all possible classes.

• Second, as shown in Figure 7 in the manuscript, BATTA-RL maintains strong performance compared to the baseline (SimATTA*) in varying numbers of annotations per batch. We additionally conducted 1-2 binary feedbacks per batch, and the results demonstrated that BATTA-RL remains effective with a few samples. We updated Figure 7 in the manuscript correspondingly.

Table R7. Accuracy (%) comparisons with different numbers of annotations per batch.

• Furthermore, we conducted new experiments during the rebuttal period to evaluate scenarios where annotators are not continuously available. We set the scenario with intermittent labeling where the labels are only partially available (for ½, ⅓, and ¼ of total batches). As in the table below (corresponding to Figure 11 in Appendix B), compared to the baseline (SimATTA*), our BATTA-RL maintains robust performance even with intermittent human feedback.

Table R8. Accuracy (%) comparisons with different numbers of labeling skipped batches.

• These results demonstrate our method's flexibility in balancing performance and annotation frequency based on practical constraints. We agree that exploring even more annotation-efficient strategies is an important direction for future work and would be happy to expand this discussion in the paper.

[1] Shurui Gui, Xiner Li, and Shuiwang Ji. Active test-time adaptation: Theoretical analyses and an algorithm. In International Conference on Learning Representations (ICLR), 2024.

W4. Computational complexity of BATTA-RL.

• We thank the reviewer for this important practical concern. Please check our new wall-clock time analysis in the global response.

Dear Reviewer afeQ,

We sincerely appreciate your constructive feedback on our work. We have substantially revised the paper to address all raised concerns and strengthen our contribution. We provide detailed responses to each point:

W1. The use of multiple epochs hinders real-time inference capabilities.

• We appreciate this important concern about real-time practicality. While our main results use multiple epochs to demonstrate BATTA-RL's full capability, we conducted additional experiments examining performance under smaller epochs. Our analysis in the table below shows BATTA-RL maintains strong performance even with single-epoch adaptation: On CIFAR10-C, reducing from 3 epochs to 1 epoch (with proportionally adjusted learning rate ×3) achieves 87.11% accuracy compared to 87.20% with 3 epochs. Similar robust performance is observed with 2 epochs (87.15%). This demonstrates that BATTA-RL can effectively adapt in scenarios where multiple epochs are impractical. We have added these findings in Section B of the Appendix, with results as in the table below (corresponding to Table 6 in Appendix B) showing consistent performance across different corruption types under single-epoch adaptation. These results suggest BATTA-RL is viable for real-time applications while maintaining its advantages over existing methods.

Table R6. Accuracy (%) comparisons with various epoch settings.

W2. Scalability of BATTA under large-scale datasets (e.g., ImageNet-C) and marginal improvement in Tiny-ImageNet-C.

• Thank you for the suggestion. Our experiment setting is acknowledged to be comprehensive (Reviewer ZvDw), and we understand that adding further experiments would strengthen our paper.

• We conducted a thorough additional experiment and discussed in the global response. Superior performance on various large-scale datasets, including ImageNet-C, demonstrates the effectiveness of our dual-path optimization framework. Please check our global response to address your concern.

Dear Reviewer Pxu6,

We sincerely appreciate your constructive feedback on our work. We have substantially revised the paper to address all raised concerns and strengthen our contribution. We provide detailed responses to each point:

W1. Compare with feature augmentation-based uncertainty estimation.

• We appreciate this insightful suggestion about comparing uncertainty estimation strategies. Following your recommendation, we conducted additional experiments comparing our MC dropout-based uncertainty estimation with augmentation-based approaches used in MEMO and CoTTA. Our findings reveal that augmentation-based uncertainty estimation leads to severe performance degradation (17% accuracy) in the BATTA setting, compared to our MC dropout approach (87.20% on CIFAR10-C). This substantial performance gap reveals a critical limitation: augmentation-based uncertainty estimates tend to overfit in the early adaptation stage, making them unreliable for active sample selection in our binary feedback setting. In contrast, MC dropout provides more stable uncertainty estimates by directly capturing the model's epistemic uncertainty, leading to more reliable sample selection for binary feedback queries. We have added these comparative results and detailed analysis in Appendix B, which demonstrate why MC dropout is better suited for uncertainty estimation in the binary feedback setting.

Table R5. Accuracy (%) comparisons of BATTA-RL original version and augmentation-based uncertainty estimation.

W2. Wall-clock time comparison of the method.

• We thank the reviewer for this important practical concern. Please check our new wall-clock time analysis in the global response.

W3. Impact of sample selection for BFA in BATTA-RL.

• We agree with this important suggestion about validating our sample selection strategy. We have findings in Figure 9 in Appendix B of our paper - we comprehensively evaluated various sample selection strategies, including random selection, maximum entropy, minimum confidence, and minimum energy, against our MC-dropout uncertainty-based selection. The results demonstrate that our approach significantly outperforms random selection and other baseline strategies in CIFAR10-C. This empirically validates that selecting samples based on MC-dropout uncertainty is more effective than random selection for guiding the adaptation process. We believe these results provide strong evidence for the effectiveness of our sample selection method in the BFA module.

Dear Reviewer 8qrn,

We sincerely appreciate your constructive feedback on our work. We have substantially revised the paper to address all raised concerns and strengthen our contribution. We provide detailed responses to each point:

W1. The signal acquisition process may be impractical; relying on an annotator could be costly or unrealistic in certain settings. However, advancements in foundation models may help address this issue soon.

• We truly agree that end-user annotation is costly and limits the applicability of full-label active TTA. This concern about labeling burden motivated our development of binary-feedback active TTA (BATTA) as a more practical alternative to full-label annotation. The novelty and usefulness of the BATTA setting are recognized by Reviewer afeQ and ZvDw.

• We strongly agree with the reviewer's insight about leveraging foundation models as oracles - this represents an exciting future direction that could further reduce dependency on human annotators while maintaining the benefits of our binary feedback framework. We plan to explore this direction in future work and would welcome the opportunity to discuss it further in the paper.

W2. The computational cost of MC-dropout.

• We appreciate this important concern about MC dropout's computational overhead. Please check our new wall-clock time analysis in the global response.

W3. The fairness of using multiple forward passes.

• We thank the reviewer for raising this important point about experimental fairness. We respectfully disagree that our use of MC-dropout creates an unfair advantage. Most state-of-the-art TTA methods also employ various forms of ensemble or multiple forward passes: SAR/SoTTA require double forward and backward passes for sharpness-aware minimization, and CoTTA/RoTTA use multiple augmentation-based forward passes with a teacher-student framework with multiple predictions. Considering the strongest TTA baselines are using multiple forward passes, we believe our experimental comparisons are fair and meaningful.

Dear Reviewer uYWh,

We sincerely appreciate your constructive feedback on our work. We have substantially revised the paper to address all raised concerns and strengthen our contribution. We provide detailed responses to each point:

W1. Experiments on additional datasets (ImageNet-C, ImageNet-R, and VisDA-2021) and scenarios (test label distribution shift, single test sample, and combination of multiple distribution shifts).

• Thank you for the suggestion. Although we still believe that our experiment setting is comprehensive (as acknowledged by Reviewer ZvDw), we understand that adding further experiments would strengthen our paper. We provide additional experimental results in the global response. For the combination of multiple distribution shifts (mixed data streams), we already included the experiments in Table 2 of the manuscript, demonstrating our BATTA-RL’s superior performance where distributions are mixed.

W2. Burden of human feedback on large-scale datasets.

• While large-scale datasets may indeed have fewer confident samples, our additional experiments on ImageNet-C/R during the rebuttal period demonstrate that BATTA-RL maintains its superior performance even on these more challenging datasets. This scalability stems from BATTA-RL's unique dual-path optimization that leverages both binary feedback and unlabeled samples, unlike SimATTA, which relies solely on labeled samples.

• Also, binary feedback remains significantly more efficient than full labeling. This efficiency becomes particularly evident in our experimental results - as shown in Figure 5 when controlling for equal labeling cost, BATTA-RL substantially outperforms full-labeled SimATTA across datasets, with the performance gap actually widening for datasets with more classes (9% improvement on CIFAR-10, 30% on CIFAR-100, and 32% on Tiny-ImageNet).

W3. Benchmarks with spurious correlations.

• To investigate the impact of spurious correlations, we evaluated BATTA-RL on ColoredMNIST, an important benchmark in DeYO [1]. Our new experiments show BATTA-RL achieves 96.75% accuracy on ColoredMNIST, significantly outperforming standard TTA methods (45.59-82.70%) and active TTA baseline SimATTA (93.69%). This strong performance on ColoredMNIST suggests that BATTA-RL's dual-path optimization effectively mitigates the impact of spurious correlations. Binary feedback helps identify when the model relies on spurious features and produces wrong predictions, while agreement-based self-adaptation prevents overconfident predictions on misleading patterns. We have added these results in Table 3 in Section 4 of the manuscript.

Table R4. Accuracy (%) comparisons on spurious correlations.

[1] Lee, Jonghyun, et al. "Entropy is not enough for test-time adaptation: From the perspective of disentangled factors." arXiv preprint arXiv:2403.07366 (2024).

Revision

The revisions made are marked with “$\text{\color{blue}blue}$” in the revised paper.

• Section 3.2: Design considerations on the reward function (Reviewer ZvDw).
• Section 4: Additional results on datasets (Reviewer uYWH and afeQ), additional analysis on the runtime (Reviewer 8qrn, Pxu6, and afeQ)
• Figure 7: Additional results on the number of active samples per batch (Reviewer afeQ).
• Appendix B: Additional analysis on the number of epochs (Reviewer afeQ), use of augmentation (Reviewer Pxu6), and intermittent labeling (Reviewer afeQ).
• Appendix C: Additional results on scenarios (Reviewer uYWH).
• Appendix D.1: Explanations on TTA with binary feedback (Reviewer ZvDw).

We are confident these revisions have significantly improved the paper's clarity and technical depth. We look forward to any additional feedback that would further strengthen our contribution.

Sincerely,

Authors