Test-Time Adaptation with Binary Feedback

We sincerely thank reviewer R67W for the comprehensive review and for highlighting both the strengths and potential areas of improvement in our paper. We appreciate your recognition of our reduced labeling costs and dual-path optimization framework.

Open-sourcing. We truly agree with your suggestion. We have already included source code zip files in the supplementary materials in the original submission, and we will open-source the code and include the corresponding repository link in the final version.

Theoretical Analysis. We thank the reviewer for highlighting the value of a theoretical comparison to full-label adaptation methods. Our theoretical analysis between binary feedback and full-label feedback from an information-theoretic perspective is in $\text{\color{blue}Appendix D.1}$, which shows that binary feedback provides log(num_classes) times less information than full-class labeling per sample, offering lightweight labeling costs. Our empirical results also suggest that binary feedback provides enough information to drive effective adaptation under domain shift. This is further supported by a comparison with full-class labeling under equal labeling cost, where BiTTA shows superior performance over the full-label active TTA baseline ($\text{\color{blue}Figure 5}$). These insights imply that BiTTA retains strong adaptation capabilities despite more limited supervision, supporting its practicality in real-world scenarios where annotation time is constrained.

Scalability and Robustness. Please note that we did include detailed per-corruption results for ImageNet-C in $\text{\color{blue}Table 6}$ in $\text{\color{blue}Appendix B}$. We will move them to the main paper in our final draft. In summary, BiTTA achieves 36.59% on ImageNet-C, outperforming all baselines by 12.0%p on average, demonstrating its scalability to larger datasets.

Regarding high model error scenarios, our experiments on challenging datasets (Tiny-ImageNet-C with source accuracy 21.48%) show that BiTTA still provides substantial improvements (+19.37%p), highlighting its robustness to high-error settings. This is because our agreement-based adaptation (ABA) can leverage samples with reliable prediction, complementing many "incorrect" labeled samples from binary feedback samples, leading to stable adaptation.

Frequency of Human Intervention. We did include additional experiments where we reduced the frequency of human intervention (e.g., labeling only 1 out of 4 consecutive batches) in $\text{\color{blue}Figure 11}$ in $\text{\color{blue}Appendix B}$. Results show that BiTTA maintains strong performance even with significantly reduced intervention frequency (outperforms the baseline by 9%p on average), suggesting that practical implementations could use less frequent feedback while maintaining performance benefits. During the rebuttal, we further examined the impact of delayed feedback where binary feedback arrives after a few batches ($\text{\color{red}Rebuttal for QyDK: Delays or inability to obtain timely feedback}$). The results showed that delayed feedback can derive the same level of adaptation performance. This showcases the practicability of our system.

We appreciate your constructive feedback and will carefully revise our manuscript accordingly. Given the strengths highlighted in our submission, we hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.

We thank reviewer SHhn for the detailed review and thoughtful questions that helped us improve our work. We appreciate your recognition of our novel problem setup and methodology.

Clarification regarding Fig. 1 caption. Traditional TTA methods indeed struggle with severe distribution shifts, which have been reported in the literature. In terms of the risk of adapting to incorrect predictions from unlabeled samples, we would tone down and further support with theoretical insights from ATTA ($\text{\color{blue}Gui et al., 2024}$), where labeled test instances can enhance the overall performance compared with unlabeled adaptation. We will update the caption to clarify this.

MC-dropout vs. softmax probabilities and sample selection (Q2). MC-dropout offers more robust uncertainty estimates than standard softmax probabilities because (1) it captures epistemic uncertainty (model uncertainty) through multiple stochastic forward passes ($\text{\color{blue}Gal and Ghahramani, 2016}$) and (2) Softmax probabilities are known to be overconfident on out-of-distribution samples ($\text{\color{blue}Lee et al., 2024b}$).

During the rebuttal, we compared the expected calibration error (ECE) between MC-dropout and original softmax probabilities. We found that MC-dropout confidence (avg. ECE 0.062) shows 33% lower ECE than the original softmax confidence (avg. ECE 0.100) during all corruptions, showcasing MC-dropout provides a more robust uncertainty estimate.

Also, in $\text{\color{blue}Figure 9}$ in $\text{\color{blue}Appendix B}$, we reported that our MC-dropout-based uncertain sample selection outperforms the original confidence-based sample selection, demonstrating a robust uncertainty estimation of MC-dropout.

Details of MC-dropout. We used N=4 iterations of dropout inferences to implement MC-dropout. We will open-source the code and integrate details of MC-dropout in the final manuscript.

Usage of foundational models for feedback. We agree that current state-of-the-art foundational models (e.g., GPT-4o) are erroneous for active labeling (please note that we used foundational models for generating full-class active labels in $\text{\color{blue}Figure 5}$). This necessitates lightweight labeling from an oracle, such as our proposed binary feedback. Please let us know if you need further clarification.

Questions about EATA performance and noise induction. EATA's performance decreases on CIFAR-100-C compared to CIFAR-10-C. This phenomenon has also been observed in previous literature ($\text{\color{blue}Lee et al., 2024b}$). This explains the vulnerability of unlabeled adaptation; if an early adaptation stage leads to an erroneous direction of adaptation, simple entropy adaptation methods such as EATA and TENT can lead to model failures where most of the predictions become incorrect. Since model failures lead to over-confidence in incorrect samples, simple confidence/entropy thresholding in EATA cannot filter out wrong predictions.

Experiment setting for the impact of labeling errors. For evaluating the impact of labeling errors ($\text{\color{blue}Figure 6}$), we randomly flipped binary feedback labels (correct↔incorrect) at specified rates (10%, 20%, 30%) to simulate annotation errors.

Additional results on DeYO, OWTTT, and TAST (Q1). Thank you for suggesting these recent works. During the rebuttal, we experimented with DeYO and TAST in $\text{\color{red}Table R4}$ and OWTTT in $\text{\color{red}Table R5}$. We observed that our BiTTA still outperforms the latest baselines.

Table R4. Average accuracy (%) comparisons. Notation * indicates the modified algorithm to utilize binary-feedback samples.

Table R5. Average accuracy (%) comparisons under the OWTTT pre-trained model. Notation * indicates the modified algorithm to utilize binary-feedback samples.

In addition, during the rebuttal, we experimented on replacing MC-dropout with an ensemble structure from TAST. The ensemble method showed 1.7%p lower accuracy than the original MC-dropout-based algorithm. This emphasizes that while our dual-path RL optimization framework is flexible to incorporate ensemble methods (e.g., TAST), our proposed MC-dropout policy estimation shows the best performance.

Rebuttal on Other Comments or Suggestions. Thank you for pointing out typos and suggesting improvements. We will update the final manuscript.

We will revise our final manuscript to reflect the rebuttal. Given the strengths highlighted in our submission, we hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.

We sincerely thank reviewer QyDK for the positive feedback on our work and for recognizing our paper's dual-path optimization strategy and its contributions.

Delays or inability to obtain timely feedback. During the rebuttal, we conducted an additional experiment where active adaptation algorithms (BiTTA and SimATTA) receive the labeled samples in delayed batches (e.g., labeled instances arrive 1/2/3 batches later). The results in $\text{\color{red}Table R3}$ suggest that the delayed feedback shows a negligible impact on the performance of TTA with binary feedback, further enhancing the practicality of the feedback framework. We will include the result in the final manuscript.

Table R3. Accuracy (%) comparisons under delayed feedback in CIFAR10-C. Zero delay is equivalent to the original experiment setting. Notation * indicates the modified algorithm to utilize binary-feedback samples.

Also, we did include experiments where annotators skipped labeling for several batches (e.g., labeling only 1 out of 4 consecutive batches, $\text{\color{blue}Figure 11}$ in $\text{\color{blue}Appendix B}$). The results show that compared with the baseline, BiTTA maintains stable performance even with such intermittent feedback, outperforming SimATTA (active TTA baseline) by 9.22%p.

Noisy/erroneous feedback scenarios. We did include BiTTA under noisy/erroneous feedback scenarios in $\text{\color{blue}Figure 6}$ in $\text{\color{blue}Section 4}$, where BiTTA maintains robust performance even with up to 30% incorrect binary feedback, constantly outperforming SimATTA significantly by 7.81%p.

We will include these additional results in the final paper to address practical deployment concerns. We believe our TTA with binary feedback framework and proposed BiTTA algorithm is practical in real-world scenarios with delayed or noisy feedback. Given the strengths highlighted in our submission, we hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.

We sincerely thank reviewer wnVe for the thoughtful evaluation of our work and recognition of our novel problem setting and technically sound framework.

Clarification on Implementation Details. While we provided the source code and additional implementation details in $\text{\color{blue}Appendix D}$, we will review them again to include all details.

Computational Complexity, Q8: Runtime Memory Comparison. Thank you for the suggestions. During the rebuttal, we analyzed the memory consumption in $\text{\color{red}Table R1}$. While MC-dropout increases computational load, the overhead remains manageable (also acknowledged by Reviewer SHhn). Efficient techniques like MECTA ($\text{\color{blue}Hong et al., 2023}$) or gradient accumulation reduce runtime memory up to 60% with maintained accuracy.

Table R1. Average GPU memory consumption (MB) in CIFAR10-C. MECTA and gradient accumulation (GA) are applied to BiTTA.

Q1(1): Relation to Prior Work. Prior work penalizes disagreement samples to lower their predictive confidence; we discard these instead, given their inherently unstable yet moderate accuracies ($\text{\color{blue}Figure 4(b)}$).

Q1(2): Advantages of using RL-based training. In this rebuttal, we experimented with directly minimizing prediction disagreement via backpropagation, which dropped accuracy by 1.08%p. This highlights our RL-based formulation, which naturally leverages prediction agreement as a reward. By doing so, the model adaptively reinforces learning from confident samples while effectively disregarding unstable samples with disagreement, achieving stable and effective adaptation.

Q2: Ablation Study without Binary Feedback. We did investigate this in “Synergistic effect of adaptation strategies” ($\text{\color{blue}Section 4, line 391}$). Adaptation without binary feedback achieved lower accuracy (82.64%) compared to BiTTA (87.20%), highlighting the importance of binary feedback.

Q3: Combination with Existing TTA Losses. Thanks for the suggestion. During the rebuttal, we experimented by replacing the ABA loss with entropy minimization. Our RL framework (87.20%) outperformed the modified one (85.98%) by balancing the gradients in a unified framework.

Q4: Evaluation on Larger Datasets. We did conduct experiments on ImageNet-C in $\text{\color{blue}Appendix B}$ ($\text{\color{blue}Table 6}$). BiTTA (36.59%) outperforms all baselines, including TENT (0.93%) and SimATTA (17.5%). Due to page limits, we initially placed them in $\text{\color{blue}Appendix C}$ but will move them to the main manuscript. Also, in this rebuttal, we ran additional baselines (e.g., DeYO), finding BiTTA consistently superior when introducing binary feedback ($\text{\color{red}Rebuttal for SHhn: Additional results on DeYO, OWTTT, and TAST}$).

Q5: MC-Dropout Configuration. We used 4 dropout iterations (N=4) for policy estimation. During the rebuttal, we conducted an abalative study, which showed the robustness of BiTTA to the dropout iterations when N>1 ($\text{\color{red}Table R2}$).

Table R2. Average accuracy (%) comparisons with varying MC-dropout inferences (N) in CIFAR10-C.

Q6: Modification to Baseline Algorithms. Baseline algorithms are modified to incorporate binary feedback by adding a cross-entropy loss on correct samples and complementary cross-entropy loss on incorrect samples. Details are in $\text{\color{blue}Appendix D}$.

Q7: Dropout Pre-training. Models were not dropout pre-trained; we injected dropout at test time. During the rebuttal, we conducted a calibration analysis that MC-dropout has a lower calibration error than the original softmax ($\text{\color{red}Rebuttal for SHhn: MC-dropout vs. softmax probabilities and uncertain sample selection}$). Also, during the rebuttal, we experimented on replacing the MC-dropout with the softmax, which resulted in 2.56%p drop, further demonstrating the importance of MC-dropout.

Q9: Comparison with Fully Labeled TTA. BiTTA outperforms fully labeled TTA (SimATTA) primarily because our approach effectively combines both binary feedback and unlabeled data, whereas SimATTA is highly dependent on source-like samples and prone to overfitting without unlabeled data.

We will incorporate this rebuttal in the final manuscript. We hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.