We thank the reviewer for your positive feedback and valuable suggestions. We are pleased that the reviewer found that “online drift detection and online model adaptation are naturally proposed and make sense.” We appreciate the positive feedback regarding the clarity of our writing. Thank you!

The experiments are relatively weak as the authors only conduct experiments on the ImageNet-C dataset, ignoring the CIFAR10-C and CIFAR100-C datasets.

Kindly refer to the global response to all reviewers.

The proposed method estimated the pseudo-entropy at testing time. However, I wonder whether this can be done when the label distribution [1, 2] also shifts because the shifted label distribution also affects the sequential entropy values.

Thank you for raising this important issue. Extending our proposal to the label shift setting remains an open question for us. The idea in [1] may serve as a promising starting point. Specifically, we found the idea of prediction-balanced reservoir sampling appealing, as it can be used to approximately simulate an i.i.d. data stream from non-i.i.d. stream in a class-balanced manner. This can potentially reduce the sensitivity of the martingale process to label shifts. In turn, we anticipate that under label shift and in the absence of covariate shift, our adaptation would be minimal; and in the presence of the latter, the adaptation would be more substantial, as desired.

Another possible approach would be to work with a weighted source CDF rather than the vanilla source CDF, where the weights should correspond to the likelihood ratio $P_t(Y)/P_s(Y)$. The use of such a weighted CDF was suggested in the conformal prediction literature to adjust for label shift between the source holdout data and test points [3], making the test loss to “look exchangeable” with the source losses. Certainly, the challenge in this context is to approximate the likelihood ratio $P_t(Y)/P_s(Y)$ in a reasonable manner. This challenge becomes more pronounced when faced with both a covariate and label shift. It may be the case that reference [2] pointed out by the reviewer could be a valuable starting point for us to explore this path.

We will include a discussion on this matter in the revised version of the paper.

[1] T. Gong, et al. "NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation," NeurIPS, 2022.

[2] Z. Zhou, et al. "ODS: Test-Time Adaptation in the Presence of Open-World Data Shift," ICML, 2023

[3] A. Podkopaev and A. Ramdas, “Distribution-free uncertainty quantification for classification under label shift,” Uncertainty in artificial intelligence, 2021.

The “Protected” in the title of this paper and name of this method should be carefully considered.

We are now seriously considering removing the word "protected" from the title. We initially used this word for three main reasons. First, our monitoring tool rigorously alerts for distribution shifts, and the ability to raise such a warning is crucial for communicating with users that the model is encountering new environments. Second, our approach has been shown to have no harmful effect when the test data follows the same distribution as the source domain, which is a significant concern in test-time adaptation. Third, we wanted to acknowledge the protected regression method [1] that inspired our proposal.

[1] Vladimir Vovk, “Protected probabilistic regression,” technical report, 2021.

The experiments show that the proposed method can achieve a lower ECE value.

Recall that the ECE is presented when applying self-training on in-distribution test data (Figure 3, left panel). This experiment demonstrates that in this in-distribution scenario, our method maintains the ECE of the source model—we are not improving the ECE of the source model. Importantly, this stands in contrast to entropy minimization methods that, by design, drive the model to make over-confident predictions, as reflected by the increased ECE value in Figure 3. This emphasizes that when the test samples do not shift, we preserve the calibration property of the model and avoid making over-confident predictions, which is desired in practice. We will clarify the exposition surrounding Figure 3 in the text accordingly.

We thank the reviewer for the positive feedback and the constructive review. We are pleased that the reviewer appreciates the novelty and clarity of our writing. We very much value the reviewer’s comment that “the experiments on two TTA settings (single domain and continual TTA) confirm its effectiveness.” Thank you!

Why is betting martingale a suitable approach for TTA?

A betting martingale is a powerful tool to monitor distribution shifts in an online manner. We designed a monitoring tool that tests whether the distribution of the classifier’s entropy values drifts at test time. This naturally leads us to ask: if a martingale can detect that the model “behaves” differently at test time, why not use this knowledge to correct the model’s behavior?

Our work shows how the evidence for distribution drift encapsulated in the betting martingale can be used to adapt the model at test time. The idea is to use the martingale to transform the test entropies to “look like” the source entropies—essentially matching the distribution of the source and the self-trained model entropy distributions. This, in turn, builds invariance to distribution drifts, which is a key principle to improve model robustness.

In the interest of space, for a more technical reply, we kindly refer the reviewer to the response we provided to Reviewer Utu5 (“Clarification on the entropy matching procedure”).

The use of the term "domain adaptation".

Your point is well taken! If given the opportunity, we will fix this issue in the revised paper and use “test-time adaptation” instead. We will also remove the word “Adaptation” from the title of the paper. Thank you for your constructive comment.

Can you elaborate on the intended interpretation of Figure 1?

Each curve in Figure 1 presents a different risk (black for entropy minimization, red for entropy matching) as a function of the weight $w$ of the classifier $f_w$. Since the optimization procedure aims to minimize a given risk function by changing the value of the weight $w$, the curves help to understand what is the optimal value that should be obtained.

By minimizing the entropy risk, the optimization ends with a self-trained classifier $f_w$ that achieves the smallest value of the black curve. This results in a trivial classifier that always predicts $+1$ (or always $-1$), regardless of the value of $X$.

By minimizing the entropy matching risk, the optimization ends with $f_w$ that achieves the smallest value of the red curve. This results in a classifier that separates the two classes as much as possible. Indeed, our online method obtained a self-trained classifier whose accuracy (nearly) matches the accuracy of the Bayes optimal classifier both under an in-distribution setting (top panel) and under an out-of-distribution setting (bottom panel).

The model requires calculating the source CDF, which requires extra time.

Recall that this is a CDF of 1-dimensional variables (the source entropies), which is computed only once and offline. The complexity of computing this CDF is dominated by the evaluation of the pre-trained model on relatively small holdout unlabeled samples from the source domain. At test time, we only need to compute the value of the pre-computed CDF at a single point (the test point’s entropy value), which amounts to accessing a small, pre-computed 1-dimensional array.

What if computational resources are limited?

Following the global response to all reviewers, the new experiments show that the runtime of our method is comparable to TENT and EATA and even lower than that of SAR. Moreover, our monitoring tool can be used to decide whether the model should be updated or not at test time (that can further reduce runtime), as the martingale process detects distribution shifts. Notably, our monitoring tool can be applied in a “black-box” manner as it only requires access to the output of the softmax layer.

Additionally, it may not be possible to perform such calculations if the source model is only made available.

Indeed, we require access to a pre-trained source model and a pre-computed source CDF. Importantly, given the two, we do not require any additional access to samples from the source data at test time. In that respect, our work does not differ significantly from EATA, which also assumes access to holdout source examples.

Limited evaluation; there is no sensitivity study.

Kindly refer to the global response to all reviewers.

This paper does not compare with state-of-the-art TTA baselines such as ROID

Our goal is to highlight why we believe it is important to transition from entropy minimization to online entropy matching. Therefore, we deliberately compared our approach to strong baseline methods that are based on entropy minimization.

The ROID method builds on a weighted version of the soft likelihood ratio loss as a self-supervised loss. This loss departs from the line of the baseline, entropy-minimization methods we focus on, but it opens an interesting future direction. Broadly speaking, our paper offers an online mechanism for matching source and target distributions of any given self-supervised loss. In the context of ROID, it will be illuminating to explore how our matching paradigm would perform in combination with the weighted soft likelihood ratio loss instead of the entropy loss. We will include this idea for future work in the text!

Moreover, ROID highlights that self-training can fail to improve or even deteriorate performance. This motivated the authors of ROID to include several, complex components within the test-time training scheme. Naturally, a SOTA method would include various components to enhance performance, and this set of ideas can be also valuable for our method. However, given that we introduce a new concept to the ever-growing test-time adaptation literature, we believe such explorations go beyond the scope of the current paper. These will divert attention away from our central message.

We thank the reviewer for your valuable feedback and suggestions. We are glad that the reviewer found our approach to be novel and an interesting research direction. It is gratifying to see that the reviewer thinks that “the entropy matching could potentially be a good alternative to the dominant paradigm of entropy minimization.” Additionally, we appreciate the positive feedback regarding the clarity of our writing and that the reviewer found the experiments on ImageNet-C show encouraging results. Thank you!

The experimental results are quite limited

Kindly refer to the global response to all reviewers.

An illustration visualizing the entropy-matching procedure

Thank you for this great suggestion, we will include such an illustration in the revised paper. We also discuss the role of each component below.

Clarification on the entropy matching procedure. What is the role and interpretation of $Q$? How can we match the entropy distributions?

Thank you for raising this question, as it touches on one of the more nuanced aspects of our work. Indeed, as the reviewer mentioned, one can interpret $Q$ as the distribution approximating the unknown CDF of the target entropy $Z_t$. We understand that this might be confusing as $Q$ is a function of $u$. However, recall that $u$ is a function of $Z_t$, as $u := F_s(Z_t)$. Observe also that $Z_t = F_s^{-1}(u)$.

To better clarify the role of $Q$, consider a case where the betting is optimal in the sense of Proposition 2. The right-hand side of Eq. (8) gives the explicit form of the ideal $Q$ being $\tilde{u} = Q(u) = F_t(F_s^{-1}(u))$. In turn, $Q(u) = F_t(Z_t)$. In practice, the test entropy CDF $F_t$ is unknown, yet the above relation highlights why the $Q$ we formulate via the betting function can be intuitively viewed as the distribution approximating the unknown target entropies CDF. This is due to the fact that any valid betting martingale is a likelihood ratio process, aligning with Eq. (8).

As for the matching property, observe that in the ideal case, the pseudo-entropy value $\tilde{Z_t} = F_s^{-1}( \tilde{u}) = F_s^{-1}(Q(u)) = F_s^{-1}(F_t(Z_t))$. This argument reveals the tight relation between our adaptation scheme and optimal transport: in the ideal case, the pseudo-entropy $\tilde{Z_t}$ is obtained by applying the optimal transport map from the target entropy distribution to the source entropy distribution. Our experiment in Figure 6 in the appendix demonstrates that we indeed achieve distribution matching via the online-estimated $Q(u)$ in practice, although we do not have access to $F_t$ that varies over time.

We will include this discussion and clarification around Proposition 2 in the text.

How does entropy matching perform on long-range adaptation? My understanding from the experiments is that only 1000 samples per corruption are used and not even the entire ImageNet-C dataset.

The experiments we conducted also involved a long-range adaptation on a test set of size $\approx 30,000$ for ImageNet; $\approx 15,000$ for CIFAR; and for OfficeHome we used the entire test data. Focusing on ImageNet-C, we highlight that in the continual setup, we also used $2000$ samples per corruption (Figure 2 bottom right panel), resulting in a test set of size $2000 \cdot 15=30,000$ samples in total. In the single corruption experiments, we use $37,500$ test samples for each corruption. Notably, we had to reserve a subset of the test set to implement both EATA and our method as the two need access to unlabeled holdout data from the source domain.

The limitations are rather vague, and it seems currently unclear which limitations the method encounters

The key limitations are the following:

1. To implement our method, we assume access to the source CDF, evaluated on holdout unlabeled samples from the source domain.
2. The choice of hyper-parameters, in particular the learning rate, can be challenging as it depends on the data and model used. This is akin to related TTA methods.
3. The lack of theory that reveals when entropy matching is guaranteed to improve performance. This issue is one of our future research directions.

We will update accordingly the discussion on the limitations in the text. We thank the reviewer for this point.