Responses to the Weaknesses

A1

• The objective of this paper is not to compete with the latest SOTA CTTA methods but to propose a plug-and-play approach for measuring the uncertainty of test results in existing methods, primarily assessing whether it can effectively address the coverage gap in conformal prediction. As a plug-and-play method, our approach has been constructed on top of some existing CTTA methods (Table 1,2,3), and the results demonstrate its effectiveness.
• Our method designs a conformal prediction (CP) technique, and we have compared it with SOTA CP methods including NexCP and QTC. The comparative results are shown in Table 4.
• To address the reviewer's concerns, we use ViDA to evaluate the uncertainty measurement outcomes, and the results are as follows: | Method | ERR | COV | INE | NLL | BS | ECE | |------------|-------|-------|------|------|------|-------| | α = 0.3 | | | | | | | | vida+TUI | 18.64 | 44.11 | 0.48 | 0.94 | 0.14 | 0.14 | | vida+TUI+CPAda | 18.61 | 69.49 | 2.72 | 0.95 | 0.15 | 0.16 | | α = 0.2 | | | | | | | | vida+TUI | 18.64 | 54.29 | 0.65 | 0.94 | 0.14 | 0.14 | | vida+TUI+CPAda | 18.63 | 76.23 | 2.86 | 0.95 | 0.15 | 0.17 | | α = 0.1 | | | | | | | | vida+TUI | 18.64 | 67.45 | 1 | 0.94 | 0.14 | 0.14 | | vida+TUI+CPAda | 18.53 | 85.29 | 3.27 | 0.95 | 0.15 | 0.17 |

A2

Our paper is plug-and-play, and builds on existing CTTA methods, aiming to assess the credibility of upcoming test results with just a small set of calibration data and to improve the adaptation effect using this assessment. The calibration set can be derived from the training set or constructed separately, and is entirely independent of the model itself.

• Fairness: We have considered that previous methods used all training data during pre-training. Therefore, all comparative methods (as shown in all experimental charts and tables) in our paper employ calibration sets constructed in the same manner for conformal prediction, ensuring there is no issue of unfairness.
• Privacy: The construction of the calibration set can be independent of the training set, serving merely as an additional reference to assist in measuring the uncertainty of test results. To illustrate this, we separate the source training data into training and calibration datasets during the pre-training phase. We train the pre-trained model only on the separated training data, without training on the calibration data, thus eliminating privacy concerns. The results are as follows:

A3

It is worth noting that the primary contribution of this paper is the proposal of a plug-and-play method for measuring the uncertainty of test results in existing CTTA methods. The main evaluation lies in effectively addressing the coverage gap in CP (as shown in the COV column) and achieving coverage close to the expected α.

Uncertainty estimation is only credible if the coverage gap (COV column) is addressed, which is the major contribution of this work. In the examples mentioned by the reviewer, including CoTTA and C-CoTTA on CIFAR100, as well as CoTTA, SATA, and C-CoTTA on CIFAR10, the CPAda method shows significant improvements in COV, making the uncertainty estimation more reliable.

A4

The computational efficiency of CPAda is equivalent to that of standard adaptation methods. It only adds sample-level weights during backpropagation (as shown in Eq. 11), and the complexity of calculating these weights is negligible.

However, the proposed TUI does increase computational time since it requires additional forward passes on the calibration set. In practice, the added time is minimal. As our method focuses on estimating the uncertainty of test results—something not addressed in previous CTTA works—we believe the added time is acceptable. The benefits of timely risk mitigation far outweigh the slight increase in computational cost.

Answers to the Questions

A1

Thanks for your suggestion.

A2

$s(y|\pi(x))$, referred to in the paper as the non-conformity score, specifically represents the score for each category $y$ as defined in Eq. 7. Eq. 9 indicates the process of collecting all categories with scores below the given threshold and including them in the prediction set.

A3

The purpose of the Concat operation is to simultaneously consider the outputs of both the source model and the current model for the same test sample $x$. The use of softmax aims to project the concatenated results into the probability space, ensuring that each value in the output comes from a valid probability distribution. Eq. 4 is the core design of the proposed method, as it accounts for model shift when constructing conformal prediction (CP). Combined with Eq. 5, the method jointly considers model shift and data shift, thereby making the CP results more reliable.

A4 & A5

These two issues are highly related and will be addressed together.

• As shown in Eq. 14, coverage (COV) and inefficiency (INE) are the two most critical metrics in conformal prediction. COV represents the actual coverage, indicating the proportion of true labels included in the prediction set. Ideally, COV should correspond to the acceptable error rate $\alpha$ provided by the user. For example, if the user specifies $\alpha = 0.2$, the closer COV is to 80%, the better. INE, on the other hand, directly reflects the size of the prediction set; an overly large prediction set implies high uncertainty. For single-label classification tasks, the optimal INE equals 1, meaning the prediction set contains only one label. If this label is the true label, COV equals 100%; otherwise, COV equals 0%.
• Lines 461-464 indicate that using TUI alone may sometimes fail to achieve a good balance between COV and INE. In some cases, COV is satisfactory, but INE is too high, while in others, INE is close to 1, but COV does not meet expectations. The subsequent paragraph shows that CPAda further enhances the results achieved by TUI, addressing these issues effectively.

Thanks for your clarification.

A1: If the plug-and-play approach is the main contribution, it is necessary to demonstrate its generalizability by integrating it with diverse CTTA methods. In addition, the results in the table for ViDA are relatively weak, making it difficult to conclude that TUI is effective.

A2: The unfairness I am referring to is not between different baselines but rather between the cases with and without TUI. Since TUI relies on an additional dataset for calibration, the observed improvement in error (though marginal) might result from the use of the calibration dataset. Additionally, an ablation study on the size of the calibration dataset would provide valuable insights.

A3:

Uncertainty estimation is only credible if the coverage gap (COV column) is addressed

I believe uncertainty estimation is meaningful when comparing two models with similar accuracy. COV, however, seems irrelevant to uncertainty estimation in this context.

Additionally, from my understanding, COV evaluates how many data points' true labels are included in the prediction set. A larger COV, which indicates a more accurate prediction set, should theoretically lead to a lower error rate. However, the results in Tables 1-3 do not support this assumption—despite a significant COV gap, the error and calibration improvements are only marginal. This raises the question: what is the value of a larger COV if it does not translate into meaningful improvements?

The use of softmax aims to project the concatenated results into the probability space, ensuring that each value in the output comes from a valid probability distribution.

What is the meaning of the probability space when the same label appears twice in the predictions? How is this scenario connected to model shift? For instance, consider a 3-way classifier where the logits from the source model are $[1,1,1]$, but the logits from the current model are $[1000,1000,1000]$ (a simple scaling of weights by 1000). While the prediction vector remains the same for both models, the computed $p(x)$ through Eqn. 4 would be drastically different. How does this discrepancy reflect the underlying model shift, and what implications does it have for the method?

Thanks for the further comments from the reviewer. The responses are as follows:

Response to Question 1:

We agree with the reviewer’s validation approach for the plug-and-play method. In fact, we have already validated it on three datasets using no fewer than five existing methods in the paper. As for the VIDA method mentioned, it shows good COV performance, which indicates that we can easily estimate which samples are uncertain within the VIDA method. For more details on estimating uncertainty, please refer to our response to the next question.

Response to Question 2:

The reviewer may have some misunderstandings in conformal prediction. In this paper, simply adding TUI does not change the original model's forward propagation or test results. In other words, there is no difference in the primary metric (ERR) when TUI is added versus when it is not. Adding TUI only provides an additional uncertainty estimation for the existing method, so there is no need to present a separate table showing the differences between the results with and without TUI. If it is necessary to include such a comparison, we suggest removing the columns for COV and INE results from the existing tables.

Furthermore, the size difference of the calibration set is already evaluated in Table 5 of the submitted paper. Experimental results indicate that a larger calibration set can improve COV, but there is no need to set a too-large calibration set.

Response to Question 3:

About the COV and uncertainty estimation:

First, COV does not directly measure uncertainty, but it serves as a guarantee for uncertainty measurement. In fact, the size of INE (the number of labels in the prediction set) directly quantifies uncertainty. COV and INE are interdependent that sufficient COV is required for INE to be meaningful. To illustrate, consider a doctor’s diagnosis:

• If the diagnosis is simply “pneumonia”, it does not account for uncertainty.
• If the doctor states that there is a 90% probability (the required COV) that the condition is one of the following—[pneumonia, emphysema, pulmonary nodules] (INE=3)—this expresses the uncertainty of the diagnosis. The size of this set determines the level of uncertainty. If the doctor lists 10 possible conditions (INE=10), it shows greater uncertainty in the diagnosis. A smaller set suggests a more confident diagnosis.
• If the doctor states that there is only a 30% probability (below the required COV) that the condition is one of the three possibilities (still INE=3), the diagnosis becomes unreliable.

In the context of domain shift, directly using Conformal Prediction would result in a lower COV, making the uncertainty estimation untrustworthy, say the coverage gap. This paper proposes to solve the problem.

The reviewer may refer to the following papers for more details on conformal prediction:

1. Shafer, Glenn, et al. "A tutorial on conformal prediction." Journal of Machine Learning Research 9.3 (2008).
2. Fontana, Matteo, et al. "Conformal prediction: a unified review of theory and new challenges." Bernoulli 29.1 (2023): 1-23.
3. Barber, Rina Foygel, et al. "Conformal prediction beyond exchangeability." The Annals of Statistics 51.2 (2023): 816-845.
4. Angelopoulos, Anastasios N., et al. "Conformal prediction: A gentle introduction." Foundations and Trends® in Machine Learning 16.4 (2023): 494-591.

About the joint logits:

In response to the reviewer’s question, when the same label appears twice in the predictions, it typically indicates that the model is confident in that label, possibly due to scaling or overemphasis in the logits. For example, if the logits from the source model are [1,1,1] and those from the current model are [1000,1000,1000], the predicted class remains the same, but the joint probabilities will differ significantly due to the scaling of logits. Despite the final prediction being identical, the underlying logit probability distributions are different, reflecting a shift in the model’s internal representation.

It’s important to note that in our method, softmax in Eq 4 is not used as a classifier, but as a normalization technique to convert the logits into a unified probability space. This is used to capture distribution differences between models. The logits from the two models represent two different stages of model knowledge: one reflecting the past model's understanding and the other reflecting the current model’s understanding. These logits are spatially shifted—their distributions are different. If this shift is not acknowledged, there is essentially no model shift. To capture this difference, we apply the softmax (as in Eq. 4), which compresses the logits into a common probability space. The divergence between these distributions is then measured using JS divergence, which effectively captures both data and model shifts. This approach sensitizes to differences in how the models process data, even when their final predictions are the same.

A1

TUI is a method for evaluating uncertainty, and directly incorporating TUI does not alter the original model's training outcomes. Guided by TUI, it can somewhat reduce the model's uncertainty and slightly improve its performance. The method we proposed is an insurance method, not an acceleration method. In the testing scenario, our method has two advantages: plug-and-play and model-agnostic. The proposed method is particularly suitable for scenarios with long-term testing, where the consequences of error accumulation are unacceptable (e.g., autonomous driving or medical applications) and where early knowledge of the model's uncertainty is crucial. It is not feasible to intervene manually only after a medical mishap or to adjust the model after a traffic accident. Being able to anticipate the uncertainty of results in advance is beneficial for subsequent operations.

A2

Sure, the calibration process requires additional computation for the calibration data, but it is small compared to the vanilla methods. Specifically, the time (s/batch) of the process can be calculated in the following table:

The method we proposed is an insurance method, not an acceleration method. In any testing scenario, our method has two advantages: plug-and-play and model-agnostic. Our method is more suitable for long-term testing scenarios where errors is unacceptable (e.g., autonomous driving, medical scenarios), and it is necessary to know the model's uncertainty in advance. We cannot intervene after a medical incident, nor can we correct the model after a traffic accident. If we can know the uncertainty of the results in advance, it will be helpful for subsequent operations.

A3

The proposed method consists of only two components: TUI and TUI-based CPAda. All the comparison methods in this paper (Tables 1, 2, and 3) incorporate TUI, and the results using CPAda are provided in the subsequent rows. Therefore, the tables in the manuscript already present the results for the two main components, and there is no need to include an additional ablation study table.

A4

The main objective of this paper is to build upon existing CTTA methods by leveraging a small calibration set to evaluate the reliability of upcoming test results and use this evaluation to improve the certainty of adaptation. The proposed method calibrates the model using the calibration set, which can be either extracted from the training set or constructed separately, making it entirely independent of the model.

In our manuscript, we have compared our method with the source replay approach used in RMT, as shown in Table 5 of the paper. The results demonstrate that our method outperforms directly using the source replay approach.

A3:

Thank you for your insightful comments regarding the datasets mentioned. We would like to address your concerns with the following clarifications and additional information:

• VisDA-C Dataset: You are correct in noting that VisDA-C is a classic dataset for Unsupervised Domain Adaptation (UDA) and does not inherently involve the multiple domain shifts present in CTTA scenarios, nor does it exhibit error accumulation. We will provide results in this scenario to further validate our method's effectiveness across different settings.
• Cityscapes-to-ACDC Dataset: This dataset represents a segmentation task, which is more aligned with the continuous distribution changes in real-world scenarios. Our proposed method is designed to estimate uncertainty at the image level, the application of Conformal Prediction (CP) for pixel-level uncertainty estimation is an unexplored area. We acknowledge the novelty and potential of extending our approach to pixel-level uncertainty estimation, which could be a valuable direction for future research.
• Many existing methods have employed the datasets used in this paper to validate the effectiveness of their methods at the image level, including CoTTA, RMT, ViDA.

A1:

Our paper is plug-and-play, and builds upon existing Continual Test-Time Adaptation (CTTA) methods, aiming to assess the credibility of upcoming test results with just a small set of calibration data and to improve the adaptation effect using this assessment. Our approach calibrates the model using a calibration set, which can be derived from the training set or constructed separately, and is entirely independent of the model itself.

• Fairness: We have considered that previous methods used all training data during pre-training. Therefore, all comparative methods (as shown in all experimental charts and tables) in our paper employ calibration sets constructed in the same manner for conformal prediction, ensuring there is no issue of unfairness.
• Privacy: The construction of the calibration set can be independent of the training set, serving merely as an additional reference to assist in measuring the uncertainty of test results during testing. To illustrate this, we separate the source training data into training and calibration datasets during the pre-training phase. We retrained the pre-trained model only on the separated training data, without training on the calibration data, thus eliminating privacy concerns. The results in this scenario are as follows:

• Other Methods Utilizing Source Data: We also note that other methods, such as RMT, leverage source data in their approach (Source replay). We have compared with source replay in our manuscript (Table 5), and found that our CP outperforms replay-based methods.

A2:

Sure, the calibration process requires additional computation for the calibration data, but which is small compared to the vanilla methods. Specifically, the time (s/batch) of the process can be calculated as follows:

The method we proposed is an insurance method, not an acceleration method. In any testing scenario, our method has two advantages: plug-and-play and model-agnostic. Our method is more suitable for long-term testing scenarios where errors is unacceptable (e.g., autonomous driving, medical scenarios), and it is necessary to know the model's uncertainty in advance. We cannot intervene after a medical incident, nor can we correct the model after a traffic accident. If we can know the uncertainty of the results in advance, it will be helpful for subsequent operations.

A4:

• The objective of this paper is not to compete with the latest SOTA CTTA methods but to propose a plug-and-play approach for measuring the uncertainty of test results in existing methods, primarily assessing whether it can effectively address the coverage gap in conformal prediction. As a plug-and-play method, our approach has been constructed on top of some existing CTTA methods (Table 1,2,3), and the results demonstrate its effectiveness.
• Our method designs a conformal prediction technique, and we have compared it with the most SOTA conformal prediction methods including NexCP and QTC. The comparative results are presented in Table 4.
• To address the reviewer's concerns, we select additional ViDA to evaluate the uncertainty measurement outcomes, and the results are as follows: | Method | ERR | COV | INE | NLL | BS | ECE | |------------|-------|-------|------|------|------|-------| | $\alpha = 0.3$ | | | | | | | | vida+TUI | 18.64 | 44.11 | 0.48 | 0.94 | 0.14 | 0.14 | | vida+TUI+CPAda | 18.61 | 69.49 | 2.72 | 0.95 | 0.15 | 0.16 | | $\alpha = 0.2$ | | | | | | | | vida+TUI | 18.64 | 54.29 | 0.65 | 0.94 | 0.14 | 0.14 | | vida+TUI+CPAda | 18.63 | 76.23 | 2.86 | 0.95 | 0.15 | 0.17 | | $\alpha = 0.1$ | | | | | | | | vida+TUI | 18.64 | 67.45 | 1 | 0.94 | 0.14 | 0.14 | | vida+TUI+CPAda | 18.53 | 85.29 | 3.27 | 0.95 | 0.15 | 0.17 |

A1:

Our paper is plug-and-play, and builds upon existing Continual Test-Time Adaptation (CTTA) methods, aiming to assess the credibility of upcoming test results with just a small set of calibration data and to improve the adaptation effect using this assessment. Our approach calibrates the model using a calibration set, which can be derived from the training set or constructed separately, and is entirely independent of the model itself.

• Fairness: We have considered that previous methods used all training data during pre-training. Therefore, all comparative methods (as shown in all experimental charts and tables) in our paper employ calibration sets constructed in the same manner for conformal prediction, ensuring there is no issue of unfairness.
• Privacy: The construction of the calibration set can be independent of the training set, serving merely as an additional reference to assist in measuring the uncertainty of test results during testing. To illustrate this, we separate the source training data into training and calibration datasets during the pre-training phase. We retrained the pre-trained model only on the separated training data, without training on the calibration data, thus eliminating privacy concerns. The results in this scenario are as follows:

• Other Methods Utilizing Source Data: We also note that other methods, such as RMT, leverage source data in their approach (Source replay). We have compared with source replay in our manuscript (Table 5), and found that our CP outperforms replay-based methods.

A2

Regarding the two hyperparameters mentioned by the reviewers, we would like to provide clarification:

• Hyperparameter α: This parameter is specified by the user in Conformal Prediction as the acceptable error rate. For instance, if α is set to 0.1, the resulting prediction set is expected to achieve a 90% coverage rate. In other words, there is a 90% probability that the true label will be included within the prediction set. A similar explanation can be found in Equation (2). In most of our experiments, we set α to three levels: 0.1, 0.2, and 0.3.
• Hyperparameter β: This parameter is a constant used to control the magnitude of the compensation term ρ. Since ρ is generally small, β is typically set to a relatively large value. The following table presents the experimental results for hyperparameter β on CIFAR10-C, C-CoTTA, and α=0.2. Note that a larger β may make the coverage to 100% and inefficiency to10, which means a full prediction set (all labels included). The CP is not working. When β is small, the coverage has a gap to the expectation (α=0.2, the expectation is 80%), and the uncertainty measure is not reliable.

A3 to the Question

The reviewer's understanding of Eq. (4) is correct. In Eq. (7), $\pi(x)$ refers to the current model. In this version, we will clarify this description to avoid confusion.