MaNo: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts

We thank the reviewer for the positive comments as well as the insightful suggestions. We will address the reviewer's concerns below. Please let us know if any issues remain, we would be happy to continue this discussion to address them.

1. The universality of the Low-Density Separation (LDS) assumption in practical applications.

This is a very interesting and important question, so our answer will be a bit detailed. We apologize for the length. In case if the reviewer wants to know more details on the LDS assumption, please refer to Chapter 1 of Olivier Chapelle et al.’s book [e].

When dealing with classification problems, we always implicitly rely on the smoothness assumption that two data points close to each other are from the same class with high probability. Originally, the LDS assumption was proposed as an attempt to apply this reasoning to unlabeled data in order to train semi-supervised models, and it could be considered as a natural hypothesis we put forward on the problem at hand. In our case, we pose this assumption with respect to the pre-trained model in order to be able to analyze its behavior on unlabeled data. Hence, the LDS assumption does not hold when the pre-trained model makes lots of mistakes on examples that are far away from the decision boundary, which, for example, can happen in the presence of unreasonable distribution shifts that exclude deploying the model on the target domain. In the paragraph Assumptions on the prediction bias of Section 3.1, we elaborated further on the assumptions that we should make with respect to the pre-trained model and amount of distribution shift for the safe application of accuracy estimation methods (because this reasoning applies, to some extent, to all our competitors).

2. The specific mathematical advantages of the $`SoftTrun`$ normalization strategy compared to traditional softmax. The softmax can be seen as a composition between the exponential and the scaling $\phi \colon u \mapsto u/\lVert u \rVert_1$. We study the nice mathematical properties of $\phi$ in Appendix D.6 and design $`SoftTrunc`$ such that it preserves those properties. As shown in Equation (6), $`SoftTrunc`$ recovers the softmax when the model is well-calibrated, and when it is not, it uses a Taylor approximation of the exponential (which is a $2^{nd}$ order polynomial) that reduces the influence of the prediction bias $\mathbf{\epsilon}$ as shown in Equation (4) and Figure 2(b).

3. To what extent do advantages of $`SoftTrun`$ rely on particular data distributions? The advantages of $`SoftTrun`$ do not rely on the data distributions, since we do not make any assumptions on the prediction bias $\mathbf{\epsilon}$ and the criterion $\Phi(D_{test})$. It autonomously distinguishes calibration scenarios on data from different distribution shifts.

4. Enhancing the performance in Table 1 and Table 2 by selecting other uncertainty measures.

We thank the reviewer for raising this question. We would distinguish the two following reasons why the performance enhancement in Table 1 and Table 2 seems minor.

• [The estimation performance under synthetic and subpopulation shifts are close to the optimal.] In our experimental protocol, we followed other papers in this domain, but it is true that the performance for synthetic and subpopulation shift datasets is almost perfect, so the improvement with respect to the state-of-the-art is not very large. Thus, the community may need more challenging datasets (e.g., natural shifts in Table 3) for these types of domain shifts.
• [The selection of metrics.] We utilize commonly-used metrics including $R^2$ and $\rho$ to measure the linear relationship between designed scores and true test accuracy, and, from our experience, these metrics may not provide a clear insight into the degree of the improvement. Nevertheless, thanks to the suggestion of Reviewer A5Zk, we will add to the revised manuscript the results evaluated by the mean absolute error. In Table G below, we can see that with this metric the superiority of MaNo on CIFAR-10 is more evident now.

Table G: Mean absolute error on CIFAR-10 with ResNet-18.

5. Does this imply that the research problem has already been solved? We believe that there exists still many open questions in this field. These are some of them:

• [Room for improvement under the natural shift] The numerical results in Table 3 show that current test accuracy estimation under the natural shift needs to be further improved, which is a more complex and practically meaningful question.
• [Generalization of LLM / other data modalities.] Unsupervised accuracy estimation papers have a focus today mostly on computer vision applications, but it would be important to explore also other data modalities and network architectures. In particular, a promising research direction would be to analyze the possibility of unsupervised performance estimation for large language models.
• [The underlying working mechanisim of generalization is still unclear.] One of the essential goals of this field is to understand when and why the model is able to generalize to unseen domains. However, the generalization capability of models is still a mysterious desideratum.

[e] Chapelle, O., Scholkopf, B., & Zien, A. (2006). Semi-supervised learning. 2006.

We thank the reviewer for their positive comments and very valuable suggestions to improve the quality of our paper. We address the reviewer's concerns below. Please let us know if any issues remain.

1. How to use $`MaNo`$ in practice? This work demonstrates the strong correlation between ground-truth OOD accuracy and the designed score, which can be particularly useful for model deployment applications. In the following, we provide two examples.

• [Find difficult (under-performed) test set] In cases such as retraining on under-performed datasets or annotating hard datasets, we only need to know the rank of datasets by accuracy. Therefore, we can calculate the proposed score for each dataset directly and fulfill the task based on this score's ranking.
• [Deployement risk estimation.] When deploying the model into production, it is important to estimate its safety. If the cost of getting test labels is prohibitive, our method can help estimate the model's accuracy on the product's test data. A practitioner can additionally look at the variability of the score on multiple test sets. When multiple datasets are not available, we can alternatively construct adequate synthetic datasets via various visual transformations.

We thank the reviewer for raising this question, we will update our paper accordingly.

2. Numerical results by using the absolute estimation error metric.

Thank you very much for introducing this metric to us, which will enhance the experiment quality significantly. In below Table E, we provide partial numerical results of our experiments by using the absolute estimation error. We will include all the results using this metric in our final version.

Table E: Mean absolute error on CIFAR-10 and Office-Home with ResNet-18.

3. Ablation study on $\eta$. We thank the reviewer for this suggestion. We provide the ablation study in Table F below, where we observed that $\eta=5$ can effectively distinguish the calibration scenarios.

Table F: Performance on CIFAR-10, Office-31, and PACS with ResNet18 for varying value of $\eta$ . The metric used in this table is the coefficient of determination $R^2$.

It is worth noting that we provide the theoretical analysis to demonstrate the general reason why we choose $\eta$=5 in Section D.3. It indicates that $\eta$=5 ensures that $\Phi(D_{test})$ deviates from its mean by the variance with probability smaller than 0.05.

4. Discussion about the two related works. We thank the reviewer for suggesting these two significant works. Both approaches utilize the disagreement between a pre-trained model and a target-adapted model to predict accuracy on each test set. In contrast, our MANO leverages the model’s logits for accuracy estimation without any training on target datasets. We will include these works and clarify the differences in our revision.

We thank the reviewer for their questions which helped us improve our work. We remain open to further discussion in case some issues remain unaddressed.

Dear reviewer A5ZK,

Thank you for your updated score and your helpful suggestions! We are glad that the clarifications were helpful. Thank you again!

Sincerely,

The Authors

We thank the reviewer for their precious comments which help us further improve the paper. We hope our answers below could precisely address the reviewer's concerns. Please let us know if any issues remain.

1. Hyperparameter tuning and OOD labels.

We thank the reviewer for this comment. In order to avoid any confusion, we would like to clarify several important aspects of problem setup:

• [No access to OOD labels.] We consider the unsupervised setting, so we do not have any validation set. This is eventually one of the main challenges of the framework as we are not able to tune hyperparameters in contrast to the supervised approaches.
• [Source-free.] We have constrained ourselves to a setup where only the pre-trained model is available without direct access to source data.

Given these constraints, we have come up with the proposed method, where we have made the following model choices.

• [Fixed hyperparameters across all distribution shifts.] We have set all the hyperparameters to fixed values ($p=4$ and $\eta=5$) across all the datasets. The empirical results show the superiority of MaNo without searching the optimal hyperparameter values. We have also performed a sensitivity study (please see Figure 7(a) for $p$ and response to Reviewer A5Zk for $\eta$).
• [Necessity of introducing $\eta$] The proposed normalization SoftTrun eventually reveals an additional level of complexity of the problem overlooked by previous methods. Table 4 shows that SoftTrun improves ConfScore and Nuclear, which justifies the importance of normalization, which, however, comes with the cost of introducing the hyperparameter $\eta$.

When there is access to the training data, a possible way to choose hyperparameters could be to generate multiple synthetic datasets from the training set via various visual transformations to determine hyperparameters' values.

We will revise the manuscript to better clarify our problem setup and the motivation behind the model choices we have made.

2. MaNo and SoftTrun in the case of the erroneous overconfidence and underconfidence.

Thanks for this insightful comment and we reply to it in two parts.

• [high errors and high confidence] This great remark concerns actual assumptions we must make in our setup (Section 3.1). One may notice that overconfidence and high errors imply that predicted probabilities are misaligned with the true ones in terms of class ranking, which is a big issue that may go beyond calibration. Considering unsupervised cases and no access to the training phase, logits are the only source of information we have, so we find it reasonable to assume that the model mistakes are mostly in low-confidence regions.

• [underconfidence] We totally agree. That is why we do not impose any assumption on the prediction bias $\mathbf{\epsilon}$, so it can have positive/negative entries covering overconfidence and underconfidence. In practice, it has been shown that deep neural networks with softmax are overconfident [a], so the situation of underconfidence is unlikely to happen. As we did not observe this phenomenon in our experiments, it would be a good direction for future work to find benchmarks where underconfidence issues are present. We will include the above discussion in the revision.

3. Relation to Agreement-on-the-Line (ALine-D). We thank the reviewer for suggesting this interesting work. We would like to clarify the differences between ALine-D and MaNo.

• [Different settings.] ALine-D operates under a model-centric setting, aiming to accurately estimate OOD accuracy across many different models. In contrast, MaNo is designed for a data-centric setting, focusing on estimating a single model’s OOD accuracy on various datasets.
• [Different assumptions.] ALine-D assumes that (1) a set of diverse pre-trained models is available during evaluation; (2) agreement-on-the-line phenomenon holds consistently. In practice, accessing a diverse set of models may be infeasible, and assumption (2) does not always hold [b, c]. On the other hand, MaNo and other data-centric methods do not face these limitations.

Based on the above, ALine-D cannot be directly compared with MaNo, but we provide two indirect ways to do that.

• In our experimental results, MaNo outperforms AgreeScore, which is based on a similar idea to ALine-D.
• In Table D, we compare the results provided in [d], Section 5.2 with ours on CIFAR-10C to further illustrate the efficiency of MaNo even without the two assumptions.

Table D: MaNo v.s. Agreement-on-the-Line (ALine-D) on CIFAR-10C with ResNet-18.

4. What makes MaNo superior to AC? Thanks for raising this question. First, we would like to clarify a possible misunderstanding between $L_\infty$ norm and AC.

• [ AC is not a specific case of $p=\infty$] When we consider $L_\infty$-norm, it means that we compute the maximum value over the whole prediction matrix, while AC extracts the maximum value of each row in that matrix.

Then, to answer the question:

• [AC ignores the distances to subconfident decision boundaries.] Our analysis in Section 3.1 shows that distance to each class boundary is important, so by considering the confidence of the most probable class only, AC may bring a loss of information.
• [AC considers each sample separatively.] MaNo puts greater emphasis on high-margin terms globally when $p$ increases, while AC considers the high-margin values of each sample with the same weight.

[a] Leveraging ensemble diversity for robust self-training in the presence of sample selection bias.

[b] ID and OOD performance are sometimes inversely correlated on real-world datasets.

[c] Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization.

[d] Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift.

Dear Reviewer 5Gdr,

Thank you for the updated score and the valuable review. We are glad that our clarifications were helpful. As suggested, we will include the discussion about potential limitation in the paper with the corresponding references (entropy minimization [1-3]). We believe the review helped us improve the paper and thank you again for your constructive suggestions.

Sincerely,

The Authors

We thank the reviewer for the positive support and constructive comments on this work! Please find our responses below.

1.1. Relation to entropy-based methods such as ATC.

We thank the reviewer for this comment. The proposed MaNo and the ATC indeed share similarities as they both belong to the class of logit-based accuracy estimation methods. However, there are several important aspects that distinguish the two methods, not only in terms of methodology but also effectiveness.

• [ATC calculates entropy that MaNo never considers.] ATC utilizes the logits to calculate the entropy and measures how many test samples have a confidence larger than a threshold, which is a hyperparameter needed to be chosen. They proposed to search it from the labeled source validation set, which might be not a good idea in cases when softmax tends to be overconfident and the prediction bias is noticeable as we discussed in Section 4.1. In contrast, MaNo directly calculates the $L_p$ norm of the normalized logit matrix, thereby avoiding the introduction of a confidence threshold and leading to a simpler and more efficient approach.
• [Stability.] Just being based purely on the experimental results, we can see that the entropy-based score function does not seem to be stable in performance and is worse compared to the proposed MaNo (regardless if it is softmax or SoftTrun normalization scheme).
• [New normalization scheme] To the best of our knowledge, the question of calibration and logit normalization has never been touched before in the unsupervised accuracy estimation domain, so the proposed SoftTrun normalization scheme makes our contribution conceptually different from other approaches including ATC.

1.2. Simplicity of the approach.
We would like to open a debate with respect to this point as we believe that the simplicity of our approach is one of its main strengths given its superiority compared to 11 competitors over 3 different shifts and 12 datasets with a very noticeable difference in natural shift setting. In addition, we provide the following arguments.

• Our approach is source-free and training-free (in contrast to ATC and ProjNorm, resp.), which makes it relevant for model deployment applications.
• As the reviewer noticed, our method has emerged from the theoretical perspective of the low-density separation assumption, which makes it intuitive and easy to analyze.
• The versatility of the approach allows us to painlessly test our method for different neural network architectures. Therefore, in order to strengthen this point, we have conducted additional experiments using two other commonly used vision models: ViT and Swin. In addition, we added a new evaluation metric (mean absolute error) based on the suggestion of Reviewer A5Zk. We will include all the experimental results in the revision, meanwhile displaying some of the results in the two tables below. Our approach has superior performance on these two architectures as well, which validates our versatility claim. Our experimental results also include an ablation baseline, which is Softmax $MaNo$ that uses softmax normalization function instead of the proposed SoftTrun.

Table A: $R^2$ on CIFAR-10C and Office-Home with ViT.

Table B: Mean absolute error on CIFAR-10.1, ImageNet-S and ImageNet-A with Swin

2.1. Clarifications regarding SoftTrun. We apologize for the possible misunderstanding regarding Figure2(b) and the design of SoftTrun. Please find our comments below.

• [Typo in Figure 2(b)] We have made a typo, and the bar named "SoftTrun (first 2-order)" should be called as "Taylor approximation 2nd order" instead. This bar refers to the case when we apply only Taylor normalization function (as if we would always choose the 1st scenario in Eq.(6)), while "MaNo w/ Softmax" refers to applying softmax only (as if we would always choose the 2nd scenario in Eq.(6)).
• [Purpose of Figure 2] The goal of this figure is to motivate the proposed SoftTrun defined by Eq.(6), where we automatically choose one of the normalization functions depending on a calibration scenario. For well-calibrated scenarios (Office-Home), we should use the softmax normalization for information completeness, while for poorly-calibrated ones (PACS), Taylor normalization is preferred in order to be more robust to prediction errors.
• [SoftTrun] Thus, the proposed SoftTrun selects one of these two normalization schemes depending on the uncertainty value, and the final experimental results are displayed in Table 1, 2, 3.

We will update the labeling of Figure 2(b) and the writing of Section 4.2 to clear out any potential misunderstanding.

2.2. Results of softmax-based MaNo in Tables 1 and 2. Thanks for suggesting this idea. We did not display this baseline for the synthetic and subpopulation shifts as we found that the prediction bias is quite low on these problems which leads to $\Phi(D_{test})$ much greater than the proposed fixed $\eta=5$, so SoftTrun always chooses softmax normalization for these problems. The key difference between SoftTrun and softmax-always normalization schemes comes when natural shift is considered. In the table below, please find the experimental results of comparing the two schemes on 4 natural shift benchmarks, confirming SoftTrun superiority.

Table C: Softmax MaNo v.s. the proposed MaNo measuring by $R^2$ under natural shifts with ResNet-18.

As the discussion period is ending, we thank the reviewers for taking the time to review our paper and discuss it with us. We believe their valuable suggestions strongly benefitted our paper.

We are glad that our clarifications and the additional experiments requested addressed the reviewers' concerns and are grateful to them for recognizing the core strengths of our work:

• Novel and innovative
• Theoretically well-motivated
• SOTA efficient approach
• Extensive comparisons

We thank them again for their insightful feedback.

Sincerely,

The Authors