Uncertainty Quantification Using a Codebook of Encoders

Thank you for the detailed and constructive comments! We are also glad that you think our contributions can be helpful to the community. Please find our comments below.

1. Specially, for OOD detection, the setting used in the paper is previously shown to be an insufficient indicator for generalization.

We extended our experiments in Appendix D.1. We have chosen regression tasks for several reasons. First, we want to highlight that our model can handle both regression and classification tasks. Second, regression problems are often overlooked by the community. Therefore, demonstrating our model on these tasks could broaden the impact of our paper perhaps more than testing it on expensive/ larger architectures for image classification tasks.

2. The reported accuracy of the models appear way below the standards in the literature.

Our model does not introduce an accuracy drop. The score 93% in Table 3 pertains to a binary classification task (where the label indicates whether the model will correctly classify the image or not) and not the image classification task. For the accuracy of the model for the image classification, please refer to Table 2.

3. I did not see any reference to the misclassification detection literature, but I might have missed it. See for example:

We added three references:

—In the introduction: The recent work of Zhu et al. (2023)

—In section 5.3: we evaluate the performance of various uncertainty scores in predicting the model’s incorrectness (Corbière et al., 2019; Zhu et al., 2022)

4. Comparisons are made with methods that are not very recent (as summarized in tables 2 and 3). The most recent reference that authors have compared with is from 2021. Could authors please explain their choice of methods to compare with?

We compared our model with single-forward pass uncertainty quantification methods that do not use image augmentations or auxiliary OOD datasets. We updated Table 1 to include two models published less than a year ago.

5. In Figure 1, the location of points in subfigures (a) and (b) are exactly the same. Is that intentional?

This is a great observation! We preserved the location of the datapoints for illustration purposes, i.e., to quickly compare x vs p( . |x). In general, we expect that the datapoints in distribution in P will be reshuffled (while the relative positions will also depend on the divergence used).

6. Experiments are thin only on small datasets.

We would like to note that although CIFAR-10 is a smaller-scale problem, it is widely regarded as a challenging OOD task. We refer you to [1], [2]. We briefly quote here:** “CIFAR Benchmark is NOT Easier than ImageNet Benchmark “** [1]. Moreover, very impactful papers such as SNGP and DUQ follow an experimental setup similar to ours while they were scaled to computer vision tasks/large-scale image classification tasks later on. However, we do agree that scaling the experiments will increase the impact of the paper.

7. For example, how many encoders and centroids would be needed to apply this method to models trained on Imagenet?

A safe estimation would be to choose a codebook size equal to the number of classes. However, smaller codebooks can still yield competitive OOD performance (see Table 7, cases k=5 vs k=10). In the revised manuscript, right above Table 10, we further discuss the codebook size selection.

8. Overall, I think introduction of the paper could be more on point, emphasizing the broad view of the paper early on.

This is another great suggestion. We have revised the introduction:

“We introduce a rate-distortion theoretic view of uncertainty quantification in deep neural networks.., Under this lens, ”

References

[1] Yang J, Wang P, Zou D, Zhou Z, Ding K, Peng W, Wang H, Chen G, Li B, Sun Y, Du X. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems. 2022 Dec 6;35:32598-611.

[2] Van Amersfoort J, Smith L, Teh YW, Gal Y. Uncertainty estimation using a single deep deterministic neural network. InInternational conference on machine learning 2020 Nov 21 (pp. 9690-9700). PMLR.

I thank the authors for their detailed response.

While I appreciate the improvements in the paper and the clarifications provided in the rebuttal, I still find the experiments on the CIFAR-10 dataset inadequate and not a good indicator for the usefulness of the proposed method. The additional experiments on regression tasks do not seem to go well with the CIFAR-10 experiments and adds to the confusion in my view. Hence, I keep my score.

It may be worthwhile to note that evaluating OOD detection methods merely on CIFAR-10 is shown to be inadequate. See this paper which was selected as the notable top 5% of papers at ICLR2023:

– Jaeger, P.F., Lüth, C.T., Klein, L. and Bungert, T.J., 2022, A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification. In The Eleventh International Conference on Learning Representations.

After further inspection of the details in [1], we understand that cifar10 vs cifar100 is also a failure case of many baselines. We quote here:

"DG-Res, Devries, and ConfidNet fail to generalize beyond the scenarios they have been proposed on, i.e. to more complex data sets (like iWildCam or BREEDS) and covariate distribution shifts (corruptions and sub-class shifts)."  CIFAR-100 is characterized as a sub-class shift: "Further sub-class shifts are considered in the form of super-classes of CIFAR-100" also noted a s-ncs (semantic new class shift) in Table 1 of [1].

In Table 1, this failure case is depicted as differences in the colormap of columns c100 and svhn under the cifar-10 dataset in the rows corresponding to the methods mentioned above. UA-IB passes this failure case: it consistently outperforms all methods at both tasks.

[1] Jaeger, P.F., Lüth, C.T., Klein, L. and Bungert, T.J., 2022, A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification. In The Eleventh International Conference on Learning Representations.

Thank you for reviewing our work and for your comments and suggestions! Please find our replies below.

1. it is unclear how lightweight the approach effectively is (compared to other approaches)

We did not observe computation overhead for training our model compared to a single, deterministic network that is not uncertainty-aware. The parameter complexity of our model is provided in Table 2.

2. the determination of the codebook size can pose serious challenges to the proposed approach

We observed that a larger-than-needed codebook will not harm the performance. We provided some guidelines on the selection of the codebook size right above Table 10 in the Appendix.

3. only one model is employed for the evaluation

We extended our experiments to also test our model on regression tasks in Appendix D.1. This set of experiments broadens the impact of our paper, especially since regression problems are often overlooked by current research. In contrast to many related works, our method does not apply only to images and/ or classification tasks.

4. the evaluation is performed on small-scale datasets

We would like to highlight that OOD tasks on small-scale datasets are not necessarily considered easier tasks. We refer you to [1], [2]. We briefly quote here: “CIFAR Benchmark is NOT Easier than ImageNet Benchmark “ [1]. Moreover, competitive methods such as SNGP [3] or DUQ [2] follow an experimental setup identical to ours. They were scaled/ adapted later on to the applications of interest.

5. how is UA-IB compared/applies to recent works like [1,2,3]?

Our work belongs to the class of single-network, single-forward pass models for uncertainty quantification. The work in [4] is not a single-forward pass method (it relies on MC-Droupout). The works [5, 6] are single-forward pass, multiple-network (merged into one) methods that still suffer from high memory footprint. More importantly, ensemble methods are not distance-aware, i.e., they do not explicitly capture the distance of an unseen input from the training data points.

For a quantitative comparison of our model with these works, please see our comments below:

Our model significantly outperforms packed-ensembles [6]. Please see Table 1, WR Network in [6], 98.1 (AUPR) vs ours 99.4 and 96.5 (AUROC) vs ours 98.6

Our model outperforms masked-ensembles [4]. Please see Table 1 in [4], 0.96 vs ours 99.4 (AUPR) and, 94 (AUROC) vs ours 98.6

References

[1] Yang J, Wang P, Zou D, Zhou Z, Ding K, Peng W, Wang H, Chen G, Li B, Sun Y, Du X. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems. 2022 Dec 6;35:32598-611.

[2] Van Amersfoort J, Smith L, Teh YW, Gal Y. Uncertainty estimation using a single deep deterministic neural network. InInternational conference on machine learning 2020 Nov 21 (pp. 9690-9700). PMLR.

[3] Liu J, Lin Z, Padhy S, Tran D, Bedrax Weiss T, Lakshminarayanan B. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Information Processing Systems. 2020;33:7498-512.

[4] Durasov, Nikita, et al. "Masksembles for uncertainty estimation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

[5] Havasi, Marton, et al. "Training independent subnetworks for robust prediction." ICLR 2021

[6] Laurent, Olivier, et al. "Packed-Ensembles for Efficient Uncertainty Estimation." ICLR 2023

Thank you for your reply! Please find our responses below:

1. As you claim that the method is "lightweight"

We used the term lightweight to differentiate our model from broad and expensive model categories such as ensembles or variational Bayesian methods and classify our model as a Deterministic Uncertainty Method (DUM), i.e., a single-forward pass and single model method. Compared to ensembles, our method is more efficient in terms of both compute time and parameters. Compared to variational methods, our method is more efficient in terms of compute time since these methods require multiple forward passes to provide uncertainty estimates. Under the DUM category, all models are considered lightweight.

2. It is clear that increasing k the performance improves too.

Please note that for cifar-10, doubling k does not provide further gains. On Imagenet, we expect a sufficient codebook size significantly smaller than the number of classes since it exhibits a coarse class hierarchy. You are right that the codebook size will need to be optimized especially when training on small memory gpus (< 32GB) since it can create a trade-off with a larger batch size.

3. I can not find the details of the different architecture

We have now added the details in section D.2.3. We note here that this is a non-image task therefore resnet architectures are not applicable.

4. here are in the literature works employing ImageNet as a medium-scale dataset to experiment

We remark here that this is very subjective since it highly depends on the availability of computing resources. The authors in [6] mention: "except for ImageNet, for which we use 2 to 8 Nvidia A100-80GB". According to https://lambdalabs.com/service/gpu-cloud assuming training takes roughly a day, training a single model could cost as much as ~300$.

5. I invite the authors to include an extended version of this comparison in the main paper

We have now updated Table 1 to include Masked Ensembles in the comparisons. We will repeat the experiments ourselves to properly fill-in the stds and performance on cifar-100 tasks in the next version of the paper.

Thanks for all the feedback! Please find our comments and replies below:

1. Classifying Guassian Process Models like SNGP as deterministic is not correct.

SNGP is a probabilistic model with deterministic uncertainty estimates. This is because estimating the likelihood of the model is a deterministic function of the data. We borrowed terminology used in [1] (for example, in Table 1 in [1], SNGP is listed as a DUM). We agree that this can be confusing and that maybe a term such as single-forward pass uncertainty method would be more appropriate.

2. The fact DUMs that apply regularisation and obtain SOTA OOD detection need to harm calibration is false. Works like RegMixup [1], AugMix [2] and PixMix [3] clearly show this is not the case.

In the main text, DUMs, as in [1], refer only to Gaussian Process models and clustering methods that do harm calibration [1]. For completeness and clarity, we now discuss dataset augmentation methods in Section 6.

3. Especially compared to the deterministic baselines mentioned above, the method induces an accuracy drop.

Our method does not induce an accuracy drop. We refer you to Table 2: It performs on par with other DUMs (95.9% accuracy) that do not use dataset augmentations.

4. There are several deterministic techniques that induces better uncertainty estimation, that the authors neglect. Besides the already mentioned RegMixup [1], AugMix [2] and PixMix [3]. These techniques add no parameters to the model. Furthermore the authors should consider other fundamental baselines from [0]

We have revised our results (Table 1) to add two recent baselines. For example, compared to the suggested RegMixup baseline, our method achieves stronger OOD performance: 96% vs 98% (AUROC) for SVHN and 89 vs 92% (AUROC) for CIFAR-100.

5. The paper trains on a single dataset, CIFAR-10, which is extremely simple.

Although **CIFAR-10 ** is a smaller-scale problem, it is widely regarded as a challenging OOD task not easier than ImageNet. [2], [3]. In Appendix D.1 we added an additional experiment to test our model on two additional real datasets for regression tasks. Several baselines in current research treat only images and/or classification tasks. Our method provides a unified uncertainty score for both tasks. UA-IB performs better or on par with an ensemble of 4 networks on all datasets and all tasks.

6. DUQ, which shares a few conceptual similarities to the proposed work, becomes unstable when the number of classes increases

We do not expect that UA-IB will not scale due to the similarities to DUQ. Some key differences:

a. DUQ is hardwired to centroids that must be equal to the number of classes. Our model, in contrast, supports arbitrary codebook sizes that do not necessarily match the number of classes (please see Table 10 in Appendix).

b. Our model can handle regression tasks as well (in contrast to DUQ).

c. In our method, we use a different loss function where the assignment of datapoints to centroids becomes in an unsupervised manner.

7. (SNGP, Deep Ensemble) do scale

Fundamental research and empirical research/ scaling experiments are both important. The focus of this work lies in the first category and is to provide a fresh analysis of UQ as a lossy compression of the training dataset. Please also note that other baseline DUMS (that do not use augmentations) do not provide performance on Imagenet OOD tasks. Finally, the impactful SNGP was scaled on ImageNet recently, after its publication as an extension of prior work.

References:

[1] Postels J, Segu M, Sun T, Sieber L, Van Gool L, Yu F, Tombari F. On the practicality of deterministic epistemic uncertainty. arXiv preprint arXiv:2107.00649. 2021 Jul 1.

[2] Yang J, Wang P, Zou D, Zhou Z, Ding K, Peng W, Wang H, Chen G, Li B, Sun Y, Du X. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems. 2022 Dec 6;35:32598-611.

[3] Van Amersfoort J, Smith L, Teh YW, Gal Y. Uncertainty estimation using a single deep deterministic neural network. InInternational conference on machine learning 2020 Nov 21 (pp. 9690-9700). PMLR.

Thanks for all the time you have put into giving feedback on the paper and clarifying the concerns.

1. On the ImageNet experiments

We have started running the experiments on ImageNet. We haven’t worked on the AugMix and PixMix yet since we were dedicating our efforts and resources to the scalability concerns you raised (which we also think are the most important) and need to be prioritized given the tight timeframe and the resources we currently have. While still very preliminary results, here are some conclusions we can draw.

With a codebook of size 30 and encoders of 128-dimension (we assume diagonal covariance in contrast to the cifar-10 experiments), we observe the following:

• UA-IB is able to match the accuracy of 76% of a single ResNet 50 deterministic network (we compare against the pretrained network provided by Tensorflow). We think this is an important result given the compression of the network due to the UA-IB regularization. A similar conclusion was reached in the VIB paper [1].

• Training takes 20 minutes per epoch on 8, 48 GB A6000 GPUs and batch size 128 per core.

• We did not observe any stability or non-scalability issues.

• Calibration AUROC reaches 0.78.

Edit: we started a second experiment right after posting this with codebook size 80. The Calibration AUROC is currently 0.80 (training has not finished yet).

We do not have results on OOD experiments yet. Moreover, we have not finetuned the codebook size and the rest of the hyperparameters yet. This result refers to a single configuration we tried and took 1.5 days to complete given the hardware we currently have ( a single 8-GPU server).

2. On the regression experiments

It either does comparatively or worse for some metrics than an ensemble of linear regressors (I have to guess, given the lack of response)

We definitely expect that large ensembles will outperform UA-IB as well as other DUMs. However, we do not consider ensembles direct competitors of DUMs.

[1] Alemi AA, Fischer I, Dillon JV, Murphy K. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410. 2016 Dec 1.

Albeit not critical, we agree with your claim:

"I invite the authors to run their code evaluating the AUROC by flipping the choice of the positive and negative class in the presence of strong class imbalance "

but one has to be careful to also flip the score appropriately.

However, we insist on the difficulty/ importance of the cifar-10 benchmarks. We refer you to empirical evidence provided by other researchers, particularly in Table 1 of: https://arxiv.org/pdf/2210.07242.pdf

In Table 1, you will see that methods on Far-OOD tasks on Imagenet achieve higher scores compared to the far-ood tasks for the cifar datasets. This conclusion is made clear in the main text:

CIFAR Benchmark is NOT Easier than ImageNet Benchmark We find that the OOD detection performance score for the ImageNet dataset is generally higher than that of CIFAR-10 and CIFAR100, which is another surprising discovery considering ImageNet is composed of more complex data than others and seems difficullt

We thank the reviewer for the thorough and detailed review. Please find our replies below.

1. the paper would benefit from a dedicated section that delves into how hyperparameter affects the ua-ib performance.

As suggested, we have added ablation studies over UA-IB’s hyperparameters in Appendix D.3 in the revised version of the paper.

2. could you discuss potential post-hoc integration strategies for ua-ib with trained models and the expected challenges or benefits?

One way this could be done is to apply UA-IB (i.e., codebook and encoder of the pretrained features) on features extracted from the pretrained network. The original VIB paper (section 4.2.5 in [1]) considers a similar setup. We have revised the paper to discuss this post-hoc integration in Sec. 7.

3. why are you not sharing the code for reviewing?

We have added the code to the supplemental materials.

4. scalability to the data sizes seen in high-dimensional image or language tasks?

a. Fundamental methods research and scaling research are both important. This work is studying the former. For example, the impactful SNGP paper [2] was not originally demonstrated on large-scale images. However, we do agree that large-scale experiments will increase the impact of our paper.

b. We are actively working on that.

c. Although CIFAR-10 is a smaller-scale problem, it is widely regarded as a challenging OOD task as difficult as ImageNet. [3], [4].

d. To broaden the impact of our paper, we expanded our experiments and tested our model on two regression tasks in Appendix d.1.

5. could you situate the ua-ib's approach in the broader landscape of lightweight uq or explain why certain methods are not relevant in your view?

At a high level, one key difference between UA-IB and all the rest of UQ methods is that it both regularizes the network (since the codebook is much smaller than the number of training datapoints) and equips it with distance-aware uncertainties (uncertainties that should grow away from the training data). Moreover, our method can measure uncertainty for both regression and classification tasks in contrast to quantile regression, interval networks, and variance prediction which handle only the former. Regarding [7], it can model only aleatoric uncertainty. The primary concern of our work, similar to Gaussian Process models, ensembles, Bayesian Neural Network etc is the estimation of epistemic uncertainty instead which is generally considered a more challenging task [5]. Finally, conformal prediction is a post-hoc uncertainty quantification method. However, leveraging uncertainty during training (or at a fine-tuning phase where UA-IB and backbone network are not jointly trained) can also help the model improve its accuracy [6]. We discuss this as future work in the draft.

References:

[1] Alemi AA, Fischer I, Dillon JV, Murphy K. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410. 2016 Dec 1.

[2] Liu, J., Lin, Z., Padhy, S., Tran, D., Bedrax Weiss, T. and Lakshminarayanan, B., 2020. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances in Neural Information Processing Systems, 33, pp.7498-7512.

[3]Yang J, Wang P, Zou D, Zhou Z, Ding K, Peng W, Wang H, Chen G, Li B, Sun Y, Du X. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems. 2022 Dec 6;35:32598-611.

[4]Van Amersfoort J, Smith L, Teh YW, Gal Y. Uncertainty estimation using a single deep deterministic neural network. In International conference on machine learning 2020 Nov 21 (pp. 9690-9700). PMLR.

[5] Kendall, A. and Gal, Y., 2017. What uncertainties do we need in bayesian deep learning for computer vision?. Advances in neural information processing systems, 30.

[6] Wilson AG, Izmailov P. Bayesian deep learning and a probabilistic perspective of generalization. Advances in neural information processing systems. 2020;33:4697-708.

[7] Gast J, Roth S. Lightweight probabilistic deep networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018 (pp. 3369-3378).

dear authors,

thank you very much for your comprehensive response.

i very much appreciate the detailed ablation experiments in d.3 as well as the release of the code.

i raised my score to accept in response to your revisions.

the experiments still leave room for improvement wrt the complexity of the data. i also think that related work should be contextualized more fully in the final version, as pointed out by me and other reviewers.

however, i do not agree with other reviewers that these shortcomings invalidate the overall contribution.

in good spirits,

reviewer 81b3