Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection

We thank the referee for their careful consideration and feedback.

Regarding efficiency and the scalability, is the reviewer referring to the computational overhead of the ensemble method ? When inference cost is an issue, the fast version, ENS-F, only incurs a 25% overhead compared to inference without detection, while still performing competitively. Such an overhead is rather low in comparison to recent work, including the acclaimed ViM, ODIN etc, that lead to overheads larger than 100% (see Table 6). While simple baselines such as MSP, max logits and logits norm incur a negligible overhead, most recently published work have comparable or higher overheads than ENS-F. ENS-R and ENS-V are indeed inefficient, but may still be adequate in scenarios where inference cost is not a significant issue.

We hope our work will encourage the community to stop over-focusing on novel class detection and instead consider the broader needs of real world applications. Ideally, novel scores will emerge which will be better suited to broad OOD detection. Such scores may not be SotA on novel class detection, but achieve significant improvement in broad OOD detection, and would not have attracted attention without the correct evaluations. However, finding a single method that can efficiently discriminate a large variety of distribution shifts, each with their own unique properties, may prove difficult. In that case, we believe ensemble methods may remain an efficient way to address this complex and multi-faceted problem. We admit this is not the most satisfying solution, but it has significant merits, and might currently be the most adequate method to real world usage in many settings.

We thank the referee for their careful consideration and feedback.

Weakness 1 As stated page 7, at the end of Section 3, we do not assume access to OOD samples to pick which scores to use for ensembling, precisely to avoid the issue mentioned by the referee. Instead, we describe the selection process for the different ensemble versions, which are only based on in-distribution ImageNet validation inputs, and are thus independent from performances on BROAD. We could likely achieve better performance by picking scores based on BROAD, but refrained from it in agreement with the referee’s remark.

Weakness 2 Overall, generative modeling with GMM is better suited because it models exactly the quantity of interest: how likely is it to observe the given set of scores, assuming we are in-distribution ? While the referee’s suggested heuristic discards valuable information.

While the referee says “these methods aim at measuring if the sample is in-distribution”, and it is indeed the claimed goal, our work precisely establishes that due to standardization of evaluation benchmarks in the literature, these methods instead only discriminate between in-distribution and novel classes. Their behavior on other types of distribution shifts is a priori uncertain. Some methods have below random AUC performances for certain distribution shifts (see table 2), which demonstrates it is not always the case that a lower score means a higher chance of being in-distribution (instead, it means a higher chance of not being a novel class).

Moreover, the sentence cited by the referee simply points out that generative modeling is able to model information in the joint distribution that is not present in marginal distributions. For instance, EBO and MaxLogits are heavily correlated in-distribution (see Figure 3). Thus, even when the scores of EBO and MaxLogits are both within reasonable range, a lower-than-usual score for EBO and higher-than-usual score for MaxLogits have low joint density, which would not be captured by marginal probabilities.

Additionally, nothing indicates that the distribution of scores must be single mode. As clearly shown in Table 7 (appendix), using a mixture of several Gaussians performs better than a single Gaussian. An average of scores might be unadapted if the distribution of each score has different modes.

Finally, even if we rescale scores to balance their variance, depending on the functional form of their distribution they may not trigger false positives at the same frequency, and redundant scores would have exaggerated weights.

Weakness 3 As the referee acknowledged, a 25% overhead is acceptable. While some methods indeed have faster inference, half of the methods have comparable – or even much higher, overhead. Many acclaimed recent works such as ViM and ODIN have overheads larger than 100% and were still found to be of high interest to the community. Besides, several of the fastest methods (MSP, max logits and logits norm), are simple baselines commonly used in the literature and unrepresentative of typical overhead in published methods. We believe Ens-F has, in fact, a smaller overhead than most methods introduced in the past 3 years, and it should thus not be considered a limitation.

Q1 It is important to note that the training of the GMM uses 45,000 in-distribution validation samples, from the validation set of ImageNet (the comparison would indeed be unfair if our ensemble had access to some OOD samples). First, it is unlikely that the use of 45,000 additional images compared to the 1.2 millions samples in ImageNet 2012’s training set would make a significant difference. But most importantly, it is common for the methods in the literature to use validation data for various purposes, such as tuning (CADet), learning parameters (MDS), or calibrating (ViM). Since we use the same validation data as most methods we compare to, we believe it is fair. Instead, it can introduce a slight overfitting of the underlying scores to the validation data on which the GMM is trained, hindering performances, but we find the GMM still largely outperforms underlying methods.

Q2 We did not evaluate the heuristics mentioned by the referee. Firstly, because a generative approach seemed better suited due to the elements mentioned in the response to weakness 2. Secondly, because we wanted to refrain from picking the ensemble method based on BROAD performances, which would incur risks of overfitting our specific benchmark, and may lead to bad generalization on unseen OOD datasets.

We thank the referee for their careful consideration and feedback.

Weaknesses

We agree OOD detection scores considered in the paper are only suited for novel class detection (though some of them generalize well on certain other types of shifts). In fact, it is exactly our point: while described as “OOD detection” methods, these are, in fact, “novel class detection” methods. While they are virtually always justified in the literature by the need to address unexpected test-time inputs, they are in fact only evaluated on novel class benchmarks. There is thus a discrepancy between the role that these methods are supposed to play, and what they are actually evaluated on and suitable for. Our work proposes to fix this issue by introducing a more comprehensive evaluation benchmark. The fact that these methods generalize unreliably to other types of shifts is indeed not surprising, since they were not evaluated on them in their original papers. However, it justifies that, in fact, there is a misalignment between the literature and the needs of real-world systems, where a large variety of unexpected inputs can occur. Existing “OOD detection” methods fail to detect certain types of OOD inputs encountered in real-world settings, as demonstrated by our work. In order to move toward useful systems, we need to move toward a more realistic definition of OOD detection, and understand how existing methods perform in such a setting, which is exactly what our submission does. Thus we believe it is not out-of-scope to evaluate OOD detection methods on distribution shifts they were not designed for – but that occur in the type of problem they aim to address.

[1] Is a very interesting work that establishes that it is impossible to discriminate the training distribution from all possible distribution shifts. However, it does not mean that it is impossible to discriminate from the specific distribution shifts encountered in practice, which are only a small subset of all possible shifts. In fact, the performance of our ensemble method formally demonstrates that, at least to a reasonable extent, it is possible to simultaneously detect all the distribution shifts considered in BROAD. We argue there is a reasonable middle point to find between only considering detection of one specific and limited type of shift, and trying to discriminate against all possible (in the mathematical sense) shifts. That middle point should be to attempt detection of the shifts reasonably encountered in practice, and our work shows that, though difficult, it is not infeasible.

Indeed, using an object detection method would be much better suited for cocomagenet samples, which would in that case not even be considered OOD. But the principle of OOD detection is to protect against input types that are not expected, thus we can not assume a priori knowledge of these input types to select a better suited model or paradigm. There are many classification models deployed in the real world, which are designed to handle single-class inputs. Due to unexpected use, they may occasionally be presented with inputs that, in fact, have several training classes present. Since such cases are uncommon and out of normal usage, it is often not reasonable to consider an object detection framework just to palliate this eventuality. This is connected to the general discrepancy noted by our work between real-world needs and the literature: the standardization of evaluation benchmarks has led recent OOD detection methods to only consider expected OOD inputs, although their point is precisely to detect inputs that are not expected.

Q1: Certain of these methods have been designed for simultaneous detection of adversarial samples (e.g., CADet) while others have been found in prior work to be powerful baselines in detecting adversarial inputs (e.g., ODIN and MDS). However, as argued above, our point is precisely to demonstrate that existing OOD detection methods are in general not suitable to detect other shifts encountered in practice.

Q2: We are not sure what would be the grounds for detecting multi-label inputs if the deployed system is tailored to and trained on multi-label inputs. In that case being multi-label would not correspond to a different distribution.

We thank the referee for their careful consideration and feedback.

Reproducibility: As indicated in the abstract, we will release the full code to reproduce our experiments for the camera-ready. In terms of describing how the GMM is trained, as stated in page 7, “we propose the learning of a Gaussian mixture of the scores computed by existing OOD detection methods”. Thus given a set of scores $\{S_1, S_2, …, S_m\}$ (the selection of which is discussed in detail), we simply train a GMM to learn the density of the score tuples $(S_1, S_2, ..., S_m)$. Given an image training set $\{X_1, X_2, …, X_n\}$, it means the GMM is trained on \{(S_1(X_1), S_2(X_1), ..., S_m(X_1)), …, (S_1(X_n),S_2(X_n), …, S_m(X_n))}. If the reviewer feels that such a process is not naturally implied by the above sentence, we can add a paragraph laying these specifications. There are two additional details we indeed forgot to specify, that should be fairly straightforward: we train the GMM using the EM algorithm as is typically done, and given a test sample $X$, we use the negative likelihood of $(S_1(X), S_2(X), …, S_m(X))$ from the GMM as detection score. That is, we detect as OOD the samples for which the learned density of the tuple of score is below the threshold. For completeness, we will explicitly mention these two parts of the detection pipeline.

Novelty and significance: We agree with the referee that an ensemble method outperforming the underlying learners is nothing surprising, and does not warrant publication. The main point here is not just that the GMM significantly increases average performances, but that it manages to achieve a level of consistency across distribution shift types that individual methods do not have. The main contributions of our work are to establish that so-called “OOD detection” methods in the literature are in fact “novel classes detection” methods, and over-specialize to that specific distribution shift types, while behaving unreliably on general OOD detection. We propose a novel benchmark to evaluate broad OOD detection, hoping it will encourage the design of more general methods that will perform reliably in the real world. In the meantime, the performance of our GMM approach indicates that by combining the respective strengths of existing scores, it is able to attain a level of reliability across distribution shifts unmatched by baselines.