ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms

Dear Reviewer, Thank you for giving us your time and energy for the thorough and insightful review of our paper. We are encouraged that you appreciate the importance of our motivation, the benefits of our novel dataset, and the clear presentation of method and results. We greatly appreciate the suggestion provided. Please see our responses to the weaknesses you pointed out below:

W1: paper focuses on dataset and analysis but does not provide its own OOD detection method

Absolutely, we do not provide a new method for OOD detection. As we note in the abstract, our contributions are as follows: “we present ImageNet-OOD, a clean semantic shift dataset that minimizes the interference of covariate shift. Through comprehensive experiments, we show that OOD detectors are more sensitive to covariate shift than to semantic shift, and the benefits of recent OOD detection algorithms on semantic shift detection are minimal. Our dataset and analyses provide important insights for guiding the design of future OOD detectors.” Our work follows a long line of works (please see section A of appendix) which provide detailed analyses and insights into existing paradigms and approaches, rather than developing new models.

W2: lack of new methods from CVPR and lack of hyperparameter tuning

Integrating recent CVPR methods is an excellent suggestion. We added a new method Max-Cosine (Zhang and Xiang) from CVPR 2023 and also another method ASH-B (Djurisic et al.) from ICLR 2023. We kindly ask that you please note that works published in CVPR 2023 are within the four months prior to ICLR submission and therefore, are considered contemporaneous by the ICLR policy.

We included Max-Cosine and ASH-B in Table 1 (pg 8), integrated them into the summary of the results in Section 4.3, and added additional qualitative analysis in Section G of the appendix (all highlighted in blue). Briefly, we find the same conclusion: the best performing detector is now ASH-B on OpenImage-O with improvement of 5.5 points on AUROC while the detector perform worse than the MSP baseline on ImageNet-OOD; Max-Cosine is the best detector on ImageNet-OOD but the improvement from the MSP baseline is still less than 1 point on AUROC.

Regarding the hyperparameters, the methods we used are either hyperparameter free (MSP, Energy, Max-Logit, Mahalanobis, and the newly added Max-Cosine) or the methods are tuned towards ImageNet, our particular setup, in the original paper of the method (ViM, KNN, ReAct, and newly added ASH-B). As a result, for fair evaluation, we used the best suggested hyperparameter based on the ablation studies performed in the original paper. We added this clarification at the end of the second paragraph of section 4.

Dear Reviewer, Thank you for giving us your time and energy for the thorough and insightful review of our paper. We are encouraged that you appreciate the usefulness of our novel dataset, insightful analysis towards susceptibility towards covariate shift, and calibration of expectations in the OOD community. We greatly appreciate the confusions and weaknesses you pointed out and updated parts of the paper to improve it. Please see below:

W1: focus on covariate shift undervalues semantic shift

One of the primary goals of our work is to actually curate a clean dataset to better evaluate new-class detection. We directly emphasize the motivation of constructing ImageNet-OOD at the start of Section 3: “...new-class detection is still relevant in practice with the growing interest in adaptive learning systems.” Additionally, we delineate the difference between new-class detection and failure detection at the end of section 2 to emphasize the importance of semantic shift. We made some minor changes at the beginning of section 3 (highlighted in blue) to make this point clearer.

W2: analysis on characteristics of examples that lead methods to confuse covariate and semantic shift

This is a great suggestion. We conducted additional qualitative analysis (included in Section G of the appendix) and found that the OOD detection algorithm ViM tends to score texture-like images as more OOD when compared to MSP and the OOD detection algorithm ASH-b tends to score text-like images more OOD when compared to MSP. Additionally, we observe that text-like/texture-like images from ImageNet-OOD are more closely aligned with the domain of ImageNet-1K. Since ASH-b and ViM are the two best performing algorithms on OpenImage-O, these visual examples explain the disappearing performance gains when evaluating on ImageNet-OOD.

W3: presentation and writing issues

Thank you for pointing this out. We clarified our high level contribution at the end of our introduction (highlighted in blue). Please note that ICLR policy this year prevents us from making changes to the abstract. We also did another pass to fix some minor grammar and presentation issues (highlighted in gray).

Dear Reviewer, Thank you for giving us your time and energy for the thorough and insightful review of our paper. We are encouraged that you appreciate the identification of crucial issues in OOD detection research, their mitigation in our novel dataset, and our unified assessment of OOD detection algorithms. We greatly appreciate the references and concerns you pointed out and updated parts of the paper to improve it. Please see our response to the concerns below:

W1-2: OpenOOD v1.5 shown that OOD detectors are susceptible towards covariate shifts and practical benefits disappear in near-OOD datasets

Thank you for pointing this out. We updated the related works section (section A of appendix) to include a pointer to OpenOOD v1.5 (blue text on pg. 14). The key distinction with our analysis is that we studied the effect of covariate shift independently and found that the baseline MSP is more robust towards covariate shifts than modern OOD detection algorithms. OpenOOD v1.5 performed the analysis in the full spectrum (ID combined with covariate shift) setting, which hid this fact, and only concluded that such setting poses a challenge for OOD detection. We updated section 4.3 and introduction to make this point more clear. Additionally, we would like to note that as you point out, this work has only appeared on arxiv and has not yet been published in peer-reviewed venues, thus we hope it does not affect assessment of novelty of our work as per ICLR guidelines.

W3: Animal and Vehicle is an example of subpopulation shift

This is a great suggestion and we updated the paper to reflect this in the “semantically-grounded covariate shifts” paragraph of Section 3.2 (bottom of pg. 4, highlighted in blue).

W4: paper is better suited for Dataset and Benchmark

We understand where you’re coming from but respectfully disagree. The Datasets and Benchmarks track provides a much-needed venue to share generalizable insights into dataset construction and present innovative approaches to benchmarking. In contrast, in our work, while the ImageNet-OOD dataset is very important to our analysis and is carefully constructed to suit this need, it does not necessarily reveal meaningful insights into dataset construction in general. Further, we believe our analyses and takeaways about OOD detection algorithms are critical towards guiding better methodological design; the construction of the dataset itself is a secondary contribution that merely enables these findings. This is why we maintain that this paper is better suited for a broad audience conference instead of a dedicated track.

Q1: how thorough and rigorous is the human inspection and how does it compare to NINCO?

Two rounds of manual human inspection are performed by the authors of the paper, first at the class level (20 hours), then at the image level (6 hours). We first manually inspected sample images from direct neighbors of ImageNet-1K classes and selected 1000 classes that are visually unambiguous to their ImageNet-1K neighbors. Then, we inspect the top 1000 images that are closest to ImageNet-1K in terms of ResNet-50 feature distance to filter out classes that are visually ambiguous with non-neighboring ImageNet-1K classes to arrive at the final 637 classes. Since the ImageNet-21K images are all validated by crowd workers from Amazon Mechanical Turk, we dedicated most of our time to class-level inspection, while still performing image-level analysis as an additional layer of reassurance. Please refer to Appendix Section H for more details on the construction of ImageNet-OOD.

NINCO was a smaller dataset designed to avoid data contamination of in-distribution classes. In comparison, in addition to avoiding data contamination, our dataset is designed to mitigate unwanted covariate shifts, which is crucial for our analysis.

W2, Q4-5: have we considered an automated process such as CLIP zero-shot embeddings to establish a threshold for removing visual similarities?

This is a great suggestion but we were solving a different issue with our manual class selection. For our use case, we performed manual inspection of the classes and some of their respective images to eliminate visual ambiguity, not identify classes with high visual similarity. Using CLIP zero-shot classification would not be able to distinguish between the two. We illustrate two examples from experimenting with CLIP zero-shot classification on ImageNet-OOD images with label of the image and ImageNet-1k classes:

• The CLIP model confuses images of Capote (ImageNet-OOD class), which is a type of coat made of wool/fabric with a hood and typically worn by Native Americans, with images of Trench Coat (ImageNet-1K class), which is a long waterproof coat. This is an example of two objects that are visually similar but not visually ambiguous. If we manually inspect the images of Capote and Trench Coat, we can clearly see the subtle but undeniable difference between the objects.
• The CLIP model confuses dump cart (ImageNet-OOD class) vs. wheelbarrow (ImageNet-1K class). Both objects are visually similar but there is a clear semantic difference between the two objects: one wheel vs. two wheels.

Visual ambiguity, in contrast to visual similarity, arises due to flaws in the data annotation process of the original ImageNet-21K dataset rather than properties of the data itself. In the ImageNet-21K dataset, data labels are derived from queries to the internet search engine and verified using human annotators from Amazon Mechanical Turk. They ask human annotators the question “is X in the image?” based on the query used. This can create labeling mistakes from visually ambiguous objects such as the violin/viola example shown in Figure 1. Given a picture of viola without any context clues to its size, it is difficult for a human to distinguish the image from violin vs viola. Since the human annotators were asked “is this a violin” or “is this a viola” instead of contrasting between the two, the labeling can go either way depending which question was asked. We circumvent this issue with manual inspection of the classes using our prior knowledge and intuition about object properties to select 637 classes for ImageNet-OOD that would not have such an issue. This way we can be a lot more confident we did not inadvertently include images potentially corresponding to ImageNet-1K classes in the ImageNet-OOD set.

Reference: Section 3 of Deng, Jia, et al. "ImageNet: A large-scale hierarchical image database." CVPR 2009

W3, Q6: have we considered other types of covariate shifts on random models? How does semantic shift compare?

This is a great question. We actually considered such covariate shifts in Figure 4 of the paper with ImageNet-R (https://github.com/hendrycks/imagenet-r). ImageNet-R is a dataset consisting of ImageNet-1K objects in different domains such as video games, paintings, etc. We did not include background shifts as such datasets do not currently exist for ImageNet-1K to the best of our knowledge and are difficult to curate. This is because ImageNet-1K, which is the most commonly-used in-distribution dataset in the field, is an object-centric dataset that contains no information on the background diversity. Due to the nature of ImageNet-1K, our paper focuses on object-level covariate shifts, which is an orthogonal issue to background-level covariate shifts.

Regarding semantic shift, this is an excellent suggestion and we updated the paper to include semantic shift results with ImageNet-OOD in Figure 4 (page 7). The results revealed that OOD detection algorithms on randomly-initialized models do perform very close to random chance when evaluated with the semantic shift of ImageNet-OOD. This is in contrast to covariate shift shown with ImageNet-R or ImageNet-C, which the OOD detector has a bias either towards detecting as ID or OOD.

Dear Reviewer, Thank you for giving us your time and energy for the thorough and insightful review of our paper. We are encouraged that you appreciate our novel dataset and the extensiveness of our experiments. We've deeply considered your suggestions and are happy to discuss improvements. Please see below:

W1, Q1: real-world scenarios have both covariate and semantic shift present. Have we considered distance measures between datasets?

Incorporating dataset distance measurements is a great suggestion and something that we have previously investigated. We tried using the OTDD work you suggested, but unfortunately it was too memory intensive to fit on ImageNet, ImageNet-OOD, ImageNet-R, and ImageNet-Sketch due to the high resolution images and high number of classes (they defined a dataset as feature-label pairs and they only tested on low resolution datasets such as CIFAR-10/Tiny-ImageNet). Instead, we did try a common distance metric from the field of generative modeling: Frechet Inception Distance (FID), which represents the Wasserstein-2 distance in feature space. We did not find a consistent relationship between dataset distance and OOD detection performance. On CLIP embedding, we receive the following FIDs against ImageNet-1K images

• 1.36 for ImageNet-Sketch
• 0.99 for ImageNet-R
• 1.07 for ImageNet-OOD

We observe a similar pattern on FIDs using embeddings of a ResNet-50 model trained on ImageNet-1K against ImageNet-1K images

• 21.8 for ImageNet-Sketch
• 13.4 for ImageNet-R
• 13.9 for ImageNet-OOD

From FID, we observe that ImageNet-R and ImageNet-OOD have similar distances while ImageNet-Sketch is significantly further away. Despite the fact that ImageNet-Sketch is significantly further than ImageNet-R, our results in Table 1 between ImageNet-OOD vs. ImageNet-R/ImageNet-Sketch have similar performance. Thus, we did not find any consistent relationship between dataset distances and OOD detection performance, and therefore, the susceptibility towards covariate shift cannot be explained by dataset distance.

Regarding real-world scenarios and what to do with covariate shift, we build off of existing literature for the proper procedure for handling covariate shift. As outlined in the introduction and described at the end of section 2 under the “modern approach” and “classic approach” subheadings (references provided in more detail in section A of appendix), OOD detection algorithms have two popular use cases: new-class detection and failure detection. The goal of our paper is to provide a dataset that can better assess an OOD detection algorithm’s performance for the task of new-class detection (i.e., in the context of semantic shift). Under the formulation of the task of new-class detection in previous works, covariate shift is not considered OOD, and therefore, should be not detected by an OOD detection algorithm. We added another sentence at the beginning of Section 3 (highlighted in blue) to further clarify.

W2, Q2-Q3: why repeat the hypernym and hyponym removal process for ImageNet-1K classes in addition to most general semantic classes? Why does the violin and viola example exist after removing semantically-grounded shifts?

You are correct that this process does not need to be repeated for the original ImageNet-1K classes. We divided them in separate sections in the paper for the sake of clarity but we thank you for this suggestion and further clarified in the end of the paragraph describing “semantically-grounded covariate shifts” (Section 3.2, bottom of page 4, highlighted in blue).

Regarding the violin and viola example: ImageNet-1K contains the classes violin and cello, which share a common parent “bowed stringed instrument”. Therefore, violin class would still remain in the most general decision boundary. As a result, viola would still create issues with the violin class.