LAION-C: An out-of-distribution benchmark for web-scale vision models

Dear Reviewer,
Thank you for your helpful review and interesting questions! We were pleased to read that you found our paper clearly structured and believe that we address a critical issue. Your main observation regarding the lack of differences in the resulting model rankings provides an interesting perspective. We conducted an investigation of other well-established OOD-benchmark datasets, which revealed a high degree of correlation between all of them, even though they measure different aspects of robustness (see here: https://ibb.co/93k5ns5). Arguably, the ranking of models is not the only relevant conclusion to be drawn from these benchmarks, but the absolute performance of models matters as well. In our case, LAION-C reveals that all models struggle more with our synthetic corruptions than they do with the more reasonable corruptions of ImageNet-C.

Q: “[A]re there observed differences in how CNNs and transformers perform on these types of distortions?”
A: We have created a new figure showing the differences in performance between CNNs and Transformer-based models: https://ibb.co/2dhpqkn. In summary, transformers always perform better than CNNs, but it is important to keep in mind that the latter class contains smaller and older models. Our figure 10 shows a more detailed breakdown of models and implies the same conclusion.

Q: “What methods or adjustments could be employed to address this disparity [of number of classes] and ensure a fair evaluation between the datasets?”
A: Thank you for this interesting question. We have also created a 1000-class version of LAION-C, but to ensure a fair comparison between human and machine performance, we implemented the 16-class version (since the 1000-class version would be practically impossible for human observers). The current intensity levels of the corruptions areis also tuned for such scenarios. Conversely, the 16-class version of ImageNet-C does not pose a significant challenge to our suite of models. We provide an additional figure comparing the different datasets here: https://ibb.co/Z2Mzv3d. The code we will release upon publication will allow other researchers to apply our corruptions to arbitrary images with adjustable strength, so re-creating a 1000-class version of LAION-C is possible, should the need arise.

Dear Reviewer,
Thank you for your insightful review. We were delighted to read that you valued our human experiments and the comprehensiveness of our model evaluations.

Q: “[Could] the same study [...] have been performed using ImageNet-C instead?”
A: Thank you for bringing up this point. The crucial difference between ImageNet-C and our new LAION-C dataset is that the ImageNet-C corruptions (such as noise, frost and blur) are fairly natural and appear in today’s large-scale training sets, thus they cannot be used for OOD testing anymore. In contrast, we measure OOD-performance on a highly synthetic, but truly OOD dataset. The fact that (the effectively in-distribution) performance on ImageNet-C still seems to be a good predictor of OOD performance is arguably nontrivial, but in line with work such as [1]. See also our response to reviewer wQbV for an analysis of the correlation between LAION-C and well-established OOD benchmarks.

Q: “[Are] these distribution shifts [...] useful?”
A: Using your nomenclature, LAION-C would indeed be a dataset of type B: fully synthetic, and allowing us to study the differences between human and machine vision. Specifically, the difference is that models pick up on features that seem inaccessible to human observers, allowing them to solve even the harder intensity levels of our corruptions. This insight seems useful to us, since DNNs have been proposed as models of human vision [2,3], but one would want a good model to exhibit similar behavior (including failure modes) as the modeled system. We think that datasets consisting of corruptions to which the primate visual system is robust are not the only useful kind of dataset, but that there also is value in showing inconsistencies in “the other direction”, i.e. datasets where models are robust, but humans are not. Going beyond that, there is another purpose that LAION-C serves: Since it is so synthetic and unrealistic (i.e., not type A) we can really use it to measure the OOD robustness of models trained on web-scale datasets, which essentially have to be assumed to contain everything that is realistic, rendering truly OOD type A datasets practically impossible.

Q: “How is this process reminiscent of the forward pass in a network?”
A: Thank you for the question, we have now clarified that section of the paper. We were not referring to the information processing in participants’ visual cortex, but only to the task itself: Human participants are presented with an image and perform a classification task in which they assign one of 16 possible classes to each image, which is similar to how supervised models are evaluated (vs. for example an open response format, in which they could provide an arbitrary textual answer).

[1] Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization https://arxiv.org/abs/2107.04649
[2] Deep neural networks: a new framework for modeling biological vision and brain information processing https://www.annualreviews.org/content/journals/10.1146/annurev-vision-082114-035447
[3] Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs https://arxiv.org/abs/1909.06161

Dear Reviewer,
Thank you for your insightful comments! We were delighted to read that you found the presentation of our work clear and see the value in the human experiments we conducted. We highly appreciate your in-depth review of our work and believe that the additional experiments we conducted to address your suggestions have greatly improved the manuscript.

Q: “Does improving on these distortions imply improving model performance on diverse unseen real world distribution shifts?”
A: That’s a great question. Did you have any specific dataset in mind for unseen real world distribution shifts? To address your question, we analyzed the correlations between model performance on LAION-C and on several established OOD datasets, including IN-C, IN-A, IN-R and IN-Sketch. Specifically, we subsampled a total of 18 models trained on both LAION-2B and ImageNet (IN) to control for architectural variations, focusing on recent architectures such as ViT and ConvNext-inspired models. Our analysis reveals a positive correlation between model performance on LAION-C and all investigated OOD datasets, suggesting that performance gains on LAION-C do indeed translate to other distribution shifts. We have updated our findings in section 3.3 , and you can also find the results here: https://ibb.co/93k5ns5.

Q: “A ratio of occlusion of the object under these distortions would be helpful to understand the severity levels”
A: Thanks for the suggestion! We have updated section A.4 to include the occlusion ratio of the objects for geometric shapes and stickers (you can also access the table here: https://ibb.co/bJjnV7p). In the same section, we also explain in more detail how we constructed each distortion. More generally, we share your intuition: At higher intensity levels, most of our corruptions are increasingly hard to solve. In principle, we could have chosen to keep only one intensity level (e.g. level 2) or limited our analysis to, say, the first four levels only. However, it is a common practice in psychophysics to test the full range of difficulties, from perfect performance to chance performance (e.g., see [1] Figure 2). ImageNet-C falls short in this regard, since even the hardest severity can often be solved too easily. We decided to include the harder images, leaving the choice of their inclusion in other experiments to the researchers conducting those. We also wanted to leave room for future models that potentially drastically improve upon this reference model, which simply did not happen for any of the evaluated models.

Q: “Fine-tuning experiment: [I]mprovements on the extreme severities could have been the result from learning spurious or shortcut features”
A: Thank you for raising this point. We fully agree that high model performance on the hardest severity levels is likely a consequence of models learning strategies that differ greatly from those that humans would use. It seems that we haven’t sufficiently explained the goal of the fine-tuning experiment: The purpose is not to suggest that fine-tuning improves robustness, but simply to quantify that even at high levels of noise there is signal left—as opposed to purely random noise, in which case fine-tuning wouldn’t be able to improve over chance performance. There is still some learnable signal that allows correct classification, even if it is inaccessible to humans (i.e., “spurious features”). We think of the performance after fine-tuning as an upper bound that serves only to put the performance of other models and humans into perspective. We have now updated section 3.4 to better explain this idea.

Q: “[W]ould [the ImageNet-C-bar] distortions be eligible to become part of LAION-C?”
A: LAION-C could indeed be extended by further distortions such as ImageNet-C-bar, provided that the authors agree and that the distortions don’t appear in a large dataset like LAION already. We appreciate the pointer to this related work as it further emphasizes the importance of perceptually dissimilar corruptions for OOD benchmarking scenarios. We have updated our manuscript to include this work in our related work section.

[1] Partial success in closing the gap between human and machine vision https://openreview.net/forum?id=QkljT4mrfs

Dear Reviewer,
Thank you for your constructive suggestions and insightful questions. We greatly appreciate that you found our experiments valuable and thorough. We have updated the manuscript to better motivate why another robustness benchmark might indeed be helpful, see section 3.3 – in short, we believe that existing benchmarks no longer suffice, since they contain corruptions that are also present in today’s web-scale datasets. Could you provide the reference [1] you were referring to, so we can take a look?

Q: “Why is there no evaluation of Vision-Language models?”
A: Thanks for this excellent suggestion. We have now evaluated GPT-4o on a representative subset (48k samples) of LAION-C images and updated figures 3, 4 and 10 accordingly (see also here: https://ibb.co/album/wYwg7y). This analysis shows no fundamental differences between VLMs and the other models we evaluated, but this is interesting in itself and we appreciate the suggestion. Since we provide LAION-C as a publicly available dataset,the community will be able to test future models as well.

Q: “What was your reasoning for selecting these 6 specific types of corruption?”
A: Our 6 specific types of corruption have been selected from a larger set of candidate corruptions on the grounds of eliciting the greatest variance in performance of a suite of models, while also being parameterized and cheap enough to compute for a large dataset. For the purpose of a “truly OOD” test set, the specific choice of corruption types matters less than the fact that they were not present in the training data distribution, a finding that we extensively quantify in Section 3.1 of the paper. We now explain this motivation more clearly in Section 2.1.

Q: “What's the rationale for including these extremely challenging cases?”
A: It is correct that at higher intensity levels, most of our corruptions are increasingly hard to solve. In principle, we could have limited our analysis to, say, the first four levels only. However, it is a common practice in psychophysics to test the full range of difficulties, from perfect performance to chance performance (e.g., see [1] Figure 2). ImageNet-C falls short in this regard, since even the hardest severity can often be solved with very high accuracies, limiting its use as a challenging benchmark. We also wanted to leave room for future models that potentially drastically improve upon this. That said, no model is expected to perform well on the hardest severity level, which we now state clearly in Section 2.1.

Q: “If even humans can't identify these images, what value does testing models on them provide?”
A: For many applications of AI (autonomous driving, medical imaging), super-human performance is desirable. As models get better, benchmarks need to become more challenging. For instance, when the popular SWE-Bench coding benchmark was introduced, the best model performed at only 4% (and many untrained humans would do even worse), while now models are at nearly 50% performance. Since humans have, historically, been the gold standard for OOD-performance (as e.g. stated in [1,2]), our experiment aims to assess how the current generation of models performs on difficult stimuli compared to humans. We find that modern models now outperform humans on OOD tasks, while earlier models still struggle to do so (see appendix A.6). However, the low error consistency between humans and models implies that contemporary models do not achieve "super-human" performance by merely replicating human visual strategies. Instead, models are leveraging unique cues and strategies beyond those used by humans.

[1] Partial success in closing the gap between human and machine vision https://openreview.net/forum?id=QkljT4mrfs
[2] Human and DNN classification performance on images with quality distortions: A comparative study https://dl.acm.org/doi/10.1145/3306241