Reassessing the Validity of Spurious Correlations Benchmarks

Thank you for your detailed and constructive feedback, and we apologize for giving you the impression that our submission had been rushed. We hope that we can assure you that this was not the case. We’ve just uploaded a new version addressing some of your feedback, including significantly larger figures. To respond to a few of your comments in turn:

Re general weakness 1

• The benchmarks we considered currently only cover classification settings, but they are not limited to computer vision: two NLP benchmarks, CivilComments and MultiNLI, are already included in our analysis.
• More broadly, however, we agree that spurious correlations are likely to continue to be a problem in more general settings and newer domains, including both text and image generation. The set of benchmarks we analyze is intended to cover the majority of present research into mitigating spurious correlations.
• We’d be happy to supplement our list if you have specific suggestions of existing benchmarks that we’ve neglected.

Re general weakness 2

• We discuss the reliance of our work, and all benchmarks within, on available (and high-quality) attributes on lines 521–530, and we’d be happy to expand this section if you think anything is missing.
• We would like to stress that our “practical guide” in section 5 is intended for practitioners faced with a choice between competing methods for mitigating spurious correlations. In reality, a researcher without access to a group-annotated test or audit set is unlikely to be considering this suite of methods in the first place.
• While an evaluation of benchmarks without group annotations would be a great next step, we don’t consider it within the scope of our contribution here.

Re general weakness 3

• If you have any specific recommendations for what you’d like to see, in terms of experiments, or to improve the presentation of section 5, we’d be more than happy to revise accordingly.
• To help clarify our results, we compare three approaches in section 5 and table 2. (1) Picking the best method according to performance averaged over all benchmarks. This produces GroupDRO. (2) Picking the best method according to performance averaged over only valid benchmarks, according to section 3. This produces ReSample. (3) Picking the best method on the closest benchmark according to K. This produces a different method per test dataset, and often improves performance compared with (1) and (2), as noted on lines 477–481.
• Table 2 currently covers 9 datasets, including Dollar Street to test the applicability of our approach to a held-out starting dataset, i.e., one not previously considered in our work. Do you have additional tests you’d like to see here?

Re weakness 6

• The caption for Figure 1 appears to be correct. In Figure 1, the legend on the color bar reads: “Rank (higher better)”. DFR is the best performing method on Waterbirds (rank 19), while the second worst on NICO++ (rank 2).
• We have now clarified this in an updated caption.

Thank you for your detailed review. We’ve uploaded a new version addressing the typos you describe, and have responded to your comments below. We’re open to making some changes to the framing as you suggest, and look forward to your feedback on our proposals below.

Title, abstract, and framing

• Thanks for suggesting this. We would be happy to consider adjusting the title and abstract, though we don’t fully agree that this paper needs to exclusively refer to subpopulation shifts instead of spurious correlations. We outline our reasoning below, and are open to working together to find a suitable middle ground here.
• While the problems we’re investigating are sometimes referred to as subpopulation shift, this terminology is not consistently used in the literature. For example, as you point out in your review, Geirhos et al. (2020) refer to the problem as “shortcuts.” We feel that “spurious correlations” is immediately legible to the broader machine learning community, and as a result it’s the language we tend to use.
• However, we propose restricting the focus of our abstract by modifying as follows (italics indicates new additions):
  • “Neural networks can fail when the data contains spurious correlations, i.e. associations in the training data that fail to generalize to new distributions.”
  • “To understand this phenomenon, often referred to as subpopulation shift or shortcut learning, researchers have proposed numerous benchmarks upon which to evaluate mitigation methods.”
  • “... we find that certain benchmarks are not meaningful measures of how methods perform when correlations don’t generalize.”
• As for the title, we’d be open to modifying it to something that reflects the above proposals, though we’d first like to seek clarification from the other reviewers and the AC that this is a mutually acceptable change.

Defining spurious correlations

• Unfortunately, we don’t think the definition of “spurious correlation” is quite so clear cut, as we explain below.
• Outside of machine learning, a statistician would define a spurious correlation as a correlation between two variables that aren’t causally related.
• Within machine learning, while Geirhos et al. (2020, p. 665) may define a shortcut as an association that fails to transfer to more challenging test conditions, Yang et al. (2023, p. 3) define spurious correlations as those that are present in the training set but not in the test set. These aren’t quite the same thing.
• Moreover, we don’t fully agree that a distribution shift needs to take place for a correlation to be considered spurious. For example, while CelebA is commonly understood as exhibiting a spurious correlation between gender and hair color, this correlation is equally present in both train and test sets. We think CelebA is an important benchmark to discuss in our work, but it doesn’t conform to a definition grounded in distribution shift.
• Overall, setting out a precise and unifying definition of spurious correlation is not the ambition of our paper, nor an area we dedicate much time to.

Group attributes

• We agree that our work is limited to the setting where group attributes are available, and don’t try to suggest otherwise. We explicitly discuss this limitation in our discussion section, and would be happy to expand this if you feel there are specific things to call out here.
• We would like to emphasize that our work isn’t to propose new benchmarks, but to evaluate the validity of existing benchmarks. Given that every benchmark we consider is a) equipped with group information, and b) commonly used for evaluating mitigations methods, we think this is a reasonable limitation.
• Considering our three desiderata specifically—ERM failure, discriminative power, and convergent validity—they are explicitly intended to be applied to benchmarks where group attributes are available. For future benchmarks without group information, then we’d very likely want a different set of desiderata, at least one of which would need to cover some detectable failure at test time. We’d be happy to clarify this in section 3 if you think that would help.
• We provide full details of each of the benchmarks, including how group attributes are defined, in Appendix D. With the exception of Dollar Street, where we define groups as geographic provenance, all group information is exactly as specified by Yang et al. (2023).
• Our evaluation already includes methods that don’t rely on group attributes during training, such as Just Train Twice (JTT) and Learning from Failure (LFF).

Predictivity

• Thank you for the suggested terminology of predictivity and availability for section 3.4.1. We’re happy to update the paper to use these, and will cite Hermann et al. (2024) accordingly.
• To clarify, the distinction between points (2) and (3) in section 3.4.1 refer to the availability of the group attribute, a, and the availability of the target label, y. Both are essential factors controlling task difficulty due spurious correlation.

References

Yang et al. (2023). Change is Hard: A Closer Look at Subpopulation Shift. ICML. Hermann et al. (2023). On the foundations of shortcut learning. arXiv:2310.16228. Geirhos et al. (2020). Shortcut learning in deep neural networks. NMI.

Yes, I agree within the scope of datasets and methods considered the core of the paper is sound and useful. I think it's important that the scope of the paper (which is somewhat limited) is correctly indicated from the outset.

Thank you for your helpful review! We’ve uploaded a new version incorporating your feedback, and respond to each in turn below.

Re weakness 1

• In section 5, we essentially make two proposals. First, we suggest using the closest benchmark to the target task. Second, we suggest K as a possible measure for establishing which benchmark is closest.
• In our experiments using K to determine the closest benchmark, you are correct that this does require two full training runs. That said, we note that several mitigations methods already make use of multiple passes (e.g., JTT), yet are still widely-used by the community.
• However, this might be a good moment to consider the practical alternatives. When practitioners need to choose a mitigations method, we imagine they might use one of three possible strategies:
  1. Pick the method that performs best on average on previously-reported benchmarks.
  2. Sweep over all methods, including all hyperparameters, and pick the best method on the dataset at hand.
  3. Pick the method that performs best on the most relevant benchmark.
• Of these, option (1) clearly requires the least compute, though is often outperformed by improved benchmark selection methods (see table 2).
• Option (2) is naturally the most compute-intensive (i.e., sweeping a worst-case of 22 methods x necessary hyperparameters).
• Our approach, option (3), falls somewhere in the middle. While requiring two full training runs, it is less intensive than the commonly-used strategy of sweeping over all methods.
• Overall, while we agree our approach does come with a non-negligible compute cost, we consider it to be a reasonable compromise.
• More broadly, we see our analysis in section 5 are more of a “first step” (line 460), and remain excited to see how future researchers might interpret the notion of benchmark similarity.

Re weakness 2

• Thanks for your suggestion to better articulate how our paper differs from prior work.
• We discuss related work, including existing benchmarking efforts, in section 1.1, and explicitly discuss similarities with respect to other benchmarking efforts in lines 088–096. To respond specifically to your question about how our work differs from e.g., Joshi et al. (2023) and Yang et al. (2023), both works are concerned with benchmarking mitigations methods:
• Joshi et al. (2023) construct 3 variants of a semi-synthetic benchmark, SpuCo, and use these to evaluate 9 spurious correlations mitigations methods. A key conclusion is that existing methods struggle with more complex spurious correlations, such as where the correlated attribute has high variance.
• Yang et al. (2023) construct a benchmarking library, SubpopBench, allowing for the consistent evaluation of mitigations methods across several benchmarks. Yang et al. group benchmarkes into a proposed a taxonomy of subpopulation shifts.
• In contrast, our work has a different aim: to evaluate the validity of the benchmarks themselves. This effort is distinct from, and complementary to, existing efforts to develop benchmarks and frameworks, and we hope that our desiderata can help guide the development of challenging benchmarks and their translation into real-world problems.
• For example, our results suggest that, if using Yang et al.’s SubpopBench, one might consider including only the reduced set of valid benchmarks to ensure downstream utility.
• Alternatively, our experiments with closest benchmark selection could prove useful in choosing between the several variants of the SpuCo benchmark proposed by Joshi et al., particularly for practitioners when faced with a new dataset.
• Based on your feedback, we have updated the title of section 1.1 to be “Background and related work” and expanded our discussion of the focus of our work.

Re weakness 3

• Thank you for the comments regarding clarity. We have uploaded a new version addressing the errors you mention, reorganized the discussion to emphasize key themes, and expanded and retitled the background and related work section (see response to weakness 2). If you have further improvements you’d like to see, we’d be happy to implement them.

Re question 1

• Thank you for this important question.
• As you pointed out, a negative K suggests that a simple ERM model outperforms a reweighted model. This would be the spurious correlation is not a principal factor in task difficulty (as we discuss on lines 373–376).
• One alternative way of interpreting K is: “how far can simple reweighting get you?” For datasets where the spurious correlation is the dominant factor, reweighting should help. For datasets where other factors dominate, reweighting is likely ineffective, or even harmful overall (by reweighting uninformative samples, for example).
• Note that we also evaluate an alternative implementation of K, asking the question “how far can gDRO get you?”, in appendix F, and find that both implementations produce almost identical results.

Thank you for your considered and thoughtful review - we’re pleased you think that our paper addresses a critical gap, and that it could lead to improved benchmark selection practices. We’ve uploaded a new version addressing the typo you mention, and respond to your other comments below:

Re weaknesses

• Thank you for highlighting these papers. We are happy to add extended discussion of new advancements in the field that are not covered by the methods we evaluate, and we will explore the feasibility of adding these methods into section 4.

Re question 2

• You are correct that M_ERM was trained using Adam, but we don’t see these as conflicting. The model was trained to minimize the mean loss over the training samples (i.e., the empirical risk) using gradient-based optimization with an adaptive per-parameter learning rate (i.e., Adam). Whether using SGD, Adam, or RMSProp, as long as the gradient is w.r.t. the mean loss over the training set, we’d still consider this to be ERM. If you think this distinction needs further clarification, we’d be happy to update accordingly.

Re question 3

• Thank you for the wonderful question! The short answer is: yes, to a certain extent, we think model architecture should influence benchmark validity. To repeat the trivial example we use in the paper, if we were to choose a single-channel, grayscale, CNN architecture, then benchmarks where color is the spurious feature would be fairly useless.
• More broadly, a core motivation in introducing our measure of task difficulty due to spurious correlation, K, is the idea that the strength of the spurious correlation is necessarily dependent on the model perceiving the correlation. That said, in practice, we imagine that many large image classification models would produce similar K for each benchmark—as a result of similar training data, and only small differences in architecture—and would be unlikely to substantially change our conclusions re benchmark suitability.
• We chose the two model architectures considered, ResNet-50 and BERT base, as they remain the most frequent models considered in spurious correlations research, forming the typical basis upon which conclusions are drawn as to which methods perform best. As the research community continues to develop new models, repeatedly evaluating the utility of current benchmarks (particularly w.r.t. ERM Failure) is likely to be necessary.
• We think this might make a nice addition to our discussion section, and thank you again for raising this question!

Thank you Reviewers qYSy and t9iK for engaging in the discussion.

Please, Reviewers BsWk and fmNP read the authors' replies and feedback from other reviewers. If any concerns remain, request clarifications. This is your last chance to engage.

Thank you for your efforts.

Best regards, Area Chair