The Invariance Starvation Hypothesis

• This work suffers from significant issues in its approach, evaluations, and conclusions. Overall, I do not find the results presented to justify the large claims scattered throughout the paper.
• In its current form, the paper argues that our conception of why models use spurious correlations, rather than learning invariant features, is incorrect, principally relying on results from simple reasoning tasks. However, given that the results on CelebA and MultiNLI are in line with current knowledge (i.e., scaling training samples is not helpful for reducing worst-group error), then I am more inclined to ask why the reasoning results differ. Do they suggest an error, an edge case, or something fundamentally different? At present, without more systematic evaluation, it is impossible to tell.
• The authors attempt to suggest that reasoning tasks differ because of their simplicity (e.g., "We observe that in reasoning tasks, the input space is comprised of only a small number of objects, thereby making all samples, and by extension all general features, highly typical"; lines 069--071). This is highly likely to be a property of the synthetic tasks (LEGO, PVR) tasks considered, rather than a general property of reasoning. Indeed, reasoning itself is often a core component of solving other tasks in language and vision, such as in performing natural language inference (e.g., MNLI). If, as the authors suggest, that increasing the number of samples is sufficient to overcome spurious correlations in reasoning tasks, then why does this approach not also help with MNLI?
• More broadly, to support such big claims, the evaluations are too limited. If the authors wish to make general claims about differences between reasoning-based tasks and tasks from other domains, then they will need to evaluate multiple benchmarks in each category to prove the generality of their results. I would suggest, at the very least, expanding their analysis to consider other well-studied benchmarks in the worst-group optimization space, such as Waterbirds for vision and CivilComments for language.
• Fig 7a (blue line) shows almost no effect of random sampling on worst-group accuracy when increasing scale. This directly contradicts the paper's claim about model reliance on spurious correlations being exacerbated with scale on vision and language tasks.
• The claim "Selection Bias does not form Spurious Correlations" is not supported by the evidence. At best, the authors could claim that "selection bias is not the sole cause of spurious correlations."
• In section 5.3, the authors state that "the number of new samples without spurious feature is a lot greater than the number of new samples with the spurious feature" (lines 353--354). This statement is misleading, as it ignores the fact that absence of the spuriously-correlated feature can also be predictive of the target label in the tasks considered.
• The paper contradicts itself in the abstract and the introduction. On the one hand, the authors wish to indicate dichotomized behaviour between simple/complex reasoning/non-reasoning tasks. Yet, at the same time, the authors wish to suggest their results are consistent across domains. For example, lines 025--026 state "We observe the same results in settings ... such as vision and language," while lines 027--028 state "we find that in such settings, drawing more samples from the training distribution ... can exacerbate spurious correlations." Both of these cannot simultaneously be true.
• The paper uses inaccurate language such as "formation of spurious correlations" and "selection bias does not form spurious correlations". Spurious correlations are a property of the data; the subject of this work is whether models learn to use them. There are also grammatical errors and unnecessary capitalizations (e.g. lines 054-055) throughout.
• Minor: Predictive power does not simply refer to which proportion of the training set contains the spurious feature (lines 040-041). Trivially, one could have a feature present in 99% of samples that is highly noisy, and therefore has low predictivity.
• Minor: Past works do not "simply state" that neural networks are biased towards using simpler spurious correlations - they present empirical and theoretical evidence that this is the case. The authors should avoid misrepresenting the state of the literature.

• The main claim of this work, presented in line 471, "Our paper is the first to show that models are not always biased to simpler features. We show that if the network is provided with enough data to encode the general, invariant function well, it will ignore the simpler, weakly predictive spurious features." is only sustained with a simple, synthetic experiment. It consists of a reasoning problem (a deterministic problem) but it is immediately invalidated once brought to a more complex distribution (a stochastic problem) on a Computer Vision and NLI problem. The paper states a strong claim on simple, deterministic settings, and later invalidates said claim on stochastic problems, but still reaffirms the claim as a novel contribution. The argument is clearly flawed and requires more work. Why is it not working on stochastic processes? Can we really "rectify" this issue by employing typical samples as suggested on section 5.4? The true value of this paper is on these questions but they are not sufficiently developed.
• This final claim on section 5.4 is not sufficiently proved either, as it is only evaluated on two datasets for one particular spurious feature each. Besides, no error bars are presented to determine if the observed effects are truly significant. Further testing of this claim is required to ensure the validity of the claim—can we extend the initial claim to stochastic, more complex and realistic settings?—, but as it is, the paper does not offer an answer.
• Experiment figures are not clearly explained. Figure 3: what is the shifted dataset? What are the different positions in a) and b)? Even if it can be understood by context, it is not clearly stated.
• The presentation is a bit difficult to follow because of the atypical titles in sections. I would recommend to use succinct titles for sections instead.

1. The paper lacks a comprehensive review of existing work on spurious correlation mitigation, leaving out relevant recent studies. Including key papers (see Questions for some examples), would better contextualize this study’s contributions.
2. The study lacks a comparison with baseline algorithms, making it difficult to assess how well the proposed method performs relative to existing techniques for mitigating spurious correlations. Including baselines from prior work on invariant feature learning and spurious correlations would strengthen the study by providing a clearer performance benchmark.
3. The study lacks clarity regarding the design and configuration of tasks, particularly around the application of rules and dataset modifications. This lack of transparency makes the study difficult to reproduce and weakens the quality of the experimental design.
4. Many claims are unsupported by references or evidence, reducing the study's credibility. The observations drawn from experiments are not always substantiated by the figures referred to. Strengthening the argument with relevant references and ensuring experimental conclusions are well-supported by visual data would enhance the clarity and rigor of the paper.
5. Although the authors propose a method to mitigate spurious correlations, the resulting worst-group accuracy remains low both in absolute terms and in comparison to state-of-the-art methods. This suggests that spurious correlations may not be fully mitigated and raises the possibility that the method is primarily filtering out noisy examples rather than addressing spurious correlations directly. Conducting ablation studies could clarify the source of observed improvements, helping to isolate the effects of the proposed method and determine whether it genuinely mitigates spurious correlations or if other factors contribute to the results.

Does this observation invariantly occur when spurious correlation is strong? Is there a possibility that this paper's claim of "starving network will ignore the spurious features with enough data to encode the general, invariant function" is only valid when such correlation is so weak that simple drawing more data could overcome it? Regarding this,

1. This paper lacks the detailed description of the severity of correlation, and the systematic analysis (evaluation) on how invariant the observation is across different severity. Thus, additional evaluation on simply controllable dataset, e.g., ColoredMNIST, should be included.

2. How efficient and effective is simply drawing more data for debiasing, against existing debiasing strategy [1,2,3]? Does simply adding more data outperform other more complicated debiasing method?

Overall, I believe the crucial factor, severity of bias, is missing in this paper while it might significantly determine the observation in preventing models from learning debiased features.

[1] Bahng et al., Learning De-biased Representations with Biased Representations, ICML 2020
[2] Nam et al., Learning from Failure: Training Debiased Classifier from Biased Classifier, NeurIPS 2020
[3] Lee and Kim et al., Learning Debiased Representation via Disentangled Feature Augmentation, NeurIPS 2021

• The results are not really surprising. It is known that over-parameterization can exacerbate spurious correlation [1]. In other words, having more data, which reduces over-parameterization, can help overcome spurious correlation. There is no real difference between having more/fewer parameters (as in [1]) and having fewer/more data (as in this work). Moreover, [1] has also shown that the opposite can happen too, i.e., fewer parameters (or more data) sometimes exacerbate spurious correlation. This is similar to the second main empirical observation of the paper presented in Section 5.
• The explanation for why more data contribute to spurious correlation reliance is not well justified. The paper attributes this phenomenon to the complexity of the data distribution but lacks good arguments to back it up. The explanation essentially says that data sampled from complex distributions are of high variance and thus it is difficult for a network to capture the invariant features from the data. However, this still does not explain why adding more data makes it even harder for the network to learn correctly.
• The empirical evidence presented in the paper is not compelling. First of all, one would expect more extensive and rigorous experiments to support general claims like “invariance starvation hypothesis”. The experiments only involve two synthetic reasoning tasks, one vision task, and one language task. They are conducted on only a very limited amount of different architectures and hyperparameter settings. Second, there are too many variables between the two sets of experiments (reasoning vs. vision & language) to clearly demonstrate the impact of the complexity of the data distribution.
• Many experiment details are unclear. Were all the networks trained until convergence? What is the training/mean accuracy of the models corresponding to the reported test/worst-group accuracy (in Figure 6)? Lines 314-316 suggest that some hyperparameter tuning was performed, but this seems to contradict the training details (Section 5.2) which involve a fixed set of hyperparameters. How exactly were the hyperparameters tuned? Was there any early stop based on validation accuracy?

[1] Sagawa, Shiori, et al. "An investigation of why overparameterization exacerbates spurious correlations." International Conference on Machine Learning. PMLR, 2020.