Severing Spurious Correlations with Data Pruning

"Scaling up the difficulty of the core features of the 100 samples."

Apologies for the confusion. In the CIFAR-10S experiments in Section 5, we take 100 samples to which we add synthetic spurious features and we keep changing the 100 samples like a 100-sample-sized sliding window to probe into. By scaling up the difficulty of the core features of the 100 samples, we simply increase the difficulty of the 100 samples into which we introduce the spurious features. We first sort the samples by difficulty. For results in Figure 2, we simply introduce spurious features in the easiest 100 and the hardest 100 samples. For results in Figure 3, we consider all unique 100 sample subsets by sliding a window of sample size 100 across the sorted samples list. Since there are 5000 total samples in the class, we have 50 different subsets in which spurious features are introduced. We will be more than happy to provide any further explanation.

As we increase the difficulty of the 100 samples into which we introduce spurious features, we see that spurious feature reliance exhibits polynomial growth instead of linear growth with increasing difficulty.

What do the authors mean by “group labels” in line 457?

We sincerely apologize for not including a definition of group labels. We follow the literature concerning spurious features/correlations, where group labels commonly refer to labels that indicate the presence or absence of spurious features in each training sample within a class.

Thus, within a class, samples with the spurious feature would have a group label = 1 and samples without the spurious feature would have a group label = 0. Note that this is different and independent from class labels in classification tasks. We make this point clearer in the paper. In identifiable settings (as in the existing literature) where the strength of the spurious signal is relatively stronger, it is trivial to identify group labels, as is shown in Section 3 (Figure 1 (b) (Right)).

Can the authors provide a brief and self contained description of the point that wants to be raised in this section (while I believe this is not the central part of the work)?

Our work is primarily interested in novel settings where spurious signals are weak, in that the signals are not easily identifiable, as you have described in sentence 2 of your review.

However, in Section 6.3, we show that even in previously studied settings where the spurious signals are “strong” (identifiable settings), simply pruning a few samples can yield state-of-the-art results. While not the primary focus of our paper, we believe this finding is very important. Current techniques that attain good performance in these settings are extremely complex and computationally expensive. Our method is very simple to understand, takes only a few additional lines of code, is easy to reproduce and yields state-of-the-art performances, even on benchmarks that do not fit into the primary objectives of this paper.

Please note that Waterbirds and MultiNLI are two of the most commonly studied benchmarks in literature concerning spurious correlations for Vision and Language tasks (Sagawa et. al. 2020 ICML, Kirichenko et. al. 2023 ICLR, Liu et. al. (ICML, 2021), Zhang et. al. (ICML, 2022), Ye et. al. 2023 AISTATS). Additionally, we highlight the robustness of our pruning technique by showing that pruning sparsities within a wide range can attain state-of-the-art or competitive performance on these benchmarks (Figure 8).

We have improved the writing of this section thanks to your suggestions and questions. Please let us know if you have any concerns and we will be happy to address them.

(Liu et. al. ICML, 2021) “Just Train Twice: Improving Group Robustness without Training Group Information,” ICML, 2021.

(Zhang et. al. ICML, 2022) “Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations,” ICML, 2022.

We thank the reviewer for their comments. We are happy to hear that they found our work to be very well written and glad that they enjoyed reading our work. We have addressed their comments below:

It is sometimes not obvious what the authors mean by "strong" or "weak". While providing precise definitions is beyond the purposes of this work, some examples during the introduction could facilitate the reading. For example, I was confused in the paragraph at line 201, where the strength of the signal is defined both in terms of the geometry of the pattern and its frequency in the data. These two aspects are fundamentally different, and putting them altogether might not result in the best model to investigate this problem...

Thank you for this comment. The strength of the spurious signal indicates the ease of learning the spurious feature. When the strength of the spurious signal is strong, spurious features are learned easily and spurious correlations are formed easily. In all literature concerning spurious correlations, the strength of the spurious signal is primarily determined by the following three factors: (1) Proportion (or Frequency) of training samples containing the spurious feature (Sagawa et. al. 2020 ICML, Shah et. al. NeurIPS 2020, Kirichenko et. al. 2023 ICLR), (2) Area Occupied and Position (if it is centered or not) in the training sample (Moayeri et. al. 2022 NeurIPS) and (3) The amount of noise in the signal (Sagawa et. al. 2020 ICML, Ye et. al. 2023 AISTATS). A feature which is present in a large portion of all training samples, occupies a lot of area, is centered, and has little to no variance, has a very strong signal. On the other hand, a feature which is present in a small portion of all training samples, occupies little area, and has a lot of noise/variance, has a very weak signal. For instance, in Section 2, to reduce the strength of the spurious signal, we reduce the proportion of training samples that contain the spurious feature. We have added this to lines 128-136 of the revised text.

We ensure that both aspects (geometry and frequency) are not modified in the same experiment. So for instance, in Section 5, the strength of the spurious signal for CIFAR-10S is varied by only changing the geometry of the spurious feature. In Section 6, the strength of the spurious signal for CIFAR-10S is varied by modifying the proportion of samples containing the presence of the spurious feature. We show that these experiments can be replicated by performing the other modification. Below, we present the results in Section 6 (Figure 5) for CIFAR-10S across three seeds by doing both: varying the geometry (or increasing the area occupied compared to the unidentifiable setting) and varying the proportion (increasing the proportion/frequency of samples containing the spurious feature compared to the unidentifiable setting):

Identifiable Setting distribution by varying geometry (Spurious Samples only):

Q1(Easiest): 53.2%, Q2: 24.26%, Q3: 16.13%, Q4 (Hardest): 6.39%

Identifiable Setting distribution by varying proportion (Spurious Samples only, already in the paper):

Q1(Easiest): 49.93%, Q2: 38.68%, Q3: 8.78%, Q4 (Hardest): 2.6%

Unidentifiable Setting distribution (Spurious Samples only, already in the paper):

Q1(Easiest): 30.93%, Q2: 22.26%, Q3: 23.73%, Q4 (Hardest): 23.06%

We observe that in both Identifiable settings, Q1 contains most of the samples with spurious features while Q4 contains few samples with spurious samples.

In other words, these modifications are interchangeable as they ultimately influence the same property: The strength of the spurious signal.

(Shah et. al. 2020 NeurIPS) “The Pitfalls of Simplicity Bias in Neural Networks,” NeurIPS, 2020.

(Sagawa et. al. 2020 ICML) “An Investigation of Why Overparameterization Exacerbates Spurious Correlations,” ICML, 2020.

(Kirichenko et. al. 2023 ICLR) “Last Layer Re-training is Sufficient for Robustness to Spurious Correlations,” ICLR, 2023.

(Moayeri et. al. 2022 NeurIPS) “Hard imagenet: Segmentations for objects with strong spurious cues”, NeurIPS, 2022.

(Ye et. al. 2023 AISTATS) “Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise”, AISTATS, 2023.

There should be a typo in line 162 "as shown in..."

Thank you for pointing this out. We agree that there is a typo in line 162 and we fix the reference.

We thank the reviewer for their thoughtful comment. We agree with their statement and have made the required changes to the revised text. We believe their comments and feedback have helped improve the quality of our work.

We would also like to thank them for encouraging another peer reviewer, Reviewer 7BQb, to reconsider their score considering our work’s value. We hope that this discussion will lead to fair assessment of our work. Thank you again.

We thank the reviewer for their comments. We have addressed all your concerns by providing all relevant definitions, detailed figure captions, technical details for the proposed approach and additional training details. Note that in addition to providing additional training details, we have also provided our code through an anonymous github repository to ensure easy reproducibility (Link: https://github.com/Anon-ICLRDP/ICLR_DP). We have included all these components in our paper but we provide them below for your convenience:

Key Concept Definitions:

Lines 82-88 in revised text: Consistent with past literature, we study the supervised classification setting where $S = {\{(x_i, y_i)}\}^N_{i=1}$ denotes the training dataset of size $N$ and network is trained to learn a mapping between $x_i$ (input) and $y_i$ (class label) using empirical risk minimization (Vapnik, 1998). Every training sample $s \in S$ contains a core feature ($c_i$) that represents its class ($y_i$). A fraction of all samples within a class contain the spurious feature $(a_i)$ associated with that class. Core (or invariant) features are causal to the class label $y_i$ and are fully predictive of the task, as they are present in all samples. Spurious features are not causal to the class labels and are partially predictive of the task, as they are present in only a fraction of all samples of a class.

Past literature has found that during training, deep networks choose to rely on spurious features over core/invariant features if the spurious features are easier to learn than the core features. They form a correlation between these spurious features and ground truth labels. Such correlations are called spurious correlations.

Lines 90-92 in revised text:

Spurious Correlations: The correlation a network forms between spurious features and class labels. Such correlations are undesirable as they are not causal to class labels and can disappear during testing or become associated with a different task, causing these networks to malfunction.

Lines 128-136 in revised text:

Feature Difficulty: Consistent with deep learning literature (specifically, those works concerned with spurious correlations), difficulty of learning a feature is determined by the following three factors: (1) Proportion (or Frequency) of training samples containing the spurious feature (Sagawa et. al. 2020 ICML, Shah et. al. NeurIPS 2020, Kirichenko et. al. 2023 ICLR), (2) Area Occupied and Position (if it is centered or not) in the training sample (Moayeri et. al. 2022 NeurIPS) and (3) The amount of noise in the signal (Sagawa et. al. 2020 ICML, Ye et. al. 2023 AISTATS). A feature which is present in a large portion of all training samples, occupies a lot of area, is centered, and has little to no variance, is easy to learn. On the other hand, a feature which is present in a small portion of all training samples, occupies little area, is not centered, and has a lot of noise/variance, is hard to learn.

(Shah et. al. 2020 NeurIPS) “The Pitfalls of Simplicity Bias in Neural Networks,” NeurIPS, 2020.

(Sagawa et. al. 2020 ICML) “An Investigation of Why Overparameterization Exacerbates Spurious Correlations,” ICML, 2020.

(Kirichenko et. al. 2023 ICLR) “Last Layer Re-training is Sufficient for Robustness to Spurious Correlations,” ICLR, 2023.

(Moayeri et. al. 2022 NeurIPS) “Hard imagenet: Segmentations for objects with strong spurious cues”, NeurIPS, 2022.

(Ye et. al. 2023 AISTATS) “Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise”, AISTATS, 2023.

Problem Statement and Pseudocode:

Lines 201-202 in revised text:

Problem Statement: How does one sever spurious correlations in settings where attaining spurious information is difficult or impossible?

Lines 290-294 in revised text:

Pseudocode:

1. Train the network on the task for n epochs, where n << t and t is the total number of epochs in the training schedule.
2. Compute sample-wise difficulty scores as $|| p(w,x) - y||_2$ , where $p(w,x)$ is the probability distribution given by the network for sample $x$, $w$ denotes the network parameters after the nth epoch and $y$ is the one hot encoding of the ground truth value.
3. Prune samples with high difficulty scores.
4. Train a new network on the pruned dataset for t epochs.

Clarifying Figure 2: Please note that we have expanded on the figure caption and mentioned figure titles in the main text for better readability: (Lines 174-177 in text, with additional references in Section 5). For the reviewer, we provide a simple explanation below:

The figures “Easiest” refers to injecting synthetic spurious features in the easiest 100 samples while the figures “Hardest” refers to injecting synthetic spurious features in the hardest 100 samples.

Explanation of evaluation metrics: Reflecting on your comment, we added a more detailed explanation of the evaluation metrics used in our paper. Additional sentences: Lines 261-264 in text.

Current practice in deep learning utilizes Worst-Group Accuracy (WGA) to assess the degree of spurious feature reliance in binary classification tasks. WGA computes the accuracy of test samples that contain the spurious feature associated with a different class during training. While suitable for simple binary classification tasks, WGA becomes insufficient to assess the reliance on spurious features in settings with multiple classes. This is because WGA cannot differentiate between loss in test accuracy due to spurious correlations, or due to lack of learnability of invariant correlations stemming from limited capacity or insufficient training data. In such settings, we measure the degree of spurious feature reliance through Spurious Misclassifications, i.e. the percentage of samples of one class (c1) containing the spurious feature of another class (c2) that are misclassified as (c2) during testing. Lower Worst Group Accuracy indicates heavy reliance on spurious correlations while high worst group accuracy indicates little to no reliance on spurious correlations. A high number of Spurious Misclassifications indicates heavy reliance on spurious correlations while a low number Spurious Misclassifications indicate little to no reliance on spurious correlations.

Experimental Details:

All experimental details are provided in the Appendix (Sections A.1 and A.2). We have already provided the models used and included all baseline methods in our evaluation. We provide additional experimental details within the same sections.

CIFAR-10S. We use the ResNet20 implementation from Liu et. al. (ICLR, 2019) that we train for 160 epochs. The network is optimized using SGD with an initial learning rate 1e-1 and weight decay 1e-4. The learning rate drops to 1e-2 and 1e-3 at epochs 80 and 120 respectively. We maintain a batch size of 64. Sample difficulty is computed after the 10th epoch.

CelebA. We use an ImageNet pre-trained ResNet-50 from PyTorch Paszke et. al. (Neurips, 2019) that we train for 25 epochs. The network is optimized using SGD with a static learning rate 1e-3 and weight decay 1e-4. We maintain a batch size of 64. Sample difficulty is computed after the 10th epoch.

Hard Image-Net. We use an ImageNet pre-trained ResNet-50 from PyTorch Paszke et. al. (Neurips, 2019) that we train for 50 epochs. The network is optimized using SGD with a static learning rate 1e-3 and weight decay 1e-4. We maintain a batch size of 128. Sample difficulty is computed after the 1st epoch.

Waterbirds. We use an ImageNet pre-trained ResNet-50 from PyTorch Paszke et. al. (Neurips, 2019) that we train for 100 epochs. The network is optimized using SGD with a static learning rate 1e-3 and weight decay 1e-3. We maintain a batch size of 128. Sample difficulty is computed after the 1st epoch.

MultiNLI. We use a pre-trained BERT model that we train for 20 epochs. The network is optimized using AdamW using a linearly decaying starting learning rate 2e-5. We maintain a batch size of 32. Sample difficulty is computed after the 5th epoch.

Additional Experimental Details:

CIFAR-10S. We follow a similar approach to Nagarajan et. al. (ICLR, 2021) for adding a spurious line where pixel values for a vertical row of pixels in the middle of the first input channel are set to the maximum possible value (255) before normalization and before any augmentations. We use the same augmentations used for training on the original CIFAR-10.

CelebA. In this setting, we maintain 5000 Female Samples without Eyeglasses and 2500 Male samples with Eyeglasses and 2500 Male samples without Eyeglasses. Consistent with the implementation in Sagawa et. al. (ICLR, 2019), Liu et. al. (ICML, 2021), we do not use any augmentations.

Hard ImageNet. In this setting, we maintain 58 Dog Sled samples with minimal spurious features and 100 Ski samples randomly drawn from the dataset. All remaining classes are maintained the same. We use the same augmentations used for training on ImageNet.

Waterbirds. We use the original Waterbirds setting commonly used in practice (Sagawa et. al. (ICLR, 2019), Liu et. al. (ICML, 2021), Zhang et. al. (ICML, 2022), Kirichenko et. al. (ICLR, 2023)). We use the augmentations used in Kirichenko et. al. (ICLR, 2023) when training, which are similar to the augmentations used for training on ImageNet.

MultiNLI. We use the original MultiNLI setting used in practice (Sagawa et. al. (ICLR, 2019), Liu et. al. (ICML, 2021), Kirichenko et. al. (ICLR, 2023)). Consistent with the implementation in Sagawa et. al. (ICLR, 2019), Liu et. al. (ICML, 2021), Kirichenko et. al. (ICLR, 2023)), we do not use any augmentations.

(Nagarajan et. al. ICLR, 2021) “Understanding the failure modes of out-of-distribution generalization,” ICLR, 2021.

(Shah et. al. 2020 NeurIPS) “The Pitfalls of Simplicity Bias in Neural Networks,” NeurIPS, 2020.

(Sagawa et. al. 2020 ICML) “An Investigation of Why Overparameterization Exacerbates Spurious Correlations,” ICML, 2020.

(Sagawa et. al. 2019 ICLR) “Distributionally Robust Neural Networks For Group Shifts: On the Importance of Regularization for Worst-Case Generalization,” ICLR, 2019

(Kirichenko et. al. 2023 ICLR) “Last Layer Re-training is Sufficient for Robustness to Spurious Correlations,” ICLR, 2023.

Liu et. al. (ICML, 2021) “Just Train Twice: Improving Group Robustness without Training Group Information,” ICML, 2021.

Zhang et. al. (ICML, 2022) “Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations,” ICML, 2022.

(3) Can authors clarify the rationale behind "the presence of strong spurious information enables the network to understand samples with hard core features better"?

By the statement highlighted by the reviewer, we mean that samples with hard core features incur a low training error as the network relies on strong spurious features to reduce training error for that sample. Thanks to your comment, we think this sentence might create future confusion and thus, rephrase it in the revised text. “The presence of strong spurious information enables the network to have lower training error for samples with hard core features + spurious features.”

To support the statement, we kindly refer the reviewer to Figure 5, where in the presence of strong spurious information (identifiable settings), most samples with spurious features have low training error. Additionally, to further reinforce the validity of this claim, we ran the following experiment:

We perform the same experiment in Section 4, where we first compute the training error of all samples when trained on clean CIFAR-10. We then introduce the spurious feature S3 into 100 samples with the highest training error, which gives us the results in Figure 2 (c) and (d). We additionally compare the average training error of the same 100 samples with and without the spurious features across three seeds:

Without spurious feature S3: 1.30

With spurious feature S3: 0.68

We observe that by simply introducing strong spurious features, samples with hard core features (those that incur a high training error initially) no longer have a high training error.

(Sagawa et. al. 2020) “An Investigation of Why Overparameterization Exacerbates Spurious Correlations,” ICML, 2020.

(Sagawa et. al. 2019 ICLR) “Distributionally Robust Neural Networks For Group Shifts: On the Importance of Regularization for Worst-Case Generalization,” ICLR, 2019.

(Kirichenko et. al. 2023) “Last Layer Re-training is Sufficient for Robustness to Spurious Correlations,” ICLR, 2023.

(Sohoni et. al. 2020) “No subclass left behind: Fine-grained robustness in coarse-grained classification problems,” NeurIPS 2020.

(Liu et. al. ICML, 2021) “Just Train Twice: Improving Group Robustness without Training Group Information,” ICML, 2021.

(Zhang et. al. ICML, 2022) “Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations,” ICML, 2022.

(Ahmed et. al. ICLR 2021) “Systematic generalisation with group invariant predictions,” ICLR 2021.

(Creager et. al. 2021) “Environment inference for invariant learning,” ICML 2021.

(Yang 2024) “Identifying spurious biases early in training through the lens of simplicity bias,” AISTATS 2024.

(Deng et. al. 2023) “Robust learning with progressive data expansion against spurious correlation,” NeurIPS 2023.

(4) Suppose a sample has a hard invariant feature and an easy spurious feature, should it be easy or hard to learn (small or large training error)? My understanding is that the sample diffculty is estimated by the training error, in this case, it is unclear how to identify "spurious samples containing the hardest core features".

When a sample has a hard invariant feature and an easy spurious feature, it becomes easier to learn (low training error) than when it only has the invariant feature. In identifiable settings (which we understand is the setting you are referring to), samples with hard core features + spurious features become easier to learn than samples without spurious features. This is why we cannot simply prune the hardest samples in the identifiable settings (Lines 456 - 457).

In identifiable settings, however, it is very easy to identify which samples contain spurious features. In other words, it is very easy to identify group labels, by clustering for instance, as explained in question 2. In such settings, since we are aware of which samples contain spurious features, we prune those samples with spurious features that have a higher training error. These would include samples containing easy spurious features + hard core features versus those that contain samples containing easy spurious features + easy core features.

We would like to re-iterate that finding group labels in identifiable settings is trivial and we follow other seminal works that directly use group labels to mitigate spurious correlations. Note that we also compare our results with techniques that directly use group labels to mitigate spurious correlations.

We will be more than happy to address any further concerns that the reviewer might have. Thank you for taking the time and effort to go through the revised version of our paper and suggesting thoughtful comments to improve the quality of the paper.

We thank the reviewer for their thoughts and comments.

We would like to clarify that in our responses, we only state that we vary frequency and area alone. If the reviewer has concerns regarding these factors, we refer them to papers (already cited in our text, Lines 127 - 135) that study their role in the strength of (spurious/core) features - Sagawa et. al. 2020 ICML, Shah et. al. 2020 NeurIPS, Moayeri et. al. 2022 NeurIPS, etc. We simply follow them. There are additional factors that can influence the strength of a feature that we do not vary/modify in our experiments. In the definition provided in the text, we also mention noise in the signal as a factor that influences the strength of a feature. To the best of our knowledge, current literature studies only these three factors extensively and this is why we only include these in our definitions.

Most works in spurious correlations consider the simple setting where samples belonging to a class contain one core feature and may contain one spurious feature associated with that class (Sagawa et. al. 2020 ICML, Liu et. al. 2021 ICML, Zhang et. al. 2022 ICML, to list a few). In Vision settings, features are described as objects. In Language settings, features are described as words or sentences.

The definitions we provide are consistent with what is currently known and accepted in the literature. We do not claim to encompass all possible factors and it is for this reason that we do not provide numeric values regarding the strengths of features in the 5 settings studied. We clarify that this is beyond the scope of this work.

As most works in spurious correlations literature, we start with settings where spurious correlations are formed (identifiable or unidentifiable/novel) and propose solutions to tackle them, while providing novel insights and attaining SOTA on previously studied settings.

(Sagawa et. al. 2020 ICML) “An Investigation of Why Overparameterization Exacerbates Spurious Correlations,” ICML, 2020.

(Shah et. al. 2020 NeurIPS) “The Pitfalls of Simplicity Bias in Neural Networks,” NeurIPS, 2020.

(Moayeri et. al. 2022 NeurIPS) “Hard imagenet: Segmentations for objects with strong spurious cues”, NeurIPS, 2022.

(Liu et. al. ICML, 2021) “Just Train Twice: Improving Group Robustness without Training Group Information,” ICML, 2021.

(Zhang et. al. ICML, 2022) “Correct-n-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations,” ICML, 2022.

Kindly let us know if there are any remaining concerns and we will be happy to address them.

Dear Reviewer 7BQb,

We hope our latest responses have helped address your concern. Kindly let us know if your concern still remains.

Thank you.

We thank the reviewer for their comments. We address their concerns below:

Weakness 1: The paper could explore more thoroughly the practicality of applying the proposed pruning method to large datasets. Specifically, even though the proposed method shows promise for datasets of moderate size, an assessment of its computational cost and efficiency on large and real-world data would be beneficial.

We thank the reviewer for their comment. We would like to emphasize that while efficiency was not the primary objective of our work, data pruning is an efficient solution compared to other techniques as one only does standard training on a pruned (thus, smaller) dataset. This is in contrast to other techniques that do sample upweighting or representational alignment, which significantly increases the training time over just standard training. Additionally, we emphasize that Hard ImageNet, Waterbirds, MultiNLI, and CelebA studied in our paper are real-world datasets with realistic/real-world spurious features, and we intentionally utilized CIFAR-10S to better understand how varying the strength of (synthetic) spurious features impacts generalizability and training distribution (Lines 219-222, Lines 301-317, Figure 5(a).) Regarding the matter of scale, ImageNet is the only dataset that contains samples on a scale that is different from CIFAR-10 but it is not studied in the context of spurious correlations. To the best of our knowledge, there is no dataset at that scale that is studied in the context of spurious correlations.

Weakness 2: The approach relies on assessing sample difficulty as a proxy for contribution to spurious correlations. Clarifying the robustness of this metric under different training regimes (e.g., varied architectures or optimization strategies) could strengthen the generalizability of the findings.

We show that the computation of these ranks and subsequent pruning are robust even across architectures and different optimization strategies/hyperparameters. Below, we present the worst Group Accuracy for the CelebA Setting (Figure 4) for different architectures and hyperparameters for the same Prune Percentage (%). We will include these results in the revised text:

ResNet18:

VGG16:

Learning Rate = 0.01:

Learning Rate = 0.0001:

Weight Decay = 0.001:

Weight Decay = 0.01:

Weakness 3: Although the paper discusses state-of-the-art methods, further comparative analysis with recently emerging pruning and robust training techniques that do not rely on explicit spurious feature identification would be helpful.

To the best of our knowledge, there do not exist any promising pruning or robust training techniques that do not rely on explicit spurious feature identifications. We will be happy to compare our approach with techniques based on the reviewer’s recommendation.

Question 1: How sensitive is the pruning method to changes in hyperparameters? Question 2: The robustness of sample difficulty estimation across different model architectures is not clear at the current version of the paper. Would it be possible to add some results or explanations for that?

Current experiments are run on ResNet-50 and BERT, which is a transformer-based model. We also include experimental results on VGGNets, smaller ResNets and different hyperparameters in response to Weakness 2.

Question 3: Table 1 shows that some SOTa methods achieve better results. Could you please give some explanation on that? Also, could you please open source the code to enhance the reproducibility?

Only one existing SOTA method achieves better results (Worst-Group Accuracy) on only one dataset (MultiNLI). While we do not have an explanation for this, it is important to note that results on identifiable benchmarks are not the primary focus of the paper. The primary focus of our paper is on novel settings where spurious information is unidentifiable. Additionally, current techniques that attain good performance in these settings are extremely complex and computationally expensive. Our method is very simple to understand and implement - requiring only a few additional lines of code, is easy to reproduce, and yields state-of-the-art performance, even on benchmarks that do not fit into the primary objectives of this paper.

Sure, to enhance reproducibility, we have added an anonymized repository for the experimental results presented in this paper, and we will open-source it upon this paper’s acceptance. Kindly let us know if you have any difficulties executing this code and reproducing the results. Link: https://github.com/Anon-ICLRDP/ICLR_DP

Question 4: Suggestions for Improvement:

1) It would be helpful to include a section that discusses the potential negative impacts or limitations of the method, such as the risk of pruning samples that are informative but rare.

We thank the reviewer for this suggestion. We believe that it is important to include such limitations. However, in our extensive empirical evaluation across multiple different datasets, we see little to no reduction in testing accuracy when pruning these key samples because they generally do not contribute much to generalizability (Lines 366 - 368, Figure 4 (Right), Figure 6 (Right), Figure 7 (Right), Table 1 (Mean Accuracy)). If one were to continue to prune more of the harder samples, test accuracy would drop (Sorscher et. al., 2022 NeurIPS). However, based on our current observations, it is clear that the amount of data needed to be pruned to severe spurious correlations is less than the amount of data needed to observe noticeable drops in test accuracy. We will be happy to perform any further analysis to assess the potential impacts or limitations of the method based on the reviewer’s recommendation.

(Sorscher et. al., 2022 NeurIPS) Sorscher, Ben, et al. "Beyond neural scaling laws: beating power law scaling via data pruning.”, 2022 NeurIPS.

2) Extending the empirical analysis to more complex and real-world datasets with non-synthetic spurious correlations could further validate the applicability of the method.

We would like to emphasize that across the 5 datasets that we study (HardImageNet, CelebA, Waterbirds, MultiNLI and CIFAR-10S), 4 of them are real-world datasets containing real-world spurious features and correlations. We intentionally included one dataset containing synthetic spurious features (CIFAR-10S) as it allows us to vary the strength of the spurious feature relative to the core, invariant feature. All of the findings from the one synthetic dataset translate perfectly to the four real-world datasets.

To the best of our knowledge, there do not exist any datasets utilized in literature for spurious correlations that could offer additional or better insights. However, we will be more than happy to include results from any datasets that the reviewer would suggest that we include in our analysis.

Dear Reviewer eCbW,

It would be great if we could hear back from you so that we can know if we have addressed all of your concerns.

Thank you.

Question 4: Are these justifications architecture/data dependent?

No, the justifications are not architecture/data dependent. Our original experiments are conducted on ResNet-50 and Transformer based architectures like BERT. We test on 5 different datasets with different numbers of classes, input dimensions, difficulties, and spurious features. To further reinforce our empirical findings, we show that we obtain similar results with VGG16 and smaller ResNets like ResNet18. We also show that our results on the CelebA setting are consistent across different hyperparameters such as different learning rates and weight decays. We will include these results in the appendix.

Worst Group Accuracy of Figure 4 for different architectures/hyperparameters for the same Prune Percentage (%):

ResNet18:

VGG16:

Learning Rate = 0.01:

Learning Rate = 0.0001:

Weight Decay = 0.001:

Weight Decay = 0.01:

Question 5: While the authors demonstrate that prunning a particular type of subset of data points will reduce spurious correlations, there is no discussion of what are the consequences. I am inclined to beleive that throwing out data is going to have some sort of negative consequence, and therefore it important to know what trade-off is being made. This will allow a better comparison to other methods that do not simply prune data as well as enable practitioners to understand/determine if the cost of pruning data is worth the benefit in removing spurious correlations.

We thank the reviewer for this suggestion. We also believe that it is important to include such limitations. However, in our extensive empirical evaluation across five different datasets with different numbers of classes, input dimensions, difficulties, and spurious features, we see little to no reduction in testing accuracy when pruning these key samples because they generally do not contribute much to generalizability (Lines 366 - 368, Figure 4 (Right), Figure 6 (Right), Figure 7 (Right), Table 1 (Mean Accuracy)). If one were to continue to prune more of the harder samples, test accuracy will drop (Sorscher et. al., 2022 NeurIPS). However, based on our current observations, it is evident that the amount of data needed to be pruned to severe spurious correlations is less than the amount of data needed to observe noticeable drops in test accuracy. We will be happy to perform any further analysis to assess the potential impacts or limitations of the method based on the reviewer’s recommendation.

(Sorscher et. al., 2022 NeurIPS) "Beyond neural scaling laws: beating power law scaling via data pruning.”, 2022 NeurIPS.

We thank the reviewer for their comments and detailed review. We are happy to hear that our work represents your favorite kind of work and that our paper is written well with compelling experiments.

Before we address your questions, we would like to emphasize that the scope of our paper is to provide empirical evidence regarding the gaps in literature, novel insights regarding the behavior of deep neural networks in the presence of spurious correlations and the effectiveness of our proposed solution. To do so, we make sure to cover most benchmarks generally studied in spurious correlations literature (identifiable settings) while introducing new ones. To the best of our knowledge, there do not exist other benchmarks that may offer new insights but we will be more than happy to verify our claims on any other benchmarks based on your recommendation.

Question 1: Is the particular type of spurious correlation the authors consider representative of all spurious correlations?

Thank you for this question. The spurious correlations we study are consistent with all previous literature that studied spurious correlations, where spurious features are easier to learn than core features and are not fully predictive of the task. In settings where this does not hold (so for instance, settings where the spurious features are harder to learn than the core features), the network will choose to ignore these features (Shah et. al. 2020 NeurIPS) and so the spurious correlations present in the dataset are not learned. In other words, we are primarily concerned with settings where the network relies on spurious correlations present in the dataset and exclude those settings where spurious correlations are present in the dataset but are not learned. The latter has never been studied in deep learning literature. We are unaware of any other spurious correlations previously studied.

Within the setting studied, we define two types of spurious correlations. One where the strength of the spurious signal is significantly greater than the core signal (identifiable) and another where the strength of the spurious signal is relatively weaker (unidentifiable, novel).

Additionally, we would like to point out that not all benchmarks studied in this paper have spurious features that are over-represented in a certain class. In Section 2, we show that spurious correlations are formed even when 10% (or 50%) of all samples within a class contain the spurious feature.

If the reviewer's concern was with respect to the kinds of spurious features studied, we would like to emphasize that our experiments consider benchmarks where spurious features are backgrounds (snow, water and land backgrounds), objects or words (eyeglasses, trees, negation words, etc.) or synthetic lines through which we believe we covered almost all commonly used benchmarks regarding spurious features.

(Shah et. al. 2020 NeurIPS) “The Pitfalls of Simplicity Bias in Neural Networks,” NeurIPS, 2020.

Question 2: Does the distributional uniformality of samples containing features with spurious correlation across the training samples generalize across types of spurious correlations?

The answer is no. In Section 6.1, we show that in settings where the strength of the spurious signal is significantly greater than the strength of the core features (Identifiable Settings), samples containing spurious features are no longer uniformly distributed (Figure 5). The key attribute which determines the distribution is the strength of the spurious signal in the training set. In settings where the strength of the signal is significantly greater, however, it is trivial to identify the presence of spurious features in training samples.

Question 3: Is the pruning of this particular type of subset as a solution to (any type of) spurious correlations?

Yes. We are unaware of any other type of spurious correlation learned by deep neural networks and we have covered almost all existing benchmarks studied in literature. However, we are happy to verify our claims on any other benchmarks based on your recommendation.

I apologize to the authors and the review team for jumping into this discussion period a little later than others, I had an unexpectedly sick infant at home this week.

We thank the reviewer for their comments. We are glad that they like our work very much, believe that the observations are exciting and that our paper is really interesting. It is encouraging to see these comments. We try our best to address their follow-up concerns below:

I would instead recommend following the definition used in Singhla and Feizi, 2022, as it seems to often be cited by others in the authors' literature, as they seem to work to precisely codify the concept, even for their MTurkers.

We thank the reviewer for this comment. We would like to clearly note that our settings cover both Vision and Language, so we cannot directly use the definition from Singhla and Feizi, 2022, because their definition is valid only for Vision settings (directly defined on visual features and objects). Also, please note that Singhla and Feizi, 2022 state that core attributes are the set of visual features that are always a part of the object definition. Additionally, in Liu et. al. 2021, there are no core features that are backgrounds. The only background features they consider are present in the Waterbirds task (Land/Water background) and these are spurious features, not core features. The core feature associated with each class is present in all samples of that class (e.g., all samples belonging to the Landbirds class contain landbirds (core feature) in them. Importantly, not all of the Landbird samples contain land backgrounds (spurious feature associated with that class). Some samples contain water backgrounds. (Section B (B.1) in the appendix). However, we are happy to alter the definitions to incorporate your comments as shown below:

Core (or invariant) features represent the class label $y_i$ and are semantically relevant to the task. They are also fully predictive of the task, as they are present in all samples. Spurious features do not represent the class labels and are semantically irrelevant to the task.

We simply use the definitions provided by Izmailov et. al. 2022 NeurIPS and Kirichenko et. al. 2023 ICLR. We make these changes to the revised text.

Izmailov et. al. 2022 NeurIPS, On Feature Learning in the Presence of Spurious Correlations

Kirichenko et. al. 2023 ICLR, Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations

While we soften our claims below, we would like to point out that our empirical analysis covers 5 diverse datasets and different architectures. Additionally, we emphasize that the papers the reviewer references (Liu et al. (2021), Kirichenko et. al. (2023), Moayeri et al. (2023)) propose solutions and insights with empirical evidence alone. However, subsequent works then used their insights to create theories and techniques (e.g. Ye et. al. 2023 AISTATS). Not only does our work make use of the benchmarks mentioned in these works, but we also propose new settings where these techniques are not suitable. We believe pushing the boundary on what is currently known is a good step toward covering all spurious correlations in the future or creating a better understanding of the behavior of deep networks. For instance, the papers cited above do not cover all spurious correlations, as we show that their methods are not suitable in the proposed settings. However, the insights and contributions of these papers were critical in moving the field forward. While we do not prove our claims theoretically, we strongly believe that the insights and contributions of our paper have their own novel value and are important for future research.

That being said, we are happy to soften the claims by making changes to the contributions in the following manner.

Changes in contributions listed in the paper:

Contribution 2: We discover that spurious correlations are formed primarily due to a handful of all the samples containing spurious features through extensive empirical investigation. Based on this insight, we propose a simple and novel data pruning technique that identifies and prunes a small subset of the data that contains these samples.

Changes in contributions in Global Response to the Review Team:

Contribution 2: Discovering that spurious correlations are primarily formed from a handful of all samples containing the spurious features through extensive empirical investigation.

Contribution 3: Proposing a novel data pruning solution that severs spurious correlations in the proposed novel settings while attaining state-of-the-art performances on previously studied settings.

Ye et. al. 2023 AISTATS “Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise”, AISTATS, 2023.

We are happy to address any further concerns the reviewer may have. Thank you for your time and effort in improving our paper and thank you for your enthusiasm towards our paper.

We thank the reviewer for their comments.

Clarifying Figures 4 and 6:

In Fig. 4, we only prune those samples that contain spurious features and hard core features. This is to make the point that samples with hard core features + spurious features are the primary contributors to spurious feature reliance. Through Fig 6, we propose the final data pruning algorithm which prunes all the samples with hard core features (which contains samples with hard core features + spurious features and also sample with hard core features + no spurious features, Lines 435 - 438), as spurious information is unidentifiable. We have added additional text to make this clearer on Line 436.

Presenting the same insights with textual data:

Insights 1 and 2:

Samples with simple core features do not contribute to spurious correlations. Samples with hard core features are primary contributors to spurious correlations.

To show this, we perform the same experiment in Section 4, but instead of CIFAR-10S, we use the MultiNLI dataset. First, we remove all samples with negation words from the training data and then we compute the sample-wise difficulty scores as we do for CIFAR-10S in Section 4. We then create two settings: one where we introduce the spurious negation word “never” at the end of the 100 hardest input samples belonging to class 1 (contradicts) and another where we introduce the spurious negation word “never” at the end of the 100 easiest input samples belonging to class 1 (contradicts). We do the same to a set of test samples belonging to class 2 (neutral with) and class 3 (entailed by).

Consistent with the standard MultiNLI setting, we measure the degree of spurious feature reliance through Worst Group Accuracy (accuracy of the set of test samples of class 2 or class 3 with the spurious feature).

We observe that WGA is significantly worse when the word “never” occurs in the hardest samples vs. the easiest samples during training.

Introducing spurious feature in easiest 100 samples: WGA = 55.22%

Introducing spurious feature in hardest 100 samples: WGA = 1.04%

A higher WGA indicates low reliance on spurious features.

The gap in worst group accuracy is 54.18%. Note that the number of samples containing the spurious feature is the same in both settings (= 100).

Additionally, we note that there are 191,504 training samples in this setting. There are 57,498 samples belonging to the contradicts class. We introduce the spurious feature in only 100 samples of the contradicts class (0.17% of samples within the class, 0.0522% of all samples in the training set.) We also observe that in a setting with no spurious features during training, Worst Group Accuracy is 67.42%.

Simply varying which 100 samples contain the spurious negation word “never” has such a huge impact on Worst Group Accuracy. This finding is extremely insightful, novel and is consistent with the results observed in Section 4 (Figure 2) of our paper.

This experiment reinforces the claim that samples with hard core features are primary contributors to spurious correlations and that samples with simple core features do not contribute to spurious correlations.

Insight 3:

Excluding a few key samples during training severs spurious correlations.

For this, we simply show the results in Figure 4 but for the MultiNLI dataset. Note: Due to computational constraints, we only show three pruning sparsities.

Worst Group Accuracy:

Kindly note that the model attains 65.9% Worst Group Accuracy on the original, unpruned dataset.

We refer the Reviewer to Fig. 10 in the appendix for a better understanding.

Insight 4:

Spurious feature strength created a specific distribution of the training data.

To show this, we use a smaller subset of the MultiNLI dataset and vary the strength of the spurious signal by varying the proportion of samples containing the spurious feature.

Distribution of samples with spurious features in Identifiable setting:

Distribution of samples with spurious features in Unidentifiable setting:

To confirm that the unidentifiable setting still causes the network to rely on spurious correlations, we create another setting where we remove all samples with spurious features and compare the Worst Group Accuracies below:

Unidentifiable setting: 64.89%

No samples with spurious features setting: 70.73%

We observe that in the setting with no samples containing the spurious features, Worst Group Accuracy is higher, indicating that the unidentifiable setting still causes the network to rely on spurious correlations but the samples containing spurious features are uniformly distributed.

Additional Results:

Distribution of the MultiNLI setting covered in the paper (Identifiable) (Figure 5(b) extension).

Distribution of samples with spurious features:

Q1 in this setting contains almost half of all samples with spurious features while Q4 only contains 15%. This shows that samples with spurious features are not uniformly distributed when viewed through the lens of sample difficulty. This is in contrast to the CelebA setting (Unidentifiable), in which samples with spurious features are uniformly distributed (Figure 5(b) right in text).

We refer the Reviewer to Fig. 11 in the appendix for a better understanding.

Kindly let us know if there are any additional concerns that we can address which can help increase our score. We greatly appreciate your detailed comments that have helped improve our paper. Thank you again for taking the time to review our work.

Dear Reviewers,

As we near the end of the discussion period, this is your last chance to engage.

Please review the authors' replies and feedback from other reviewers. If any concerns remain, request clarifications. The authors have one more day to respond before the reviewer-only discussion week begins.

Thank you for your efforts.

Best regards, Area Chair