The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations

We graciously thank Reviewer ax3f for their detailed comments, questions, and references. We appreciate that the reviewer recognizes our contributions to the understanding of class-balancing methods, robustness impact of scaling pretrained models, and spectral analysis of model representations. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.

Weakness 1, Question 1, Question 2

We thank the reviewer for the insightful comments. Our intention was to avoid a comprehensive discussion of last-layer retraining methods, as we exclusively focus on model finetuning without held-out data or group labels. Our results are not directly comparable to those of DFR [1] and AFR [2] as they train on held-out data (up to 40K additional points) not included in standard model finetuning, as well as group labels (for training in [1] and model selection in [2] -- see the global response for mixture balancing model selection without group labels). On the other hand, since models finetuned with class-balancing are used as initializations for last-layer retraining [3], we expect our deeper understanding of finetuning phenomena to influence more sophisticated robustness algorithms.

Our key reason for discussion of DFR [1] in Section 3.2 is not to compare to last-layer retraining, but instead to compare our mixture balancing method to their technique of averaging the last layer weights over multiple independently-sampled class-balanced data subsets, which like our proposed method intends to increase exposure to majority group samples without over-sampling the minority group. To our knowledge, this is the only technique in the literature which attempts to balance this trade-off. We did not cite AFR [2] in Section 3.2 because they do not mention this averaging technique, instead using a method akin to upweighting, and therefore make no attempt to increase exposure to majority class samples without over-sampling the minority class. Their approach of upweighting points on which the model does poorly is closely related to Just Train Twice (JTT) [4] and orthogonal to our methods. We will clarify these points in the paper.

With that said, we agree with the reviewer that a more comprehensive discussion of where our mixture balancing method lies within the broader context of group robustness methods is appropriate. Please see the global response for further discussion.

Weakness 2, Question 3

We thank the reviewer for the detailed questions. While we considered including a simple theoretical analysis of subsetting and upsampling in class-balancing, a preliminary investigation revealed some interesting connections to the literature which we believe merit their own, purely theoretical submission in the future. In particular, we observed that while upsampling and upweighting have similar WGA dynamics, both differ greatly from subsetting, in contrast to [5] which proved a broader theoretical equivalence between the three techniques in the population setting. We believe theoretically investigating this discrepancy is an interesting direction, but it requires the development of some new techniques to analyze WGA dynamics when under-represented data is seen repeatedly by the model during training, which would be out of scope for this paper. We will add further technical discussion in our updated version.

For the reviewer’s request for further investigation of class-balancing methods, we investigated an additional balancing technique (upweighting), proposed model selection without group labels, and performed additional experiments/ablations across two additional model families. Please see the global response for results and analysis.

For the reviewer’s request of connecting spectral analysis to class-balancing, we re-ran our spectral analysis for Waterbirds with all balancing methods from the paper. Overall, we found that the magnitude of the eigenvalues is significantly affected by the chosen class-balancing method. However, the relative ordering of minority/majority group eigenvalues is consistent across class-balancing techniques. We note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class-balancing may bring about additional stability in the representation. Please see the global response PDF for figures.

Finally, we did not discuss Spuriosity Rankings [6] because it is focused on explainability/interpretability methods for discovering spurious features, which is orthogonal to our contributions. If the reviewer has a specific comparison they would like us to make with [6], we would be happy to address their concerns during the discussion phase.

I would like to thank the authors for their detailed response. To acknowledge that some concerns have been addressed, I will update my score.

We graciously thank Reviewer E5Mf for their insightful comments and questions. We appreciate that the reviewer recognizes our comprehensive experiments which challenge existing notions in the literature and leverage previously unknown nuances to improve model performance. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.

Weakness 1, Question 3

We appreciate the suggestion to replicate our experiments with other popular pretrained model families. We have implemented these experiments in two settings: Swin Transformer [10], a more efficient variant of Vision Transformer [11] with three pretrained sizes available, and ResNet [12], the most popular convolutional backbone with five pretrained sizes available. Overall, we find our results are consistent across pretrained model families, with the model affecting the raw accuracies but typically not the relative performance of class-balancing techniques or scaling behavior. Please see the global response PDF for figures.

Weakness 2, Question 1

We appreciate the reviewer’s comments and we agree that we could be more explicit about our recommendations for practitioners. Below, we provide more practical recommendations based on the “nuances” we propose in our paper.

1. Since group proportions are unknown in practice, a good proxy for which class-balancing technique to use is the size of one’s dataset. If the dataset is on the smaller side (a couple thousand points, like Waterbirds), one should use upsampling or mixture balancing. If the dataset size is larger than 100K (like CivilComments), subsetting or mixture balancing is recommended.

2. With appropriate pretraining and class-balancing, model scaling is generally beneficial for group robustness. Therefore, practitioners should utilize the largest model appropriate for their use case (especially if it does not interpolate, like MultiNLI).

3. Practitioners should typically apply a post-hoc group robustness technique like AFR [2] or SELF [3] after finetuning, since as seen in the global response, class-balancing can only push WGA so far. With that said, the performance of the initial finetuned model affects downstream WGA (especially due to representation quality for last-layer methods [8]), so it is still worthwhile to carefully select model pretraining parameters.

We would appreciate it if you could point out the general prior beliefs you refer to. The literature appears to reflect a lack of consensus even with respect to phenomena as seemingly simple as model scaling. For example, in our related work section, we discuss conclusions from the literature which state that model scaling could either be beneficial or detrimental to robustness performance, depending on the particular setting and assumptions used for analysis.

In the global response, we address the reviewer’s request for additional experiments/ablations by investigating an additional balancing technique (upweighting), proposing model selection without group labels, and performing additional experiments/ablations across two additional model families. If there is a specific ablation the reviewer wishes to see that has not been addressed in the global response, please let us know and we would be happy to address it during the discussion phase.

Weakness 4

We agree with the reviewer that a more comprehensive discussion of where our mixture balancing method lies within the broader context of group robustness methods is appropriate. We would like to clarify that, in contrast to Group DRO [9], our methods do not use group labels in the training dataset (they are discarded before training). Please see the global response for a more explicit comparison with Group DRO and other methods, where we clearly delineate if held-out data or group labels are utilized.

Furthermore, while performing model selection with respect to worst-group accuracy is a common assumption in the literature [1, 4, 8, 9], it is nevertheless unrealistic when group labels are not available. We investigate several methods for model selection and conclude that our methods are robust to validation that does not use group labels (indeed, validation with respect to worst-class accuracy is sufficient). Please see the global response for results and analysis.

Weakness 5, Question 2

We appreciate the reviewer’s suggestion and we agree that it would be interesting to observe the behavior of our methods on multi-class datasets. However, the dataset Spawrious [15] suggested by the reviewer is class-balanced a priori and hence not suitable for evaluating class-balancing methods (similarly to MultiNLI). With that said, we were able to run model scaling experiments. We used the most rigorous O2O Hard split and include the results below:

Despite the trend being less monotone than other datasets, it is clear that scaling pretrained and class-balanced models is broadly beneficial for robustness on Spawrious, especially as one nears the 100M parameters mark.

We anticipate that our class-balancing methods would generalize to multi-class datasets without additional nuance, since the failure modes we identify for each method remain the same no matter the number of classes. Specifically: (1) for upsampling, any small class will be over-represented during long training runs no matter the number of classes, so we expect overfitting to the minority group within this class and poor WGA as a result; and (2) for subsetting, any minority group within a large class will be severely downsampled no matter the number of classes, so subsetting will disproportionately harm that group.

We warmly thank Reviewer kRBT for their detailed comments and questions. We appreciate that the reviewer recognizes the improved performance of our mixture balancing method and the independent interest of our model scaling and spectral analysis experiments. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.

Weakness 1, Question 3

We appreciate the reviewer’s request for further empirical evidence and evaluation settings. In the global response, we investigate an additional balancing technique (upweighting), propose model selection without group labels, and perform additional experiments/ablations across two additional model families. Please see the global response for results and analysis.

Weakness 2

For the reviewer’s request of connecting spectral analysis to class-balancing, we re-ran our spectral analysis for Waterbirds with all balancing methods from the paper. Overall, we found that the magnitude of the eigenvalues is significantly affected by the chosen class-balancing method. However, the relative ordering of minority/majority group eigenvalues is consistent across class-balancing techniques.

We note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class-balancing may bring about additional stability in the representation. Please see the global response PDF for figures.

Weakness 3, Question 1

We agree with the reviewer that while performing model selection with respect to worst-group accuracy is a common assumption in the literature [1, 4, 8, 9], it is nevertheless unrealistic when group labels are not available. In the global response, we address these concerns by investigating several methods for model selection without group labels -- using 3-4 mixture ratios per dataset -- and conclude that our methods are robust to validation without group labels (indeed, validation with respect to worst-class accuracy is sufficient). Please see the global response for results and analysis.

Question 2

We thank the reviewer for the question and we address it in the global response. In addition to subsetting, upsampling, and mixture balancing, we have investigated another common class-balancing technique called upweighting not studied by [7]. In this method, minority class samples are directly upweighted in the loss function by the class-imbalance ratio, and it is used by state-of-the-art group robustness algorithms including AFR [2]. We find that upweighting experiences a similar catastrophic collapse as upsampling, even though they are only equivalent on average over the upsampling probabilities (i.e., not in practice). Please see the global response PDF for figures.

We warmly thank Reviewer Rwfv for their comments and suggestions. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.

Weakness 1

We thank the reviewer for the relevant comments. While robustness performance indeed differs across datasets, our explicit goal is to elucidate these nuances, analyze their causes and effects, and provide the community with a rigorous foundation for further research. Furthermore, we do find a potentially generalizable pattern for our class-balancing results. Specifically, we investigate two new failure modes of class-balancing which depend on the particular group/class structure of each dataset: (1) mini-batch upsampling and loss upweighting experience catastrophic collapse with standard hyperparameters when imbalance is large, and (2) removing data to create a class-balanced subset can harm WGA when a small minority group is present in the majority class.

We further investigate the generalizability of our results in the global response. In particular, our observations remain consistent even when tested on new class-balancing methods, without group labels for model selection, and in new settings including two different model families.

Finally, we remark that case-by-case descriptions of experimental outcomes have strong precedent in the community. For example, Izmailov et al. [8] investigate the case-by-case effects of architecture, pretraining dataset, and regularization on multiple benchmark datasets. Similarly to us, they analyze how their results vary across different datasets and leverage their insights to improve model performance and make practical recommendations.

Weakness 2

We believe the reviewer is referencing the fact that we utilized alphabetical/lastname citations instead of numerical citations. We respectfully refer the reviewer to line 150 in the NeurIPS 2024 Formatting Instructions, which states “Any choice of citation style is acceptable as long as you are consistent.” If the reviewer was referencing some other aspect of our citations, please feel free to clarify the specific concerns, and we would be happy to address them during the discussion phase.

Weakness 3, Weakness 4

We thank the reviewer for the suggestions and we will include them in our updated version.

Thank you for your detailed response. I would like to further elaborate on my concerns.

Q1. Inconsistencies in Claims: There appears to be a contradiction within the manuscript regarding the effectiveness of subsetting in the presence of a small minority group within the majority class. Specifically, in Lines 157-159, the authors state that subsetting improves WGA when there is a small minority group within the majority class. However, in Figure 1, the results for the Waterbirds dataset show that subsetting performs worse than no class-balancing. Additionally, in Lines 188-189, the authors mention that "subsetting can be effective except when there is a small minority group present in the majority class," which directly contradicts the earlier statement. Could you clarify which of these claims is correct?

Q2. Lack of Controlled Variables: The experimental setup seems to lack control over various factors, making it difficult to draw logical or theoretical conclusions from the results. For instance, regarding the claim made in Question 1, is it the case that the authors derived the conclusion from the results shown in Figure 1? In that case, there is a concern that the proposed reason for the difference in performance observed in the Waterbirds dataset compared to CelebA and CivilComments might not be a generalizable finding but rather a result of simply choosing one dataset out of several with inherent differences. For instance, unlike CelebA and CivilComments, the Waterbirds dataset shows lower performance for subsetting, which could be attributed to the significantly smaller training set size (Waterbirds: 4,795 samples; CelebA: 162,770 samples; CivilComments: 269,038 samples). The smaller size of the Waterbirds dataset could lead to a substantial decrease in overall performance when further reduced by subsetting, which also can explain the observed results. Additionally, the degree of group imbalance within classes is not as significant in CelebA and CivilComments, which distinguishes them from the Waterbirds dataset and could be a factor influencing WGA. These factors, if uncontrolled, could have influenced the results, and I wonder if they were considered in the analysis. How can we distinguish between the interpretation of the paper and these alternative explanations? Considering this, I find it challenging to fully agree with the statement, which underpins the paper, that "removing data to create a class-balanced subset can harm WGA when a small minority group is present in the majority class.

Q3. Dataset Suitability and Analysis: Given that the Waterbirds dataset shows high WGA for no class-balancing methods and the minimal difference compared to upsampling, could it be that this dataset is less sensitive to the degree of imbalance in the training set and therefore not well-suited for analyzing the behavior of class-balancing methods? If Waterbirds is excluded, it seems that conducting case studies on class balancing methods with only the CelebA and CivilComments datasets might explore only a limited range of cases.

Q4. When comparing the performance of upsampling, evaluating it by epoch might not provide a fair comparison, as upsampling leads to more iterations. Wouldn't it be more rigorous to compare the results based on the number of iterations with the same batch size?

Q5. (minor question) I understand why ConvNext-V2 was chosen for the experiments in Section 4, but wouldn't it be beneficial for the field to also include results using ResNet50 for the other experiments? Additionally, I’m curious whether the results in Figures 1 and 2 would be consistent if ResNet50 were used.

Thank you for your continued engagement. We provide responses to your questions below.

Question 1

We appreciate the catch; we agree that our sentence in lines 158-159 was unclear. We will rewrite the sentence to say “…it can in fact improve WGA conditional on the lack of a small minority group within the minority class.” Thus, the claim is consistent with Figure 1 and lines 188-189.

Question 2

We agree with the reviewer that a more controlled investigation would be beneficial. To show that our conclusions hold in a controlled environment, we extend the synthetic dataset of [Sagawa et al., 2020: An investigation of why overparameterization exacerbates spurious correlations] to our class-imbalanced setting.

We generate a dataset of $100000$ points with labels $y\in \\{-1,1\\}$ and spurious attributes $s\in\\{-1,1\\}$ as follows. Each $(y, s)$ group has its own distribution over input features $x = [x_{core},x_{spu}]\in\mathbb{R}^{2d}$, corresponding to core features in $\mathbb{R}^d$ generated from the label $y$ and spurious features in $\mathbb{R}^d$ generated from the spurious attribute $s$: $x_{core}\sim \mathcal{N}(y\vec{1}, \sigma^2_{core} \mathbf{I}\_d)\qquad x_{spu}\sim \mathcal{N}(s\vec{1}, \sigma^2_{spu} \mathbf{I}_d).$

We set $d=100$, $\sigma^2_{core}=100$, and $\sigma^2_{spu}=1$ following [Sagawa et al., 2020]. Different from their setup, we generate the data according to a class-imbalanced distribution where a small minority group is present within the majority class. To do so, we introduce a variable $\alpha\in [0.5, 1.0)$ which controls the size of the minority group within the majority class. Our dataset composition is detailed below:

We train a two-layer ReLU neural network with hidden dimension $64$ using full-batch gradient descent with learning rate $0.01$ and momentum $0.9$. We class-balance with the subsetting method and evaluate on a held-out dataset of $100000$ points balanced across groups. Our WGA results are detailed below, averaged over ten seeds:

As seen in the table, the smaller the size of the minority group in the majority class (i.e., the larger $\alpha$ is), the worse the subsetting method performance becomes. We believe this contributes to the justification of our conclusions in a controlled environment, and we would be happy to discuss further if the reviewer has additional concerns.

Question 3

We appreciate the reviewer’s concern. We believe the sensitivity to the degree of imbalance is not just a property of the dataset, but of the model as well. In Figure 3 in the global response PDF, we show that Swin Transformer Base exhibits a larger degree of sensitivity on Waterbirds compared to ConvNeXt-V2. Below, we include a table with a more explicit comparison (WGA averaged over 3 seeds):

Contrary to the reviewer’s concern that “the Waterbirds dataset shows high WGA for no class-balancing methods and the minimal difference compared to upsampling,” the suboptimality of no-class balancing and upsampling on Waterbirds is clearly observed for Swin Transformer Base. Together with the strong precedent in the literature of evaluating class-balancing methods on Waterbirds [3, 7], we believe Waterbirds is well-suited for our investigation.

Question 4

We would like to clarify that upsampling does not lead to more iterations compared to training without class-balancing. In upsampling, we sample the mini-batches uniformly over the classes, but each mini-batch contains the same amount of data and we train for the same amount of steps as without class-balancing. We will make this more explicit in the paper.

Question 5

We agree that these results will be important for the community. In Figures 2 and 3 in the global response PDF, we replicated our class-balancing and scaling experiments for different class-balancing methods using the ResNet family (including ResNet50) and Swin Transformer. We will include more comprehensive ResNet50 results in the updated version of the paper.