Enhancing Robustness of Last Layer Two-Stage Fair Model Corrections

We thank the reviewer for their thoughtful comments, especially their acknowledgment of the strength of our evaluation and method in general. We would like to address each question (Qx) and weakness (Wx) individually. Note that our references continue numbering from the review.

(W1) Regarding the baseline level of noise in the data, while it may be true that these datasets have inherent noise, the noise is consistent across all data splits, meaning that there is no distribution shift from training time to test time. Therefore RAD and SELF are likely not “robust,” they just fit to the distribution they are trained on. Further discussion about the limitations of current benchmarks is always warranted, though, and we thank the reviewer for their comment.

(W2/Q3) To clarify the noise model, we currently make no assumptions about the pretraining set (e.g., ImageNet) and assume that the full-finetuning dataset (e.g., training split of CelebA) is clean. While this assumption may seem strong, as we pointed out in section 3 of our paper and the discussion section, Iscen, et al. [3] find that embeddings are generally fairly robust to label noise, and only the final classification layer is strongly affected by noise. Furthermore, all last-layer retraining methods should be affected similarly by the quality of the embeddings. Still, we think it is important to examine this assumption more closely, and have run preliminary experiments to demonstrate that our method is robust to the violation of this assumption. Full results are shared in Tables 1-2 in the General rebuttal. We full finetune models for both CelebA and Waterbirds with 20% label noise and then use these embeddings to test the LLR methods. We see that both RAD and SELF perform very poorly in this scenario, though RAD seems much more resilient to poor-quality embeddings. As the noise increases in the finetuning set, RAD and SELF both decline quickly in WGA. Utilizing kNN label spreading, however, provides much-improved robustness, on par or better than gains we see in the clean-embeddings experiments. Thus, we conclude that our method is robust to violations (even major ones) of our “clean embeddings” assumption.

(W3) We focus on the misclassification variant of SELF because it requires the least side information about the unfair base model. While ES-SELF performs well on CivilComments, it requires access (as suggested in [2] and confirmed in their codebase) to early-stopped versions of the base-model to which we do not assume access. Additionally, there is no reason to believe that disagreement should solve the problem of noisy labels. Indeed, noisy points may be unnecessarily included in the error set for upweighting.

(Q1) Regarding the capitalization of $X$ in section 3, we intend the use of capitals to denote random variables and use this notation when discussing theory. For the algorithms, we use lowercase x to denote that we have been given a realization of the random variable $X$. We will ensure that this is clarified in the camera-ready version of the paper.

(Q2) Currently, the hyperparameter is selected using the clean, labeled validation set, but this requirement is not firm. Indeed, in our testing, we see strong agreement between the target predictive power of kNN on validation data and WGA on clean validation data (that is, the better we recover target labels, the better our downstream WGA is). Thus, while we currently use domain annotations to tune $k$, there is promise in proxy methods for tuning this important hyperparameter. We will make this point clear in the camera-ready version of the paper.

We hope that this rebuttal helps to assuage your concerns and highlights the important contributions of our method.

References

[3] A. Iscen, J. Valmadre, A. Arnab, and C. Schmid. Learning with neighbor consistency for noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4672–4681, 2022

Thanks to the authors for their comprehensive response.

For (W2), my concerns have been partially addressed, especially with the good performance of the kNN method using noisy finetuning datasets -- though I still have reservations about the practicality of the clean finetuning dataset assumption, since in practice one could just perform LLR/RAD/SELF using a held-out subset of the clean dataset. I look forward to reading more comprehensive experiments on noisy finetuning datasets in the final version.

For (W3), the authors bring up an interesting point about whether disagreement can solve the problem of noisy labels. In [2] the authors provide some discussion on this matter, as they claim misclassification selects "difficult" data (more likely to be noisy) while disagreement selects "uncertain" data (more likely to be legitimate minority group data). While somewhat orthogonal to the proposed kNN method, it would be valuable to the community to provide some discussion of this point, perhaps including experiments showing whether misclassification or disagreement methods are better at filtering noisy labels.

Regardless, though, I think some comparison to early-stop disagreement SELF should be included (and the authors can justify in the text that the numbers are not directly comparable due to the additional information of an early-stopped model), since [2] is clear that misclassification SELF is not the recommended method. In fact, they get much better performance on CivilComments by training on a random subset of data.

For (Q2), I suggest that the authors include model selection using worst-class accuracy [4] or the bias-unsupervised validation score [5] in the final version. These are recent proposals for proxy methods that do not use group annotations; they seem to perform well and are not difficult to implement.

Recommendation

Overall, this paper is borderline for me. The problem is important and the proposed method is interesting, but I have remaining concerns about the assumptions, evaluations, and comparisons as discussed above. With that said, I now lean slightly towards acceptance instead of rejection, so I have raised my score to a 5.

References

[2] LaBonte et al. “Towards Last-layer Retraining for Group Robustness with Fewer Annotations”. NeurIPS 2023.

[4] Yang et al. "Change is Hard: A Closer Look at Subpopulation Shift". ICML 2023.

[5] Tsirigotis et al. "Group Robust Classification Without Any Group Information". NeurIPS 2023.

We appreciate the reviewer’s thorough analysis of our submission and hope that we can answer some of the questions presented. We answer each question (Qx) and weakness (Wx):

(Q1/W3) Regarding the need for clean embeddings, as we point out in the discussion and in section 3 of our submission, Iscen, et al. [1] explicitly exploit the robustness of these embeddings to label noise in order to detect and correct outliers. To assess the downstream effect of noise in the embedding, we test our method on CelebA and Waterbirds using embeddings learned with label noise. We see that both RAD and SELF perform very poorly in this scenario, though RAD seems much more resilient to poor quality embeddings. As the noise increases in the finetuning set, RAD and SELF both decline quickly in WGA. Utilizing kNN label spreading, however, provides much improved robustness. Thus we conclude that our method is robust to violations of our “clean embeddings” assumption.

(Q2/W2) Regarding the selection of k nearest neighbors, it is possible to estimate the noise parameter from data as suggested in [2], but in practice k is relatively easy to tune through cross validation. On CelebA, we examine the performance of kNN-RAD with suboptimal k and see that selecting k larger than optimal results in a steady decline in WGA, but selecting k too small yields much worse downstream performance. This suggests that erring on the side of large k (10-20 in our tests) is generally better than too small. Even suboptimal performance (too large of k) results in vastly increased robustness over vanilla RAD (0% WGA). Full results are in Table 3 of the general rebuttal.

(Q3) The problem of label propagation in the multi-class setting is well studied, and the implicit geometry of multi-class classifiers is still amenable to propagation in the latent space. The noise model can be more complex in the multi-class setting, but if the noise flips to other classes completely at random, then our proposed method will still be effective. Additionally, both RAD and SELF have been shown to be effective in the multi-class setting. Due to time limitations, we do not have results on a multi-class dataset, but this certainly a good direction to explore.

(W1) Regarding additional theoretical analysis, our method is inspired by prior theoretical work which proves the robustness of kNN to label noise and combining this with empirically effective, but fragile, two-stage correction methods. We carry over the guarantees of robustness for kNN, but work remains to be done to understand the theoretical guarantees of two-stage corrections on their own. We are currently pursuing this direction, and we believe theoretical analysis for the interplay between kNN and two-stage corrections is promising future work. Experiments suggest that not only the separation of the classes makes a difference as suggested, but also the separation of subgroups within classes.

(W4) We appreciate the reviewer’s suggestion of additional label propagation techniques and believe that this could be an interesting path moving forward. Label propagation on the kNN graph is appealing because of its relative simplicity and the relationship between kNN and the downstream linear classification task in RAD and SELF. Indeed, as the number of nearest neighbors grows, the classification boundary becomes more and more (locally) linear, which in turn aids robustness of the downstream LLR model learned for fairness correction. On some datasets (e.g., CMNIST), this linearity induced by large k may be undesirable, and another graph structure or label propagation method may be more appropriate. The joint design of fairness correction and label correction in this manner is a compelling area of future research.

(W5) Regarding comparison to robust loss functions, in our preliminary exploration, we believed that these methods may have promise, but our experimental results were poor. Fundamentally, the objective of a robust loss is to learn a classifier on noisy data that predicts well on clean data. This means that noisy data is (correctly) misclassified by these models and, thus, if used in the first stage of RAD or SELF, would promote the inclusion of these noisy points into the error set. This, in turn, assigns more weight to noisy examples in the final retraining step, thereby dramatically reducing performance and fairness. Alternatively, label propagation corrects these labels explicitly before training, which prevents noisy points from dominating the error set.

We hope we have adequately addressed the presented concerns and demonstrated the significance of our contribution.

References

[1] A. Iscen, J. Valmadre, A. Arnab, and C. Schmid. Learning with neighbor consistency for noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4672–4681, 2022

[2] Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., and Qu, L. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.

We appreciate the reviewer’s considered response to our submission. We would like to politely push back on a few points and answer the reviewer’s questions (Qx) and weaknesses (Wx) in turn. Note that our references continue numbering from the review.

(Q1/W1) Regarding the need for clean embeddings, as we point out in the discussion and in section 3 of our submission, Iscen, et al. [3] explicitly exploit the robustness of these embeddings to label noise in order to detect and correct outliers. To assess the downstream effect of noise in the embedding, we test our method on CelebA and Waterbirds using embeddings learned with 20% label noise. See Tables 1-2 in the general response for full results. We see that both RAD and SELF perform very poorly in this scenario, though RAD seems much more resilient to poor quality embeddings. As the noise increases in the finetuning set, RAD and SELF both decline quickly in WGA. Utilizing kNN label spreading, however, provides much improved robustness. Thus we conclude that our method is robust to violations of our “clean embeddings” assumption.

(Q1) Regarding the performance of the RAD and SELF using only the clean holdout, we thank the reviewer for their suggestion of a new baseline. While RAD and SELF are generally sample efficient, SELF specifically downsamples the error set and so will likely suffer more in this setting. Regardless, we have run a preliminary experiment in this direction: we restrict our finetuning set to the clean holdout (used for validating our method) and train RAD and SELF in the usual manner. On CelebA, RAD achieves 74 $\pm$ 9.92 and SELF achieves 80.89 $\pm$ 0. This is beaten by kNN-RAD up to 30% SLN, suggesting that there is more information to be gained by utilizing the noisy embeddings. On Waterbirds, RAD achieves 83.4 $\pm$ 6.35 and SELF achieves 60.57 $\pm$ 12.29. Here using a kNN method is superior at every tested noise level. These tests use half the available data (we assume the validation split is clean), but this could conceivably be reduced by different hyperparameter tuning strategies. In that case, we expect the gap between kNN-corrected methods and vanilla two-stage methods to grow.

(W3) We appreciate the reviewer’s suggestion of [1], a very recent work on unsupervised concept discovery. While their method has the advantage of reduced reliance on holdout annotations, it appears significantly more computationally intensive, involving training two embedding networks to identify concepts. The aim of our paper is to leverage existing embeddings to train a fair classifier with limited data, thus increasing the reuse of powerful pre-trained models. We believe, however, that these concepts could be used to improve future last layer retraining (LLR) methods.

(Q2) The multi-class, multi-bias setting is a very interesting one and how well shortcuts (unintended decision rules that are unable to generalize, i.e., spurious features) can be mitigated with access only to the LLR tools without access to jointly optimizing the embeddings is not fully clear. For LLR-only methods, indeed, approaches such as DFR [4] and other related LLR methods explicitly optimize to reduce the reliance of the pretrained model on multiple shortcuts which have already been learned by the embeddings. Unfortunately, due to time constraints we are not able to run experiments with these additional datasets, but these datasets present meaningful benchmarks for future investigation.

We once again thank the reviewer for many valuable suggestions, and we hope that we were able to address the reviewer’s concerns sufficiently and demonstrate the valuable contribution of our method, especially when clean data is limited.

References

[3] A. Iscen, J. Valmadre, A. Arnab, and C. Schmid. Learning with neighbor consistency for noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4672–4681, 2022

[4] P. Kirichenko, P. Izmailov, and A. G. Wilson. Last layer re-training is sufficient for robustness to spurious correlations. In The Eleventh International Conference on Learning Representations, 2023.

We are very grateful to the reviewer for their valuable suggestions, especially regarding new baselines. We believe that we were able to address the reviewer’s concerns and hope that it warrants an increase in score.

Thank you to the authors for addressing my questions. The authors presented preliminary experiments regarding my concern about the clean data assumption, where they claim their method shows improvement over RAD and SELF. However, it is unclear why RAD and SELF perform differently on CelebA and Waterbirds, and the significance of their improvement when other methods are trained on similarly clean data. I recommend finishing these experiments and including them in the paper, along with a detailed discussion of the sample complexity of each experiment. Considering the importance of the problem studied and the promising results, I have decided to increase my score.