Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness
We propose a framework to mitigate simplicity bias in neural networks to encourage the use of a diverse set of features, leading to improved subgroup robustness, out-of-distribution generalization and fairness.
摘要
评审与讨论
The paper presents a new method to avoid the "simplicity bias" of neural networks, and learn more diverse features. This leads to models with better OOD generalization.
优点
-
Simple method.
-
Includes a theoretical analysis of the method under 2 lenses, a Gaussian mixture model, and causal structural model (appendix F2). These appear sound, given that the starting assumptions are met (see related comments in W1 below).
-
Large set of experiments on various benchmarks (from toy to realistic data). Good performance on some realistic ones (e.g. Camelyon17).
缺点
W1. The benefits of the proposed method are entirely based on the premise that simple features = bad features.
I think this is clear to the authors and also made clear upfront, but the limitations should be stated more prominently. In the introduction: "we take the viewpoint that features that are usually regarded as bein spurious for the task are often simple and quite predictive" This is not just a "viewpoint", the authors should clearly say that this is the critical premise on which the entire work depends.
Later on: "Models trained with ERM tend to focus on simple features (such as background) that do not generalize" This is not correct. Simple features can very well be features that do generalize. The fact that deep learning is successful more often than not, and that OOD generalization is only a subfield of research, is the proof that the features learned by default are usually "good" ones.
In appendix E: "alleviating simplicity bias can be a useful inductive bias in such cases. This is consistent with our experimental evaluations on several datasets" I'm not sure that the authors realize that there is a heavy selection bias in the datasets that they use (which are datasets specifically selected on even purposefully builts to contain the kind of simple-but-spurious features that they propose to avoid). The only dataset where the distinction between the relevant/spurious feature is more realistic and not as clear (CelebA) is also the dataset on which the proposed method is much less effective.
In the summary of contributions: "We empirically evaluate and validate the hypothesis that spurious features are simpler than invariant features, with respect to representative datasets from Table 1" I do not think this is a valid claim: most of these datasets are purposefully selected or built to showcase the benefits of methods that "debias" simple features (most prominently: colored MNIST, waterbirds). So by definition, this hypothesis will be verified. If the authors want to validate the hypothesis, they should sample a random selection of datasets (NOT only from the OOD/debiasing literature!).
Note that these issues are not specific to this paper and plague the whole field of debiasing methods. The authors do already take steps to make explicit the meaning of "spurious features" so I'm hopeful that this paper can rise above the standard of the field with just a few more clarifications.
W2. Novelty of the method. The general idea of training one standard (biased) model, then training another regularized by the first one, has been thoroughly explored in the literature. All the references below (some of which are cited in the literature review) are based on this idea of training one initial biased model, with various improvements on top.
The method proposed in this paper is an implementation this general idea with a conditional-MI regularizer. The technical novelty therefore seems very limited.
[1] A too-good-to-be-true prior to reduce shortcut reliance
[2] Learning from failure: Training debiased classifier from biased classifier
[3] Towards Debiasing NLU Models from Unknown Biases
[4] A Conservative Approach for Unbiased Learning on Unknown Biases
[5] BiasEnsemble: Revisiting the Importance of Amplifying Bias for Debiasing
[6] BoosterNet - Improving Domain Generalization of Deep Neural Nets using Culpability-Ranked Features
[7] Chroma-VAE Mitigating Shortcut Learning with Generative Classifiers
[8] Learning Debiased Classifier with Biased Committee
[9] MaskTune Mitigating Spurious Correlations by Forcing to Explore
[10] Measures of Information Reflect Memorization Patterns
[11] Roadblocks for Temporarily Disabling Shortcuts and Learning New Knowledge
[12] Self-supervised debiasing using low rank regularization
[13] Unsupervised Learning of Unbiased Visual Representations
[14] BiasAdv: Bias-Adversarial Augmentation for Model Debiasing
W3. The results on subgroup robustness are not that great (Table 8). I suggest replacing the table (which is tedious to read) with a scatter plot showing average vs. worst-group accuracy. It then becomes clear that the proposed method (and others, with the exception of GroupDRO that can leverage the additional knowledge required to solve OOD generalization) "simply" shifts the balance of relevant/spurious features such that the gains in worst-group accuracy are almost matched by a decrease in average accuracy.
W4. The "future directions" proposed in the conclusion are very generic. For example, proposing the following is pretty much useless as a statement: "further explore the capabilities and limitations of our approach and to also further understand its theoretical properties"
What are you thinking about exactly? Why wasn't it done yet for this paper?
I do not understand what this means: "auditing large models with respect to much simpler models".
I also do not understand: "explore the power of similar approaches for other desiderata".
问题
Please comment on W1 and W2.
W1 is a very important issue, but I think it could be addressed with some edits.
W2 is the critical issue (lack of technical novelty).
W3 is (I believe) a consequence of W1.
W4 is a very minor comment/suggestion.
Thank you for reviewing our work and providing helpful suggestions to improve the paper.
W1: “...the limitations should be stated more prominently.”
We have made some minor edits to the introduction/Fig. 1 caption to clarify some of the points mentioned by the reviewer.
“... there is a heavy selection bias in the datasets used… If the authors want to validate the hypothesis that spurious features are simpler, they should sample a random selection of datasets (NOT only from the OOD/debiasing literature!).”
We note that in our experiments we consider several real-world datasets, such as ImageNet-9, Camelyon17, MultiNLI, CivilComments, Bias-in-Bios, and our approach is effective across all of these datasets. These results present strong evidence that mitigating simplicity bias is a useful inductive bias for several applications and modalities. Even on CelebA, where CMID is not competitive with recent methods, it significantly improves in terms of the worst-group accuracy compared to ERM.
We also note that our approach is mainly designed to improve generalization under distribution shifts, and as discussed in Appendix E, mitigating simplicity bias might not be that useful for ID generalization. Therefore, we only consider representative datasets from Table 1 for OOD generalization and subgroup robustness to validate our hypothesis that spurious features are often simpler than invariant ones.
W2: Novelty
Please see Comment 1 in the general response section. Appendix D includes discussion on the differences with [1-2] and we will update it to discuss differences with [3-14]. We note that all of these methods (except [10]) are designed (or evaluated) specifically for either vision [4-9, 11-14] or language [3] datasets, whereas our approach is more broadly applicable to several modalities including vision, language and tabular data. [3] is similar to JTT and can suffer from similar drawbacks. [4, 6] are designed specifically for CNN-based architectures. Several of these methods are computationally intensive: [5, 8] involve training several biased models before training the final debiased model, [7] involves training a VAE in the first stage, [9] involves using an explainability approach to create a masked version of the dataset based on the first stage model to train the final model, [10] proposes a model selection strategy that requires training multiple models before selecting the best one, [11] involves using three models: a pretrained model, and an autoencoder and the final model that are trained simultaneously, [13] involves a three-stage approach, [14] involves creating adversarial samples for a biased model which are used to train the final model.
W3: Scatter plots showing average vs. worst-group accuracy for the subgroup robustness datasets
As suggested by the reviewer, we have added scatter plots showing average and worst-group accuracy for the four subgroup robustness datasets in the paper (Fig. 12 in the Appendix). Please see the related discussion in Appendix H.3.4.
W4: Conclusion
We will revise the conclusion to clarify these points.
Dear reviewer pjci, Thank you for your time and effort in reviewing our work. We have responded to the weaknesses/questions you raised in the review and hope that our responses have addressed your concerns. We would be happy to address any additional/remaining questions that you may have.
In this work authors propose a debiasing method based on minimizing conditional mutual information between a biased model and an unbiased one. The proposed method is tested on standard debiasing vision benchmarks and shows promising results.
优点
-
The proposed method does not require bias group labels
-
The proposed method shows good results overall
-
It does not require substantial training complexity compared to other methods
缺点
-
The approach is not completely novel; MI has been investigated in many works as a way to mitigate bias eg. [1,2,3] and most importantly [4] which is very similar to the proposed method.
-
The experimental evaluation does not compare to many established debiasing techniques eg [5-10] just to name some of the most recurring techniques.
[1] Zhu, Wei, et al. "Learning bias-invariant representation by cross-sample mutual information minimization." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
[2] Han, R., Wang, W., Long, Y., & Peng, J. (2022). Deep Representation Debiasing via Mutual Information Minimization and Maximization (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 12965-12966. https://doi.org/10.1609/aaai.v36i11.21619
[3] Ragonesi, Ruggero, et al. "Learning unbiased representations via mutual information backpropagation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
[4] Tartaglione, E. (2022, November). Information Removal at the bottleneck in Deep Neural Networks. In 33rd British Machine Vision Conference 2022,{BMVC} 2022, London, UK, November 21-24, 2022.
[5] Lee, Jungsoo, et al. "Learning debiased representation via disentangled feature augmentation." Advances in Neural Information Processing Systems 34 (2021): 25123-25133.
[6] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[7] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In International Conference on Machine Learning (ICML), 2020.
[8] Mohsan Alvi, Andrew Zisserman, and Christoffer Nellaker. Turning a blind eye: Explicit removal ˚ of biases and variation from deep neural network embeddings. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 0–0, 2018.
[9] Tartaglione, E., Barbano, C. A., & Grangetto, M. (2021). End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13508-13517).
[10] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. In Advances in neural information processing systems, pp. 841–852, 2019.
问题
-
See weaknesses;
-
How does MI minimization relate to adversarial debiasing setups, i.e. when a classifier is trained on the bias labels (on the same feature space) and the main model is trained to minimize its accuracy while minimizing CE on the target task?
Thank you for taking the time to review our paper.
Novelty; "MI has been investigated in prior works on debiasing"
Please see Comment 1 in the general response section. While MI has been investigated before, all of these works [1-4] require access to spurious feature labels. This information is often difficult to obtain in practice and in contrast to these methods, our approach does not have such a requirement.
Comparison with recent debiasing methods
Please see Comment 3 in the general response section. We compare with [7] and [10] on the ImageNet-A dataset and with [5], [7] and [9] on the colored-MNIST dataset.
How does MI minimization relate to adversarial debiasing setups?
Generally, debiasing methods that involve MI minimization require access to group/sensitive attribute labels whereas adversarial debiasing setups involve using an adversarial model to predict the group/sensitive attribute labels, while encouraging the main model to learn features that don’t correlate well with the sensitive attribute.
I thank the authors for their response and I appreciate the additional empirical comparison with other techniques. I still think that the difference wrt to some of other works employing MI is somehow limited; nonetheless, this work still represents a good methodological contribution. I am willing to increase my score, and I suggest the authors incorporate a discussion on the similarities with other methods in the final version.
Thank you for reviewing our work and for increasing the score. We will include further discussion on related work in the paper, as suggested.
The paper focus on the issue of spurious correlation that affects the neural networks and make a connection between this phenomenon and simplicity bias. As models tend to favor learning simpler features over more complex ones, even if the latter are more informative, the paper links the similar phenomenon of learning spurious feature to learning simple features. To counteract this, the authors propose a framework that first trains a simple model and then regularizes it to encourage the use of a diverse set of features for predictions. The approach is demonstrated to be effective in various scenarios, enhancing OOD generalization and robustness.
优点
- The paper is well-structured and easy to follow. The authors made good efforts to motivate their study with empirical findings. The illustrative figures are helpful.
- The authors presented experiments across a wide range of datasets. The analysis is comprehensive and the evaluation of CMID is done on multiple aspects. CMID does achieve satisfactory results on a majority of these tasks.
缺点
- Definition 1 lacks formality. The terms "high (in-distribution) accuracy" and "certain complexity" require clearer specification. Additionally, the description of a "simple model" as one with significantly lower complexity than benchmark models is vague. I understand that there might not be a specific threshold on "high accuracy", but I think this definition can be formalized better. Can we maybe define models with "certain complexity" as those capable of learning the training data so that the training loss converges to near-zero? Conversely, "simple models" could be characterized as those lacking the capacity to adequately fit the training data.
- Definition 2 also lacks clarity and formality. Another concern is that, spurious feature has been an established concept. It would be more appropriate to treat Definition 2 as a hypothesis, which needs to be substantiated with sufficient empirical observations.
- In Assumption 1, which outlines the data model, there is a need to ensure that the spurious feature here is consistent with that in Definition 2. Currently, it does not seem evident that the spurious feature in Assumption 1 matches that in Definition 2.
- (minor) Several recent works [1-2] also present theoretical results on the study of spurious correlation. Especially, [1] seems to align well with the finding in this paper but through different perspectives. It would be better have a brief discussion on the alignments and differences of these studies.
[1] "Robust Learning with Progressive Data Expansion Against Spurious Correlation." Advances in neural information processing systems. 2023.
[2] "Understanding and Improving Feature Learning for Out-of-Distribution Generalization." Advances in neural information processing systems. 2023.
问题
See in weakness.
Thank you for reviewing our paper and providing constructive feedback to help improve our work.
Better formalization of Definition 1
Thank you for this suggestion, we will update Definition 1 as follows:
Consider a task on which benchmark models which achieve high (in-distribution) accuracy and can attain near-zero training loss have a certain complexity (in terms of number of parameters, layers etc.). We consider models that have significantly lower complexity than benchmark models and cannot attain arbitrarily small loss on the training data as simple models. Similarly, features that can be effectively learned using simple models are considered as simple features.
“It would be more appropriate to treat Definition 2 as a hypothesis.”
We agree with the reviewer, and we do treat this as a hypothesis, e.g. as mentioned in the first point of the contributions, “We empirically evaluate and validate the hypothesis that spurious features are simpler than invariant features, with respect to representative datasets from Table 1 (Section 2).” Definition 2 is based on the experimental results in Section 2, which validate this hypothesis on some datasets. We have made some edits to Section 2 to clarify this.
“It does not seem evident that the spurious feature in Assumption 1 matches that in Definition 2.”
Please see Comment 2 in the general response section.
Discussion on recent works [1-2]
We will include these in the discussion in Appendix D.
Dear reviewer 5jvr, Thank you for your time and effort in reviewing our work. We have responded to the weaknesses/questions you raised in the review and hope that our responses have addressed your concerns. We would be happy to address any additional/remaining questions that you may have.
The paper proposes to mitigate simplicity bias by incentivizing complex feature learning via a new regularization term. Specifically, the conditional mutual information between the model’s features, and those of a pretrained “simple” model is added to the training objective.
Edit Raised score to 5 based on author responses and additional experiments.
优点
The paper is well-written and easy to follow. The basic proposal of using CMI for regularization makes intuitive sense, and the evaluation covers a number of datasets and metrics for robustness.
缺点
The contributions can be considered among 3 axes: proposal, empirical evaluation, and theory. In this reviewer’s assessment, each of these axes have weaknesses, and the sum total falls somewhat short of a solid contribution.
- Proposal: Novelty w.r.t. other feature diversity methods e.g., ESB [1] which is closest in spirit, and uses a gradient-based diversity incentive instead of CMI. Also many other related papers cited by the authors in the appendix.
- Evaluation:
- Weak empirical results, eg, a simple method like JTT[2] is beating CMID across almost all datasets for Group robustness. Refer to table 7.
- In generalization, too (WILDS / Fairness), surprisingly CMID does not have the best OOD, something that would be expected of a method designed to avoid spurious features.
- Missing head-to-head comparisons against more recent methods (some of them cited but not compared), including LWBC [3], SIFER [4], ESB [1].
- Not necessary, but authors may consider evaluating on other datasets such as BAR, NICO, Imagenet-A/C/R, other datasets from DomainBed, etc., for a more thorough comparison for the above and other methods.
- Theoretical results appear weak / not directly applicable to the problem setting (please see below.)
- Conceptual: A key, unaddressed, question (as far as I could tell) is what happens when the simple model also depends to some extent on complex features. It seems reasonable to worry that the regularization can cause more harm than help.
- relatedly, the idea that simple=bad/spurious, complex=good/invariant, for some mathematical definition of simple & complex, seems unlikely to hold water in real world applications.
Other questions/comments.
- As I understand it, Theorem 1 considers features {invariant, spurious} that are in the same complexity class (i.e., gaussian), and a linear predictor. The “simple model” is guaranteed to only use the spurious features. Under these circumstances the CMI biases the model towards the invariant features. This seems not very convincing:
- It would appear that given the strong assumptions above, the claim follows in a straightforward manner. In any case, guaranteeing the “simple model” only uses simple features seems difficult in practice.
- The central claim that simple and complex features are from different complexity classes (see e.g., Section 2 and figure 2) is not captured here. Even in that case, the interesting result / question would be when the "simple" model has nonzero weight on invariant features as well.
[1] Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization by Teney et. al.
[2] Just Train Twice: Improving Group Robustness without Training Group Information by Liu et. al.
[3] Learning Debiased Classifier with Biased Committee by Kim et. al.
[4] Overcoming Simplicity Bias in Deep Networks using a Feature Sieve by Tiwari et. al.
问题
- Results:
- Texture vs shape: this is good, but a) does it materially improve accuracy on IN-9? Or generalization on IN challenge datasets? b) how does it compare against the other feature diversity approaches?
- Generalization: Why, in your understanding, do other methods have better generalization but potentially worse in-domain accuracy than CMI? Given the nature and intent of the regularization (learn features that are conditionally independent of "simple features" presumably also learned on indomain data), one would have expected the exact opposite finding.
Thank you for taking the time to provide the detailed review and constructive comments for our work.
Novelty
Please see Comment 1 in the general response section. We also discuss differences with prior work including [1] in detail in Appendix D.
Evaluation
- “...JTT is beating CMID across almost all datasets for Group robustness.”
We consider several applications in our work: mitigating simplicity bias, enhancing OOD generalization and improving subgroup robustness and fairness. Although JTT is more effective than CMID on three subgroup robustness datasets, the gap on two of these is not very large. On the fourth dataset (CivilComments), CMID outperforms JTT by a considerable margin. In addition, CMID is significantly better than JTT for mitigating simplicity bias, and leads to improved OOD generalization, without a significant drop in IID performance, as seen in the experiments in Section 5. In addition, in contrast to JTT, our approach comes with theoretical guarantees.
- “CMID does not have the best OOD in WILDS / Fairness.”
We note that while CMID and EIIL have the same OOD accuracy on the Adult-confounded (fairness) dataset, CMID has a much smaller gap between in-distribution (ID) and OOD accuracy compared to all other methods. Similarly, while JTT has slightly better OOD accuracy (0.1) than CMID on the Camelyon-WILDS dataset, it has a much larger gap between ID and OOD accuracy compared to CMID. We will clarify this in the paper.
- Comparison with more recent methods/evaluation on ImageNet-A dataset
Please see Comment 3 in the general response section. We compare our approach with [2-4] on the ImageNet-A dataset.
Conceptual
- "What happens when the simple model also depends to some extent on complex features? It seems reasonable to worry that the regularization can cause more harm than help."
We note that in Fig. 2, we compare the effect of using various model architectures of different complexities as the simple model for our approach for the Waterbirds dataset. Models such as ResNet18 and ResNet50 can rely on complex features as well, and indeed we observe some drop in the worst-group accuracy when we regularize the CMI with respect to these models to train the final model. However, we note that these worst-group accuracy values are still significantly better compared to ERM. We will update the figure and include this discussion in the paper.
Further, we note that we don’t see any adverse effects of our approach in any of our experiments. Even on CelebA, where CMID is not competitive with recent methods, it significantly improves in terms of the worst-group accuracy compared to ERM.
- “The idea that simple=bad/spurious, complex=good/invariant, for some mathematical definition of simple & complex, seems unlikely to hold water in real world applications.”
We respectfully disagree with this comment. We note that in our experiments we consider several real-world datasets, such as ImageNet-9, Camelyon17, MultiNLI, CivilComments, Bias-in-Bios, and our approach is effective across all of these datasets. These results present strong evidence that this assumption is true across several cases and that mitigating simplicity bias is a useful inductive bias for several applications.
Regarding Theorem 1:
- “Theorem 1 considers a setting where the simple model is guaranteed to only use the spurious feature.”
We consider this setting for our theoretical results because it allows us to isolate the effect of the CMI regularization. We agree that it is also important to consider the case where the simple model has some nonzero weight on the invariant feature, particularly in practice. Therefore, we empirically compare the effect of using various model architectures of different complexities as the simple model for our approach in Fig. 2. We believe that this addresses the concern that “guaranteeing the “simple model” only uses simple features seems difficult in practice” and offers some insights about our approach in that case.
- “The central claim that simple and complex features are from different complexity classes (see e.g., Section 2 and figure 2) is not captured here.”
Please see Comment 2 in the general response section.
“Why, in your understanding, do other methods have better generalization but potentially worse in-domain accuracy than CMI?”
We would like to clarify that such an effect is only observed in two cases: JTT on Camelyon17 and EIIL on Adult-Confounded. In other cases, CMID either improves on both i.i.d and OOD accuracy or improves in terms of the OOD accuracy, while leading to some decrease in i.i.d accuracy. We note that depending on the strength of the CMI regularization (and similarly values of the relevant hyperparameters for other methods like JTT and EIIL), the model might still use simple features to some extent, which can lead to better i.i.d accuracy and lower OOD accuracy, and similarly a stronger regularization will have the opposite effect.
Dear authors, thank you for investing the time and energy in crafting a detailed, thoughtful response to the reviews. I have carefully read through the response, the other reviews and discussion.
I am raising my score to a 5 as a result. Unfortunately, I still believe the paper falls short of acceptance.
The primary reason for raising score is the shared additional comparisons, and the commendable honesty in sharing shortfalls compared to other baselines (SIFER, DFA). I'm ok with overlooking the large advantage of REBIAS as it appears to be hand-crafted for the specific synthetic scenario. (A related nit, for future rewrites, is that it is strongly preferable to compare all baselines together on all datasets, or at least on one of the most challenging datasets. Results on synthetic datasets very often do not extend to real-world).
The reason for not accepting is that I continue to find the theoretical result on the one hand completely expected (if the simple model is guaranteed to not use complex features, are we at all surprised that CMI works? shouldn't it work simply by definition?) and on the other hand irrelevant to real-world scenarios. Unfortunately the results do not stand on their own as a strong empirical findings paper either.
Thank you for your detailed response and for increasing the score.
We note that in our theoretical results for the Gaussian features, we also consider a case where there are multiple spurious features (Appendix F.1). Even in that case, we theoretically show that CMI regularization is effective in increasing the weight on the invariant feature, which is a bit surprising.
We also note that previous methods in the problem areas that we consider generally do not have theoretical guarantees (discussed in more detail in Appendix D), and although the setting we consider is relatively simpler, it does offer insights about the regularization and the effectiveness of our approach. We believe that this is a step towards developing more theoretically grounded methods.
We thank the reviewers for taking the time to review our paper and providing valuable comments and feedback to help improve our work. We are grateful that they find the paper well-written (reviewer Q6kW) and easy to follow (reviewers Q6kW, 5jvr), and our approach to encourage conditional independence with a simple model to learn the final model intuitive (reviewer Q6kW), simple (reviewer pjci), having lower training complexity (reviewer h3b4), and theoretically motivated (reviewer pjci). Our experimental evaluations on several benchmark datasets to show the effectiveness of our approach in mitigating simplicity bias, enhancing OOD generalization and improving subgroup robustness and fairness were also appreciated (reviewers Q6kW, 5jvr, h3b4, pjci).
We include three main comments here addressing some common concerns and include other details in the responses to each reviewer. We will incorporate all feedback in the paper.
1. Novelty/Differences with Prior Work (reviewers Q6kW, h3b4, pjci)
We note that in general, prior work focuses on one or two of the applications that we consider: mitigating simplicity bias, enhancing OOD generalization and improving subgroup robustness and fairness, while our approach proves effective across several datasets for all cases. We agree that the general idea of training one standard (biased) model, and using it to train another model that is debiased with respect to the first one has been explored in the literature. However, we note that one of the contributions of our work is to show that debiasing can be useful across several applications, which makes our approach applicable in more general settings compared to prior work.
In addition, our approach has other advantages compared to previous methods: it is simpler/less computational intensive compared to feature diversification methods, it is theoretically grounded and explicit in its assumptions, which could make it easier for a practitioner to evaluate and use, and it does not require access to group/spurious feature labels or additional unlabelled data and allows for feature diversification directly on the training data.
2. Consistency between Assumption 1 and Definition 2 (reviewers Q6kW, 5jvr)
We thank the reviewers for this comment. We note that the current setting considered in Assumption 1 is the simplest setting that allows us to show the result in Theorem 1. However, we can directly extend our result to the setting where the complex feature is -dimensional, where , instead of 1-dimensional, while the simple feature is still 1-dimensional. Hence, there is a clear distinction between the function classes required to learn the simple feature and the complex feature. The current setting is easy to visualize and interpret, and hence we focus on that in the paper. We will add this clarification in the paper and an example in the Appendix.
3. Comparisons with more recent methods (reviewers Q6kW, h3b4)
As suggested by reviewer Q6kW, we compare the accuracies of ERM, JTT and CMID on ImageNet-A. We also include comparisons with recent debiasing methods, including LWBC and SIFER, as suggested by reviewer Q6kW, and RuBi and ReBias, as suggested by reviewer h3b4. We observe that CMID is comparable to SIFER and significantly improves over all other methods in terms of test accuracy on the ImageNet-A dataset.
| Method | ERM | LfF [1] | RuBi [2] | ReBias [3] | JTT [4] | SSL+ERM [5] | LWBC [5] | SIFER [6] | CMID |
|---|---|---|---|---|---|---|---|---|---|
| Accuracy | 24.9 | 24.6 | 27.7 | 29.6 | 29.6 | 34.21 | 35.97 | 39.98 | 38.3 |
We also compared our approach with recent debiasing methods, including EnD, DFA and ReBias, as suggested by reviewer h3b4. Following the setup in [9], we consider the 10-class colored MNIST dataset. In the train set, the samples in every class are injected with a color with 95% correlation, while the colors in the test set are uncorrelated with the label. The table compares the test accuracies for various methods on this dataset. We observe that CMID significantly improves over HEX, LfF and EnD, and is comparable to DFA, while ReBias significantly outperforms all other methods as it focuses particularly on debiasing with respect to texture or color bias.
| Method | HEX [7] | LfF [1] | EnD [8] | DFA [9] | ReBias [3] | CMID |
|---|---|---|---|---|---|---|
| Accuracy | 74.62 | 80.57 | 81.15 | 89.66 | 96.96 | 87.12 |
We will add these results and details about the experimental settings in the Appendix.
References:
[1] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
[2] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. In Advances in neural information processing systems, pp. 841–852, 2019.
[3] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In International Conference on Machine Learning (ICML), 2020.
[4] Just Train Twice: Improving Group Robustness without Training Group Information by Liu et. al.
[5] Learning Debiased Classifier with Biased Committee by Kim et. al.
[6] Overcoming Simplicity Bias in Deep Networks using a Feature Sieve by Tiwari et. al.
[7] Haohan Wang, Zexue He, Zachary L. Lipton, and Eric P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019.
[8] Lee, Jungsoo, et al. Learning debiased representation via disentangled feature augmentation. Advances in Neural Information Processing Systems 34 (2021): 25123-25133.
[9] Tartaglione, E., Barbano, C. A., & Grangetto, M. (2021). End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13508-13517).
This paper introduces a regularizer encouraging models to leverage more complex features. The authors discuss how this mitigation of “simplicity bias” can aid various generalization and robustness properties.
Steering which features are used by neural networks is a large and important area of research. This work follows a line of works that have focused on encouraging feature diversity or steered learning to more complex features (e.g. texture vs. shape) to avoid learning overly simple features that do not generalize in an intended way.
However, many concerns by reviewers remain. The empirical gains of the method are modest, the general idea not novel enough and the theoretical analysis relies entirely on the assumption that simple features are not desirable to learn, which is often not true. While the direction of this paper is certainly interesting and relevant to the community, it requires further work to meet the requirements for an ICLR publication.
为何不给更高分
Too many concerns raised by reviewers.
为何不给更低分
There is no lower score
Reject