SEBRA : Debiasing through Self-Guided Bias Ranking

W1: The findings in the SEBRA heavily rely on Assumption 1.

• We agree with the reviewer that SEBRA heavily relies on Assumption 1 for spuriosity ranking. [1] theorically proves that unsupervised bias discovery is fundamentally impossible without further inductive biases or additional information. We adopt 'Hardness-Spuriosness symmetry' as an inductive bias to obtaining a continuous measure of spuriosity. Thus eventhough, Assumption 1 forms a central assumption of the paper, the therorical impossibility result neccesites this.

Q1: Can you delve on the motivation a bit? Why is this kind of de-biasing important?

• Training data often contains spurious correlations due to inherent biases or unintended patterns. When deep learning models are trained on such data, they tend to rely on these spurious correlations for predictions. This reliance can lead to poor generalization, particularly when the deployment environment lacks the same spurious patterns present in the training data.

• The consequences of such failures can be severe, especially in critical domains like healthcare, where inaccurate predictions can directly impact patient outcomes. Therefore, it becomes imperative to develop debiasing methodologies that enable models to focus on core, meaningful attributes even when trained on biased datasets. This motivates our proposal to learn a robust, debiased model capable of generalizing effectively from biased data.

Q2: Is Assumption 1 a part of known literature or is it something discovered during the behavioral exploration of this work? If it is a known fact in debiasing literature, please cite relevant works.

• The Hardness-Spuriosity symmetry has been discussed in prior works such as [2, 3]. In this work, we reconcile and formalize this concept to derive a continuous measure of spuriosity. To provide further empirical support for this assumption, we conducted an additional experiment during the rebuttal phase in which we plot the training loss for samples with varying levels of spuriosity in Fig.8 of appendix D4.

• As illustrated in the figure, highly spurious samples are relatively easier to learn, as indicated by the faster decline in their loss compared to samples with low spuriosity.

References:

[1] ZIN: When and How to Learn Invariance Without Environment Partition? , NeurIPS 2022

[2] Complexity Matters: Feature Learning in the Presence of Spurious Correlations, ICML 2024

[3] Learning from Failure: De-biasing Classifier from Biased Classifier, NeurIPS 2020

Dear Reviewer APoD,

As today marks the conclusion of the discussion phase, we kindly request any additional feedback you may have regarding our submission. We would also appreciate it if you could consider our rebuttal response during your final evaluation.

Thank you for your time and valuable input throughout this process.

Best regards,

Authors

We thank the reviewer for their constructive feedback and thoughtful suggestions. Below, we provide detailed clarifications addressing the questions and concerns raised.

W2: Optimization Parameters

• The method involves three parameters that influence the optimization process. However, two of these parameters, $v_i$ and $u_i$, have closed-form solutions, as presented in Equations 2 and 3 of the paper. Since these solutions do not require optimization, the number of parameters that need to be optimized is reduced to just one.

• Additionally, the convergence time of Sebra is primarily influenced by the size of the training dataset rather than the dimensionality of the data. This makes Sebra scalable to high-dimensional datasets. Furthermore, the ranking phase of Sebra is very efficient, taking only 5 minutes on the UrbanCars dataset, which indicates quick convergence.

W1/Q1: Handling Outliers and Label Noise

• We appreciate the reviewer’s concern regarding the potential failure of Sebra in datasets with outliers or label noise. Sebra ranks data points within each class based on their spuriousness. In the presence of label noise or outliers, such samples tend to be ranked highest in the obtained rank order. Since Sebra isolates these outliers and mislabeled samples in the highest-ranked set, they can be excluded from the contrastive debiasing stage. This exclusion effectively mitigates the negative influence of such samples on the debiasing process.

• To provide empirical evidence, we applied our ranking scheme to the Living17 dataset, which contains 17 classes derived from ImageNet. Qualitative visualizations of the highest- and lowest-ranked samples for several classes are provided in Fig 7, Appendix D3. As shown in the figure, the highest-ranked images (bottom row) consist predominantly of outliers and mislabeled samples, while the lowest-ranked images (top row) are devoid of such issues. By discarding the highest-ranked samples, the method can effectively circumvent the adverse effects caused by outliers or noisy labels.

Q2: Support for Assumption 1 (Hardness-Spuriousness Symmetry)

• The Hardness-Spuriosness symmetry has been discussed in some of the prior works such as [1,2], which we reconcile and formalize for the purpose of obtaining a continuous measure of spuriosity. Sebra distinguishes itself from these works by going beyond a simple binary partitioning of the training data. Instead, it produces a per-class rank ordering of the data. This additional layer of derived knowledge enables more efficient utilization of the diversity within the training data, leading to superior bias mitigation. To provide further empirical support for 'Hardness-Spuriousness Symmetry', we provide a plot of training loss of samples with different levels of spuriosity in Fig 8. of appendix D.3. As illustrated in Fig.8 highly spurious samples are relatively easy to learn indicated by a faster decline in their loss compared to samples with low spuriosity.

References:

[1] Complexity Matters: Feature Learning in the Presence of Spurious Correlations, ICML 2024

[2] Learning from Failure: De-biasing Classifier from Biased Classifier, NeurIPS 2020

W3: The overall writing of this paper is not very straightforward: We thank the reviewer for their suggestions in improving the writing of our paper in terms of conciseness and coherence. Based on their recommendations, we have now incorporated several updates across the Methodology and the Introduction sections, which are as follows:

• Merged several redundant portions of text such as the discussions on the concept of spuriousness, the assumption of the Hardness Spuriosity Symmetry, descriptions of the various phases of Sebra such as Selection, Upweighting, and Ranking, etc.
• Added connecting sentences around the Methodology subsection (3.1 to 3.3) to improve coherence.
• Substituted some of the abstract descriptions of our methodology with an intuitive example in Section 3.1, Intuition behind Sebra.

We summarise the additional set of experiments that were performed during the rebuttal phase:

• Additional qualitative results on the Living 17 dataset (Appendix D.3 and Fig.7), to demonstrate the capability of Sebra in segregating and isolating outliers and mislabeled samples.

• Additional empirical evidence supporting hardness-spuriosity symmetry (Appendix D.4 and Fig.8)

We appreciate the reviewer’s insightful suggestions and comments. Below, we provide detailed responses to the questions and concerns raised.

W1-[1/3]: How does this approach offer high-level advantages over debiasing methods that do not utilize ranking?

• Most prior debiasing methods primarily partition the training data into two subsets: bias-aligned(samples with spurious correlations) and bias-conflicting samples(samples without spurious correlations). These methods predominantly rely on identified bias-conflicting samples for bias mitigation. However, the number of bias-conflicting data points is typically very small, often comprising only 0.5% or even less, particularly for datasets with multiple biases. This limited reliance on a small subset of samples prevents these methods from fully leveraging the diversity of the bias-aligned samples in the training set. We have discussed this briefly in section 3.3 (lines 340-345), and we will revise this for additional clarity.

• In contrast, Sebra's ranking framework enables a more effective utilization of the diversity inherent in bias-aligned samples. By leveraging the obtained rankings, Sebra facilitates the construction of positive and negative pairs even within the bias-aligned subset. This approach allows the model to learn target attributes from a broader range of samples, including bias-aligned data, thereby harnessing the full diversity of the training dataset. As a result, Sebra does not require complex data augmentation strategies like many of the recent debiasing works like Logit Correction.

• Furthermore, the framework of spuriosity ranking was introduced in [1] though they rely on human supervision. Sebra enables the same without reliance on human supervision.

W1-[3/3]: Advantages over B2T

• We thank the reviewer for drawing attention to this closely related work. While B2T demonstrates strong debiasing performance, it faces key limitations when compared to Sebra:

  • Inherited spurious correlations from pre-trained models:
    B2T relies on pre-trained CLIP models for bias identification. However, vision-language models like CLIP are known to contain various biases [2]. If there is an overlap between biases in the dataset and those captured by the pre-trained model, the bias identification process may be compromised. In contrast, SEBRA does not depend on any pre-trained models that might contain spurious correlations, making it more robust in such scenarios.

  • Higher Computational Cost: B2T incur significant computational overhead due to the need for caption generation and CLIP score computation. This overhead can increase considerably depending on the complexity of the model used. In contrast, Sebra leverages smaller models, resulting in improved computational efficiency for bias discovery compared to B2T.

  • Limitations on non-natural images:
    B2T relies heavily on the rich knowledge acquired through pretraining on large-scale natural image datasets. However, such pre-trained models may not generalize to datasets from other domains (e.g., medical or satellite imagery). As demonstrated in Appendix F of the B2T paper, pre-trained CLIP models fail on datasets outside their natural image domain. Addressing this limitation often requires pre-trained vision-language models aligned with the specific dataset, which can be challenging to obtain. SEBRA, by design, does not rely on such pre-trained models, making it broadly applicable across diverse domains.

W2.1: Reporting Worst-Group Accuracy (WGA)

• We agree with the reviewer that reporting WGA enhances comparability. Below, we provide the Worst-Group Accuracy (WGA) for datasets with group information:

As shown in the table below, Sebra demonstrates competitive WGA, significantly outperforming other methods like ERM, LfF, and JTT, particularly excelling on CelebA (65.3) and achieving near Group-Dro levels on UrbanCars (73.8). These results will be added to our revised submission.

W2.2: Additional Datasets and Baselines

• We are actively extending our baselines and datasets to ensure broader coverage and comparability. We aim to share these results before the end of the discussion phase.

Q1: Computational Complexity

• To better illustrate Sebra's computational complexity, we provide a detailed breakdown of the time (in wall-clock minutes) required for both the ranking and debiasing phases of our proposed framework. For comparison, we also include the corresponding time for ERM on the UrbanCars dataset. All the measurements were done using a single Nvidia RTX 3090 GPU.

  Time Comparisons

  Method	Time
  Sebra - Ranking	5 minutes
  Sebra - Contrastive Debiasing (Bias Mitigation)	52 minutes
  ERM (Empirical Risk Minimization)	32 minutes

• Please let us know if further clarifications or additional experiments would strengthen our response. We greatly appreciate the reviewer’s time and thoughtful feedback.

References:

[1]Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases, NeurIPS 2023

[2] CLIP the Bias: How Useful is Balancing Data in Multi modal Learning? ICLR 2024

W1-[2/3]: Potential applications of fine-grained spuriosity rankings beyond the contrastive learning framework?

• The contrastive debiasing framework proposed in our method is a design choice adopted to enable efficient debiasing. Beyond the contrastive learning framework, fine-grained spuriosity ranking could be combined with other approaches, such as DRO (Distributionally Robust Optimization), which involves the minimization of worst-group loss. The groups in this scenario could be formed by utilizing the ranks obtained by Sebra. Other alternatives could utilize last-layer retraining-based approaches or leverage metric learning to enable efficient debiasing.

• Beyond debiasing, the fine-grained spuriosity ranking could also be utilized for:

  • Revealing minority subpopulations: High-rank images obtained via Sebra's rankings often correspond to minority subpopulations in the data where spurious correlations are absent.

  • Mislabelled samples and Outlier Identification: Sebra's rankings can assist in identifying mislabeled samples. For example, high-ranked images within each class are more likely to include mislabeled instances. We present qualitative results in Appendix D3 (Figure 7), providing empirical evidence to support this claim.

I thank the authors for their detailed replies to my questions and concerns.

Most of my concerns and questions have been addressed, but the effectiveness of the proposed method remains insufficient. Methods like LfF, JTT, and SelecMix also utilize bias-aligned samples in training, and it is not yet fully convincing whether the proposed approach has a clear advantage over these methods at a high level. Therefore, it is crucial to demonstrate experimentally that the proposed method leverages bias-aligned samples more effectively, outperforming other methods. However, this aspect appears to be inadequately addressed.

I appreciate the additional experiments provided, but I noticed that the baseline performance on CelebA differs significantly from the performance reported in JTT. For a fair comparison, it is important to present results aligned with the JTT experimental setup. Furthermore, the authors have not yet shared results demonstrating the proposed method’s strengths in NLP tasks. I will finalize my score after reviewing these results.

We sincerely thank the reviewer for their active engagement during the discussion phase and are pleased to note that most of the concerns have been addressed. Below, we provide further clarifications on the remaining points:

Q1. Whether the proposed approach have a clear advantage over other methods at a high level?

Methods like LfF and JTT explicitly downweight the bias-aligned samples during debiased training relative to bias-conflicting samples, thereby restricting the model's learning from bias-aligned samples:

• JTT upweights the contribution of bias-conflicting points by a factor $\lambda_{up}$ (typically taking values between 50–100). As a result of this relative upweighting, the contribution of bias-aligned points is reduced.
• LfF upweights training samples using a derived weighting factor $W(x)$, with biased samples receiving smaller weights, again reducing their overall contribution to debiased learning.
• SelectMix partitions the training data into bias-aligned and bias-conflicting sets, then utilizes MixUp for augmenting the bias-conflicting samples. Even though there is no explicit down-weighting of bias-aligned samples in this case, all the bias-aligned samples are treated alike (there is no distinction between severely bias-aligned and slightly bias-aligned points). As a result:
  • During the MixUp phase, slightly bias-conflicting samples might be augmented using severely bias-aligned samples, making the resulting sample predominantly biased. These points, when utilized for training, could cause the model to capture undesirable bias attributes.

Advantages of Sebra

Sebra offers clear advantages over these previous methods in the following aspects:

• Unlike LfF and JTT, Sebra does not downweight any of the training points, enabling the model to learn from all the data points. This is possible because the fine-grained ranking obtained allows the formation of informative contrastive pairs, enabling the model to learn core attributes even from bias-aligned points.

• Unlike in select mix, wherein often the augmented samples generated by mixup could be highly biased (as described above), our approach because of the availability of the ranking would only contrast a highly bias-aligned point with highly bias-conflicting samples.

We would like to emphasize additional key advantages that set our framework apart from prior debiasing approaches:

• Mitigating Multiple Biases: Unlike previous methods (including LfF, JTT, and SelectMix), which struggle when datasets contain multiple biases, Sebra can mitigate multiple biases simultaneously.

• Applications Beyond Debiasing: Sebra goes beyond traditional debiasing by enabling valuable applications such as the identification of mislabeled samples and outliers, broadening its utility across various tasks.

Experiemental Support

We deeply appreciate the reviewer for highlighting the lack of evidence supporting our claim that Sebra leverages bias-aligned points more efficiently compared to existing methods. This is a critical observation, and we recognize its importance in establishing the credibility of our results.

To address this gap, we conducted an additional experiment on the BAR dataset. In this experiment, we selected a couple of the state-of-the-art (SOTA) methods identified by the reviewer (JTT, LfF) and evaluated their performance after excluding the bias-aligned points from the training set. We followed the same procedure as our proposed method, Sebra. Specifically:

Here, $\Delta$ denotes the improvement in test accuracy upon the introduction of bias-aligned training samples.

As observed in the above table, the inclusion of bias-aligned points to the training paradigm of JTT, and LfF only marginally impacted the performance by 0.9% and 1.9% respectively. Debiasing using Sebra produces a significant improvement of 7.36% supporting our claim that Sebra leverages bias-aligned points more effectively compared to existing methods.

Dear Reviewer CwjJ,

Thank you for your constructive comments and for raising your score! We greatly appreciate your support!

Best regards,

The Authors

I appreciate the detailed response. Most of my concerns are addressed and I will raise my score.

We thank the reviewer for their valuable suggestions. Below, we address your concerns in detail:

Q1: Extension to Semi-Supervised Learning with New Classes

• Sebra is orthogonal conceptually to the learning paradigms (e.g., supervised, semi-supervised learning), as no added assumption is made on the amount of labels and if new classes present or not. Semi-supervised learning is not evaluated in our initial submission since supervised learning is the setting adopted in all existing works, where debiasing remains a challenge. Extending to more challenging setups like the suggested is critical as part of future work. One potential approach is to initialize the backbone architecture with weights obtained from unsupervised pretraining on unlabeled data, and then apply Sebra to achieve rank ordering on labeled data. For instance, in the case of the UrbanCars dataset, our training protocol follows a similar strategy: we first initialize the feature extractor with pretrained ImageNet weights and then proceed to obtain a rank order for the two classes of UrbanCars (urban/country), which are not included in the ImageNet dataset. We agree that extending our framework to the semi-supervised setting is quite interesting, but our paper focuses on the bias in supervised learning and makes a unique and important contribution to this research area by proposing the ranking-based debiasing method.

We plan to provide additional experimental evidence to further demonstrate the applicability of Sebra in a semi-supervised learning framework with new classes before the conclusion of the rebuttal period.

W1, W2: Improvement in Writing

• We appreciate the reviewer’s feedback and suggestions for improving the writing, as well as for pointing out some overlooked errors. We would incorporate these suggestions in further revisions. Additionally, we will move some of the qualitative results from the appendix to the main paper to provide more examples and improve the overall clarity.

W3: Requirement of Class Labels

• We appreciate your observation regarding the use of class labels. To clarify, our method requires labels for the target variable as it operates within a supervised learning framework, consistent with prior works in this domain. In the given problem setting, spurious correlations are defined and measured with respect to the target variable, making target labels an inherent part of the problem setting. Our method does not introduce any additional label requirements beyond those specified by the problem setting.

• Importantly, our approach does not require labels for the protected attribute, a key aspect of its design that ensures applicability in scenarios where such information is unavailable or unreliable.