DeNetDM: Debiasing by Network Depth Modulation

We thank the reviewer for the valuable feedback.

W1. Empirical Evaluation: Including newer more challenging datasets could further improve this paper.

A. In response to the reviewers' suggestions, we have evaluated the effectiveness of our approach on the CelebA dataset, where blonde hair is the target attribute and gender is the spurious attribute. Due to time constraints, instead of using the full CelebA dataset, we employed a subsampled version as done and described in [A], maintaining the same data splits for consistency. We employed the same architectures that were applied to the BFFHQ dataset, ensuring that the target architecture remains consistent with [A]. To ensure a fair comparison, we reference results from [A] on the same dataset split for methods such as ERM, JTT, Disentangled Feature Augmentation, and DCWP. Currently, we lack results for LC and LfF on this specific dataset version, as they used the original CelebA dataset in their study. We plan to incorporate these comparisons before finalizing the camera-ready version of our paper.

As the results indicate, DeNetDM achieves relatively high worst-group accuracy compared to other approaches.

We also conducted experiments on two variants of SpucoMNIST with low and medium spurious difficulty magnitude respectively and compared our results to those reported in [B]. The results were averaged over three random seeds. It can be observed that DeNetDM performs on par with its supervised counterparts, such as GroupDRO and Group Balancing on SpucoMNIST dataset.

W2. Insufficient discussion of similarity to related work: Two related works seem very similar in method to the proposed method. 1) Overcoming simplicity bias in deep networks using a feature sieve and 2) Learning from failure: De-biasing Classifier from biased classifier.

A. We appreciate the reference to the following papers: [C] "Overcoming Simplicity Bias in Deep Networks Using a Feature Sieve," [D] "Learning from Failure: De-biasing Classifiers from Biased Classifiers," and [E] "Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias." A commonality between our work and these approaches is the emphasis on examining the early training dynamics of neural networks to identify and address biases. Specifically, the authors of [C] propose that simple features are learned quickly, appear in early layers of the neural network, and tend to propagate throughout the subsequent layers. Similarly, [D] posits that biases that are easy to learn are captured during the initial phases of training. In [E], the authors demonstrate that, in the early training iterations, the influence of a spurious feature on the network output increases linearly with the level of spurious correlation. While our approach also focuses on the early training dynamics of neural networks, it uniquely characterizes the differences in these dynamics across models with varying depths using a Product of Experts architecture, an aspect not explored by the aforementioned approaches. Instead of concentrating on a single neural network, we investigate how the early training dynamics differ between deep and shallow models, demonstrating how core and bias attributes are learned throughout the training process for each model. As suggested by the reviewer, we will incorporate this discussion into the final version of our paper.

[A] Training Debiased Subnetworks with Contrastive Weight Pruning, CVPR 2023.

[B] Towards Mitigating Spurious Correlations in the Wild: A Benchmark & a more Realistic Dataset, Arxiv, Sep 2023.

[C] Overcoming Simplicity Bias in Deep Networks Using a Feature Sieve, ICML 2023.

[D] Learning from Failure: De-biasing Classifiers from Biased Classifiers, NeurIPS 2020.

[E] Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias, AISTATS 2024.

We thank the reviewer for the valuable feedback and suggestions.

W1. Empirical analysis on networks' depth to learning of different ranks (bias-aligned / bias-conflicting) is limited to CMNIST. Additional experiments on other benchmark datasets, e.g., C-CIFAR10, BAR, and BFFHQ, are required for validity.

A. Please refer to the global response.

W2. As shown in Tables 8 and 10, different pairs of networks result in significantly different results (about 20% differences in conflicting accuracy in Table 8). How sensitive are the bias-aligned (bias-conflicting) attributes with regard to depth of deep (shallow) networks, and the following performances of DeNetDM in stage 2?

A. As the reviewer correctly pointed out, Table 8 in the main paper provides insights into how variations in network depth affect the performance of DeNetDM. As shown in Table 8, DeNetDM can better distinguish between bias and core attributes when there is a significant difference between the depths of shallow and deep branches. However, this also depends on the complexity of the features. For instance, in the case of C-CIFAR10, ResNet 32 may capture core features more effectively than ResNet 8 due to the architectural differences. Therefore, pairing ResNet 50 with ResNet 32 in our approach enhances the learning of core attributes in ResNet 32, as reflected in the conflicting accuracy shown in Table 10.

Regarding sensitivity, if the shallow architecture can partially learn the core attributes and there is a significant difference in the depth between the deep and shallow networks, DeNetDM efficiently aids in learning the bias and core attributes within the deep and shallow models, respectively. Debiasing the target architecture in Stage 2 is more effective when the deep and shallow models accurately capture the bias and core attributes respectively in the first stage, allowing us to leverage the distilled information from these networks. Thus, it can be concluded that DeNetDM is sensitive to the depth variations between the deep and shallow branches.

W3. It is unclear whether DeNetDM maintains the accuracy when applied to unbiased datasets.

A. Our approach operates under the assumption that the data is biased. In such scenarios, deep networks tend to prioritize biased attributes, while shallow models focus on core attributes. When the data is unbiased, the model treats all attributes equally, allowing them to be considered core attributes for the prediction task. In a Product of Experts (PoE) setting, each expert learns a portion of these core attributes, which are then used for joint prediction using both the deep and the shallow branches.

As a result, it cannot be guaranteed that either the deep or shallow branches will perform well individually on the prediction task, since both models contribute to the joint prediction through PoE. We validate this intuition by training stage 1 of DeNetDM on an unbiased CMNIST dataset using a 5-layer MLP and a 3-layer MLP. The deep model and shallow model achieve accuracies of 75.04% and 67.95%, respectively. However, the PoE model, which learns to jointly predict from both deep and shallow experts, achieves an accuracy of 98.64%.

Our approach remains effective if we consider the PoE model from stage 1 as a whole (as the core model) in stage 2 of DeNetDM, rather than focusing on individual shallow or deep models as in the current setting. However, this requires a strategy for model selection (PoE or the shallow model) based on the degree of bias in the data, which could be explored in future work.

We appreciate the reviewer's insightful suggestions and comments. Below, we provide detailed responses to each of the questions and concerns raised.

W1/Q1. One key weakness in approaches of this flavour is the reliance upon use of a “debiased’ validation set to pick crucial hyper-parameters.

A. The authors of [A] provide empirical evidence that group annotations in the validation set are crucial for hyperparameter tuning in bias mitigation. They show that tuning hyperparameters for worst-group validation accuracy significantly improves worst-group test performance compared to tuning for average validation accuracy. Following this approach, as well as the methodology in [B], we tune hyperparameters using a small validation set with group annotations whenever available. For the CMNIST and C-CIFAR10 datasets, we create an unbiased validation set with 250 samples to optimize hyperparameters. In the case of BFFHQ, where an unbiased validation set is not available apart from the test set, we use 40 out of 1000 samples from the test set for validation. For the BAR dataset, we do not perform hyperparameter tuning due to the lack of group annotations and instead apply the same hyperparameters used for the C-CIFAR10 dataset.

We thank the reviewer for citing a reference that employs a differently biased validation set for hyperparameter tuning. However, we believe this is a completely different scenario that warrants independent investigation and could be explored as a future work.

W2/Q3. It might make more sense to consider both aligned and conflicting accuracies of all baselines. Would it be possible to compare both aligned and conflicting accuracies for all methods?

A. We computed the aligned and conflicting accuracies for all baselines, as well as for our approach, on the BFFHQ dataset. Due to rebuttal time constraints, we could not compare the accuracies with the baselines for the other datasets. As shown in the table, approaches like LfF and DFA reduce accuracy on aligned points while attempting to improve accuracy on conflicting points for BFFHQ dataset. This is likely because these methods focus on enhancing performance in conflicting distribution by reweighing or augmenting minority groups. In contrast, our approach achieves strong performance on conflicting points without sacrificing in-distribution accuracy.

W3/Q2. The learning dynamics of decodability throughout training are interesting, but based only on the simpler datasets with a more drastic difference in the “complexity” of the core and biasing attributes. It is unclear how these trends play out for more realistic data.

A. Please refer to the global response.

W4. Minor corrections.

A. We thank the reviewer for these minor corrections. We will address them in the final version of the paper.

[A] Liu et al. (2021). Just Train Twice: Improving Group Robustness without Training Group Information, ICML 2021.

[B] Wang et al. (2022). On Feature Learning in the Presence of Spurious Correlations, NeurIPS 2022.

We thank the reviewer for the comments and questions.

W1/Q1. Why not build a shallower biased branch?

A. TL;DR: Both core and spurious features are indeed available at shallower depths. However, since we do not use explicit bias annotations, we have no a priori way of telling them apart. So, we aim to segregate them automatically. We observe (in Theorem 2) that although both core and spurious features are available at shallower depths, at greater depths, only spurious features are available, while core features are not. So, deep networks provide us with a unique way to identify spurious features (by looking at the features that survive to greater depths), something that is not possible with shallow networks.

Longer answer: In Theorem 2 (via Lemma 2), we show that the propagation probability $\pi_i$ is inversely proportional to the rank of an attribute $r_i$. Specifically, for any two attributes $a_1$ and $a_2$ with ranks $r_1$ and $r_2$ respectively, if $r_2 > r_1$, then at any given depth $d$, $a_1$ is more likely to propagate through depth $d$ than $a_2$. Simply put, attributes with a lower rank would propagate to a greater depth than those with a higher rank.

Since spurious attributes are typically of a much simpler nature (say, color, texture, etc.) relative to core attributes (such as shape), they are more likely to lie on a lower dimensional manifold, effectively having a lower rank. Combining this with Theorem 2, we thus have that for deep networks, spurious attributes survive to greater depths than core attributes. For this reason, deep networks give us a unique way to segregate bias and core attributes that shallow networks cannot offer since both bias and core attributes can survive in the early layers (shallow network) due to their higher propagation probabilities (stemming from higher effective ranks (Huh et al., 2023)), but only spurious / bias attributes can survive up to the later (deeper) layers.

W2/Q2. Does the approach need to be multi-staged?

A. It is possible to train the deep model, shallow model, and target model in a single stage. In each iteration, the deep and shallow models can be updated using a product of experts (PoE) approach, followed by updating the target model through distillation within the same iteration. We believe that this approach may be less stable than the two-stage method. This instability could arise because the segregation of the deep and shallow models as experts in bias and core attributes, respectively, would not have been established in the early epochs. Consequently, the target model may receive mixed signals during the initial iterations. While parallel training is feasible, it introduces the additional challenge of stabilizing the training dynamics of the target model. Also, training and backpropagating through all three networks simultaneously demands significant GPU memory resources which can be reduced using multiple stages of training. Due to time constraints, we could not experiment with DeNetDM in this setting, so the single-stage approach could perform better or worse than anticipated.

W3. Would deeper networks show similar tendencies toward spurious features with spectral decoupling?

A. Thanks for the suggestion, which led to valuable insights. To test the hypothesis that “depth captures biases” under debiasing regularizers like spectral decoupling, we performed ERM on the CMNIST dataset with both deep and shallow networks. We found that while accuracies on bias-aligned points remained near 100%, spectral decoupling(SD) improved accuracies on bias-conflicting points more significantly for the deep network than the shallow network (see "Reference to Fig. A in the PDF attached to global response" for results and the best bias-conflicting accuracies in the table below).

This indicates that the deep network, being more prone to capturing bias, benefited more from SD in terms of debiasing and accuracy on bias-conflicting points. The shallow network, already focusing on core attributes, saw only marginal improvement. This supports our claim that deeper networks are more bias-prone than shallower ones.

Additionally, the deep network with SD slightly outperformed the shallow network. We hypothesize that SD alleviates rank bottlenecks in deep networks, improving the propagation of core attributes while leveraging the network's larger capacity. This suggests that a deeper network with SD can be both bias-free and more capable, a phenomenon warranting further theoretical exploration.

W4. Generalization to other tasks.

A. We evaluated the performance of DeNetDM on the CivilComments dataset [A], which involves natural language debiasing. The task requires classifying online comments as toxic or non-toxic, with labels spuriously correlated with mentions of certain demographic identities. As shown in Table 1, our approach performs comparably to state-of-the-art methods. Due to the constrained rebuttal timeline, we just applied our model out of the box, without any reasonable hyperparameter tuning. This was just to illustrate its applicability to domains beyond vision. We believe that with further hyperparameter tuning, we would be able to even surpass the marginal differences that we have with SOTA on CivilComments.

[A] WILDS: A benchmark of in-the-wild distribution shifts, ICML 2021.

I thank the authors for the response. One follow-up I have is:

W1/Q1. Is it necessary to use explicit bias annotations to tell apart the core and spurious features? Existing works [1,4] use bias amplification to segregate the spurious features.

Apart from that, I find the responses for Q2, W3, W4 to be satisfactory. I think the additional results will strengthen the paper. I

[1] Shrestha, Robik, Kushal Kafle, and Christopher Kanan. "Occamnets: Mitigating dataset bias by favoring simpler hypotheses." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022. [4] Nam, Junhyun, et al. "Learning from failure: De-biasing classifier from biased classifier." Advances in Neural Information Processing Systems 33 (2020): 20673-20684.

Thank you for your feedback. We are glad that we could address most of your concerns. We will add the additional results to the final revision of the paper.

W1/Q1: As the reviewer rightly pointed out, bias amplification is a well-established method for addressing spurious features without needing explicit bias annotations, as shown in [1], and [4]. However, our approach leverages depth modulation as an alternative strategy for debiasing. Specifically, our deeper branch amplifies bias via depth modulation, a concept grounded in our theoretical insights. Our method also eliminates the need for explicit bias annotations to distinguish between core and spurious features, as the deep and shallow branches inherently act as biased and debiased models, respectively. Furthermore, we demonstrate that DeNetDM outperforms existing approaches like LfF, where bias amplification is achieved using GCE loss.

We hope this clarifies your concern.