Removing Multiple Shortcuts through the Lens of Multi-task Learning

We sincerely appreciate your insightful feedback and constructive suggestions that helped improve our paper substantially. All the suggestions will be incorporated, and additional discussions will be added to the main text. Please find our detailed responses to the comments below.

[Weakness 1] Debiased training as MTL: Debiased training actually seems closer to multi-domain learning (MDL) [W1] rather than MTL. Because MTL clearly means learning different tasks (i.e. task labels are different) as the paper also mentions on Page 2.

• On the other hand, MDL involves the same target classes but data coming from different domains, and the goal is to achieve good performance on all domains simultaneously. Also it seems like debiased training has a lot in common with long-tailed learning [W2]. For example, upweighting and upsampling baselines are also employed in long-tailed learning.
• Overall, I wonder why MTL is chosen over these other two. Also, it might be interesting to see how more advanced techniques from MDL or long-tailed learning perform when adapted to debiased training.
• Note: the above two references are just examples, there are many more papers in both sub-topics.

Thank you for interesting and insightful suggestions! As you mentioned, both long-tailed classification and multi domain learning are relevant in mitigating spurious correlation in the context of data imbalances. Based on your comments, we will consider integrating techniques from MDL or long-tailed learning for debiasing in the future work. However, we believe MTL can be more naturally incorporated with debiased training for the following reasons. Debiasing focuses on training an unbiased model with respect to spurious correlations. While MDL addresses data imbalance among different domains and long-tailed classification deals with data imbalance among target classes, debiasing tackles the imbalance among bias attributes within a given target label in training dataset. For instance, in bird species classification, if most waterbirds in the dataset are with water background and ERM erroneously uses the water background as a shortcut for classifying waterbirds, waterbirds and water background have spurious correlation. Therefore, we designed a learning algorithm that causes task conflicts when a model relies on shortcuts by defining tasks according to adherence of samples to spurious correlations using our grouping policy. Adopting MOO to resolve task conflicts is an effective approach for training an unbiased model by removing shortcuts.

[Weakness 2] It is unclear how MultiCelebA is a new dataset compared to CelebA. There are no new images or new labels. Even the choice of attributes is based on the analysis from a prior work. The contribution of a new dataset seems misleading, only a new experimental setting based on an existing dataset is proposed.

Thanks for the comment, and we have replaced “new dataset” with “new benchmark” in the manuscript to address the concern. Despite this change, we still believe that, as a realistic and challenging benchmark, MultiCelebA will contribute substantially and promote future research in the field of debiased training. We also would like to highlight that reprocessed datasets/benchmarks like MultiCelebA have long been recognized by the community; examples include bFFHQ, a dataset for debiasing derived from FFHQ, and those for long-tailed classification such as ImageNet-LT (from ImageNet), CIFAR-LT (from CIFAR), and Places-LT (from Places).

Thanks for your efforts in the discussion period. Some of my concerns have been addressed.

• However, I am still unconvinced by the argument that MTL is more suitable than MDL. I agree that either may be practically used but MDL still seems more intuitive. As per the author response, there is a task conflict when the model relies on spurious correlations, which is what MDL also tries to solve. For example, in a setup with two domains: day and night images, the spurious correlation is time of day and the model may rely on that to make the prediction. Then, it is the domain that is changing and not the task itself, the task of classification stays the same.

Overall, I update my rating to "6: marginally above acceptance threshold".

Thank you for dedicating your time and expertise to review our manuscript. Your insights are invaluable to us. However, it appears that some aspects of the review may have been based on certain confusions regarding our methodology and experimental results. We are eager to address these points in the rebuttal, and will accordingly revise our paper for improving its clarity.

[Weakness 1] While MultiCelebA is interesting and useful, selling it as a "new dataset" is too much.

Thank you for your constructive suggestion! In response to your comment, we have replaced "new dataset" with "new benchmark" in the manuscript. Despite this change, we still believe that, as a realistic and challenging benchmark, MultiCelebA will contribute substantially and promote future research in the field of debiased training. We also would like to highlight that reprocessed datasets/benchmarks like MultiCelebA have long been recognized by the community; examples include bFFHQ, a dataset for debiasing derived from FFHQ, and those for long-tailed classification such as ImageNet-LT (from ImageNet), CIFAR-LT (from CIFAR), and Places-LT (from Places).

[Weakness 2] Saying that this is the first work connecting "unbiasing" with MOO or MTL is arguable. One could argue that even importance-weighting approaches (ex. FairGrad) are already interpreting the problem as MOO.

Thank you for your valuable and insightful feedback! We recognize the widespread use of reweighting methods like GroupDRO in the field. It is important to note that none of such prior work, including GroupDRO, explicitly relates their frameworks with MOO/MTL. However, our work explicitly frames debiasing from a perspective of MOO for the first time, which enables us to develop a new and effective algorithm based on recent advances in MOO. Our algorithm substantially outperformed standard reweighting techniques, and we believe its outstanding performance highlights the impact of viewing debiasing through a lens of MOO explicitly.

Regarding FairGrad (TMLR 2023): According to the ICLR 2024 Reviewer Guidelines, FairGrad, published one month before our submission date, is considered as concurrent work. However, we sincerely appreciate your diligence in bringing this to our attention, and have revised the draft to discuss the paper and compare our method with it. Moreover, we would like to clarify that FairGrad focuses on grouping based on sensitive labels for fairness, a notable distinction from our work that aims to remove spurious correlations by MOO. Despite the existence of FairGrad, we thus assert that our paper is still the first to introduce MOO for mitigating spurious correlations.

[Weakness 3-1] I don't understand what it means to "Finding the Pareto-optimal parameter" and "address spurious correlations based on a theory of MOO". There are many Pareto-optimal parameters. And, Pareto-optimality does not guarantee that not being biased towards a certain group.

Thank you for your perceptive observation. You correctly note that, in typical MTL settings, Pareto optimality does not necessarily guarantee unbiased performance across groups: there is a trade-off between tasks in terms of performance due to the conflicts between them (i.e., different tasks have different goals), and consequently a Pareto optimal model may be biased towards a certain task. However, in our setting, since tasks share exactly the same objective, a single model that works for all the tasks exists in principle (i.e., a model free from spurious correlations); “the” Pareto optimal parameter indicates such a single, ideal model in our paper. We will clarify this in the draft soon!

[Weakness 3-2] MGDA is known to be biased towards tasks with low magnitudes.

MGDA tends to be biased towards tasks with low magnitudes as commented, and such a behavior is in fact beneficial for our framework. We first would like to note that, as demonstrated in Section A.2, our grouping strategy and group-wise loss result in the minority group (CC) with significantly low loss magnitude. This is mainly because (1) all the groups contribute to training equally, regardless of their population, through the group-wise loss, and (2) it is easy to reduce the loss for the minority group due to the low complexity of its data distribution. Hence, MGDA allows our algorithm to focus more on the minority group (with low loss magnitudes) during training and consequently resolve the data imbalance issue between the groups.

[Weakness 3-3] Saying that MoCo is the SOTA of MOO is quite a stretch.

We agree, and have revised the draft to refer to MoCo as "the latest technique for MOO" instead.

[Weakness 5-1] MGDA minimizes in each iteration the regularizer in Eq. 3, and it is biased towards dominated tasks (which is observed in Table 5). However, the proposed approach works well.

First of all, we would like to correct potential misunderstandings: The records of MGDA in Table 5 suggest that it works well for the minority task (CC) but not for the dominant task (GG). Please note that G and C indicate whether a group includes bias-guiding or bias-conflicting samples for each bias type, respectively, as defined in Section 5.1. This is also discussed in our answer to [Weakness 3-2].

[Weakness 5-2] Is the proposed method an interpolation between ERM and MGDA, similar to CaGrad? Or learning $\alpha$ using the parameter?

Our objective is not an interpolation between ERM and MGDA since both the first and the second terms of Eq. (3) involve $\alpha$. Furthermore, $\alpha$ is not a scalar weight but a learnable vector, enabling dynamic adjustment of group-wise importance during training. This is a significant departure from CAGrad, which employs a constant $c$ for interpolation of ERM and MGDA. Please check out Algorithm 1 in the paper, which clearly illustrates how our method utilizes $\alpha$ in a fundamentally different way from interpolation.

[Weakness 5-3] It is rather intriguing that the grouping policy does not work well on its own. Why should the grouping and the training approach not be independent?

We would like to emphasize that our grouping policy is not designed to operate independently. Its purpose is dedicated to facilitating the effectiveness of our training algorithm in mitigating spurious correlations. Our policy groups data according to their adherence to spurious correlations and defines each task for each of such groups so that, if a model is spuriously correlated with a bias type, tasks that adhere to the bias and those that do not adhere conflict with each other. Then mitigating such conflicts by MOO resolves the spurious correlations and leads to an unbiased model consequently. Therefore, the grouping policy and the training strategy are inherently interconnected in our algorithm, each playing a vital role in addressing spurious correlations, as empirically demonstrated in Table 5 and 6 of the paper.

[Weakness 5-4] It is unclear what makes the proposed method work.

The key ideas in our method involve framing the mitigation of spurious correlation as MTL and adapting MOO for debiasing. Specifically, due to the following reason, achieving Pareto optimality under our grouping policy aligns that the model learns a feature space free from spurious correlations.

1. There is an optimal solution that fits perfectly across all groups since they all share the same objective. (Please see our answer to [Weakness 3-1].)
2. If a model falls into shortcuts during training, the model’s performance will be reduced on certain groups, which deviates from Pareto optimality in the context of our grouping policy.
3. Hence, optimization towards Pareto optimality with our grouping policy drives the model towards being unbiased.

[Weakness 4] The arguments towards "finding a flat optima" are rather hand-wavy and unconvincing.

This statement does not imply that a flat minima leads to an unbiased model. Our intention in the paper was that the second term in Eq. (3), which is originally introduced to achieve Pareto stationarity, can be also interpreted as an approximate curvature of the loss, according to previous work [1]. To be specific, the trace norm of hessian tr(H) is known to represent loss curvature, and the trace norm of the empirical Fisher information $tr(F)=\mathbb{E}\left[\left\lVert \boldsymbol{\alpha}^{\top}(\nabla L(\theta))\right\rVert_2^2\right]$, which is equivalent to the second term in Eq. (3), approximates $tr(H)$ [2, 3]. Therefore, minimizing the second term in Eq. (3) encourages to find a flat optima. Also, such a flat optima has been known to improve models’ generalization capability [1, 4, 5, 6]. To further understand the impact of the term on the loss curvature, we have included a new ablation study in Section A.7. The experimental results suggest that a higher weight on this term leads to a smaller loss curvature in the model parameter space.

[1] Xian Li and Hongyu Gong. Robust optimization for multilingual translation with imbalanced data. In Proc. Neural Information Processing Systems (NeurIPS), volume 34, 2021.

[2] Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, and Krzysztof Geras. Catastrophic fisher explosion: Early phase fisher matrix impacts generalization. arXiv preprint arXiv:2012.14193, 2020.

[3] Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. Ernie: Enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223, 2019.

[4] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- ter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In Proc. International Conference on Learning Representations (ICLR), 2017.

[5] Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv:1703.11008, 2017.

[6] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. In Proc. International Conference on Learning Representations (ICLR), 2020.

We sincerely thank you for your response! Based on your comments, we will add the details in the manuscript and address remaining concerns below. We hope this response effectively resolves the remaining concerns.

[Weakness 5-2] I still do believe your method to tune $\alpha$ is an interpolation between solving ERM (first term) and MGDA (second term), controlled by $\lambda$ (instead of c in CAGrad).

Firstly, we would like to clarify that the first term in Eq. (3) is not the ERM but rather the weighted sum of group-wise losses. In contrast to conventional multi-task scenarios that compute all task losses from a single input, we group the dataset and calculate a task (group) loss within the group to which the input belongs. Therefore, the sum of group-wise losses is not equivalent to ERM in our grouping. ERM is a special case of the first term when $\alpha_i$ is proportional to the number of samples in the $i$-th group since the number of samples varies in each group. The first term becomes ERM on MultiCelebA when $\alpha_i$ is $\frac{\text{the number of samples in } i\text{-th group}}{\text{the number of samples in the training set}}$. However, since $\alpha$ is initialized as $[\frac{1}{(\text{the number of groups)}}, …, \frac{1}{(\text{the number of groups)}}]$ and dynamically updated to minimize Eq. (3), the first term is hardly considered as ERM.

Furthermore, we update $\alpha$ by minimizing interpolation with a learnable parameter $\lambda$ between the weighted sum of group-wise losses and MGDA. The weighted sum of group-wise losses serves as a prior for increasing the group weight of the CC group, as detailed in Section A.2. As training progresses and the $\lambda$ increases, this prior intensifies in the early stages of training, and the impact of MGDA gradually strengthens.

We would like to note that CAGrad updates model parameters by minimizing the interpolation between ERM and MGDA, whereas our algorithm minimizes the weighted sum of group-wise losses without MGDA for updating model parameters.

[Weakness 3-1 and 5-4] I still believe the term Pareto-optimality is not fully clear: it does not give any guarantee of the ones claimed in the manuscript and rebuttal. One solution could perfectly fit a group while completely neglecting another, and it would still be Pareto optimal. Indeed, any point in the Pareto front (which can go from balanced solutions to unbalanced ones) is Pareto optimal.

We would like to illustrate this concept with an example. Assume that the task is to classify images in MultiCelebA as either having high cheekbones or not. We categorize the dataset into groups using our grouping policy. All groups have the common training objective of classifying images based on high cheekbones. When considering training a group, the solution space for groups with bias-guiding images can be broader than that of the bias-conflicting groups because they can exploit the spurious correlations to solve the task. The simple example is as follows:

$\text{Solution}\_{GG}=${$\theta_{\text{high cheekbones}}, \theta_{\text{gender}}, \theta_{\text{age}}$}

$\text{Solution}\_{GC}=${$\theta_{\text{high cheekbones}}, \theta_{\text{gender}}$}

$\text{Solution}\_{CG}=${$\theta_{\text{high cheekbones}}, \theta_{\text{age}}$}

$\text{Solution}\_{CC}=${$\theta_{\text{high cheekbones}}$}

In this example, $\theta\_{\text{high cheekbones}}$ is the optimal solution that performs well on all groups. $\theta_{\text{gender}}$ or $\theta_{\text{age}}$ (i.e., the solutions that perfectly fit a group while completely neglecting the other) are not a Pareto optimal because while they decrease the group-wise losses only for bias-guiding groups and increase the group losses for bias-conflicting groups, there exists a better solution $\theta\_{\text{high cheekbones}}$ that decreases the groups losses for all groups.

[Weakness 5-1], [Question 1]

Thank you! We will add the details in the manuscript.

Finally, we sincerely thank you for your dedication and providing generous suggestions and comments until the end of the author-reviewer discussion period. They are indeed of great help that improves our work substantially!

We sincerely appreciate your insightful feedback and valuable suggestions that helped improve our paper substantially. All the suggestions will be incorporated and additional experiments will be added to the main text and the supplementary. Please find our detailed responses to the comments below.

[Weakness 1] No related works on shortcuts. Relevant literature can be found in Discover and Cure: Concept-aware Mitigation of Spurious Correlation Wu et al. ICML 2023.

We appreciate your thoughtful comments. We will refer to them as soon as possible.

[Weakness 2] The paper is not easy to follow. The writing could have been better.

We have made revisions in line with comments from other reviewers. If there are any aspects that remain unclear or if you have any additional questions, please let us know.

[Weakness 3] No code, so limited reproducibility.

In camera ready, we are committed to making our code publicly available. We understand the importance of transparency and community access in research, and we will ensure to adhere to these principles.

[Weakness 4] The major issue with the approach is knowing so many different subgroups a priori. In a more challenging setting, it is almost impossible to know all possible different subgroups beforehand to design the training strategy. I would like see an experiment where out of 3 subgroups the authors include two in their training, leaving one unidentified and how their method performs.

Thank you for your constructive suggestion! We have added a new experimental setting in which only ‘Gender’ and ‘Age’ labels are visible during training on the three biases setting of MultiCelebA . As shown in Table R2 and Table A7, our method significantly outperforms existing methods. Interestingly, GroupDRO and upweighting, all of which utilize bias labels, are outperformed by LfF, a debiasing algorithm that does not use bias labels. These results indicate that our method could be effective in mitigating not only known biases but also unknown biases.

[Table R2] Performance in Unbiased, Worst, and InDist (%) on MultiCelebA in three biases for evaluation two biases for training setting. We mark the best and the second-best performance in bold and underline, respectively.

[Weakness 5] There is a recent notion of difficulty in shortcuts. If the authors are using different subgroups to design multitask losses, the losses should be weighted corresponding to the shortcut difficulty. For example, a hard shortcut loss will be penalized less than an easy shortcut loss, as the mode is more prone to latching on the easy shortcut.

According to the ICLR 2024 Reviewer Guidelines, the paper by Murali et al., published one month after our submission date, is considered concurrent work. We sincerely appreciate your diligence in bringing this to our attention and will revise the draft to discuss the paper and compare our method with it. To clarify, Murali et al. categorize spurious correlation as either benign or harmful, while our goal is to eliminate ‘shortcuts’ stemming from easy-to-learn spurious correlations. We empirically demonstrated that the bias types in MultiCelebA are such easy-to-learn spurious correlations, as shown in Section A.3.

[Weakness 6] This setting will be more realistic: The dataset MultiCelebA is good for evaluation, but I would like to see a more realistic dataset like NIH-chesttube in the shortcut paper in [Weakness 5].

Thank you for your suggestion! However, the metrics used for evaluating models on the NIH-chesttube benchmark differ from those commonly used to evaluate debiasing methods, which makes direct comparisons of debiasing methods on this benchmark challenging. Currently, it is difficult to complete the experiment immediately due to the lack of time, but we aim to include it in the camera-ready if it is completed in time.

We sincerely appreciate your insightful feedback and constructive suggestions that helped improve our paper substantially. All the suggestions will be incorporated, and additional discussions will be added to the main text. Please find our detailed responses to the comments below.

[Weakness 1] The relationship between multiple bias issue and multi-task learning is intriguing. However, the absence of comparisons with traditional multi-task learning (MTL) methods raises a question. If traditional MTL methods can also effectively optimize the problem, it would strengthen the connection between multiple biases issue and multi-task learning.

Thank you for your valuable comments! As you mentioned, the application of an existing MOO/MTL method along with our grouping policy showed improved performance compared to not adjusting group weights, as shown in Table 5, which strengthened the connection between multiple biases and MTL/MOO. However, the methods fall short of the desired level of unbiased performance. This highlights the superiority of our method in scenarios involving multiple spurious correlations. Specifically, in response to your comment, we have included the results of GradNorm in Table 5, as well as in the subsequent Table R1. GradNorm adjusts group weights based on a specific criteria, namely the inverse training rate. However, it has been known that reweighting task losses based on specific criteria can lead to instability [1], as demonstrated by the high standard deviation in Table W1. GradNorm improved the performance on bias-guiding samples, but it showed lower performance on bias-conflicting samples, even compared with that of (i) No adjusting grouping weights. Therefore, under our grouping policy, finding Pareto optimal with MOO is effective in training an unbiased model.

[Table R1] Comparison among different strategies for adjusting $\alpha$

[1] Fernando, Heshan, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, and Tianyi Chen. “Mitigating Gradient Bias in Multi-Objective Learning: A Provably Convergent Stochastic Approach.” ICLR 2023.

[Question 1] GradNorm [2] also adopts the gradient of loss weight to optimize the loss weight. Could you please discuss the relation and the difference between your algorithm and GradNorm? In addition, GradNorm is an important reference for this paper.

Thank you for informing us! Our algorithm updates group weights by minimizing the weighted sum of group losses and the norm of the weighted sum of gradients to perform well on all groups, while GradNorm updates group weights towards inverse training rate so that different tasks train at similar rates. We will add GradNorm in our revised manuscript.

[2] Chen, Zhao, Vijay Badrinarayanan, Chen-Yu Lee and Andrew Rabinovich. “GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks”. ICML 2018.

[Question 2] Could you please conduct additional comparisons with traditional MTL methods to solidify the connection between multiple biases issue and multi-task learning?

Please see our answer to [Weakness 1].

Thank you for dedicating your time and expertise to review our manuscript. Your insights are invaluable to us. We will do our best to address all of them in the revision. Please find our responses to the comments below.

[Weakness 1-1] Rigorous definitions of spurious correlations and the conditions under which they are fully eliminated would benefit readers.

Thank you for your comprehensive suggestion! We will revise our manuscript to include the following content to our paper.

A spurious correlation is a statistical relationship between two variables that appears to be causally related but is actually due to a coincidence. Spurious correlations appear in a majority of training data, but not the majority in testing data. For example, in bird species classification, if most waterbird images in a dataset have water backgrounds, there exists a spurious correlation between waterbird and water backgrounds. A model trained with ERM may erroneously use the water background as a shortcut for classifying waterbirds, leading to misclassification of test waterbird images with no water backgrounds. The model that learns such shortcuts is referred to as a biased model. A biased model well classifies bias-guiding samples (i.e., samples that agree with the spurious correlations) but fails to correctly classify bias-conflicting samples (i.e., those disagreeing with the spurious correlations). On the contrary, when a model learns the target task while avoiding shortcuts, it learns a feature space free from spurious correlations while representing the target task. As a result, the model exhibits unbiased accuracy between bias-guiding samples and bias-conflicting samples.

[Weakness 1-2] The rationale behind the algorithm design remains ambiguous. Although the authors mention that a Pareto-stationary point with a flat loss landscape helps resolve between-group conflicts, the underlying logic is not evident.

In the presence of a biased training dataset, when a model relies on spurious correlations as shortcuts for target classification, it becomes a biased model. This biased model performs well when classifying bias-guiding samples but struggles with bias-conflicting samples. Our policy groups data according to their adherence to spurious correlations and defines each task for each of such groups so that, if a model is spuriously correlated with a bias type, tasks that adhere to the bias and those that do not adhere conflict with each other. These conflicts are analogous to task conflicts in MTL, and MOO has been adopted to resolve task conflicts of MTL. Following this, we proposed a new training objective based on MGDA, which is one of MOO methods, to relieve task conflicts by finding Pareto stationary which is a point where no further improvement can be made in any objective. In our scenarios, since all tasks aim to solve the same target classification, there in principle exists an optimal solution i.e., a feature space free from spurious correlations, and fits perfectly across all groups. Therefore optimization towards Pareto optimality with our grouping policy drives the model towards being unbiased.

[Weakness 2] The MultiCelebA dataset appears inadequate in distinguishing between spurious and non-spurious correlations. The authors have not elaborated on the dataset's construction, even in supplementary materials. Furthermore, while they label certain correlations between target attributes and specific attributes as spurious, these correlations seem to compose of the spurious and non-spurious ones. An isolated evaluation of spurious correlations is essential for gauging the efficacy of methods designed to counteract them. Thus, relying on experiments with the MultiCelebA dataset might be questionable.

We would like to kindly note that Section A.3 and Figure A3 empirically demonstrated that the bias types in MultiCelebA are spuriously correlated to the target class, High Cheekbones. Additionally, Figure 2 and Table A1&A2 present statistics of MultiCelebA. If you have already reviewed these sections and find any aspect unclear or have specific concerns, we would greatly appreciate further clarification on which parts you found problematic.