Fairness Improves Learning from Noisily Labeled Long-Tailed Data

Thanks for your time for reviewing our work! Please refer to our following responses regarding your insightful suggestions!

Weaknesses 1:

[Table 3: empirical improvements seem marginal and lack confidence intervals or pairwise statistical significance tests.]

Response:

Why there is no statistical test in Table 3

Our main results are summarized in Table 1, whose statistical tests are given in Table 2. Only Table 3 is not provided with hypothesis testing, since our aim for giving Table 3 is the ablation study – we want to show $\lambda>0$ generally improves learning from the noisily labeled long-tailed data. What is more, Clothing 1M is not a standard benchmark like MNIST/CIFAR and it has a single realization.

New highlight criterion

We recognize the need for a more robust method to highlight significant improvements. In light of your suggestion, we plan to revise our highlighting criteria. In the revised manuscript, we will only highlight cells in Table 3 where the increase in test accuracy is greater than 0.1. We believe this threshold represents a substantial difference, particularly in large-scale datasets such as ImageNet and Clothing-1M. This change aims to provide a clearer and more meaningful distinction of the results that are statistically significant.

Weaknesses 2:

[It's not clear whether access to group membership is useful given that the work is not targeting fairness improvements. To target the noisy long-tail sub-populations couldn't we just up-weight samples with higher loss and down-weight samples with lower loss? (an approach that is also quite common in the fairness literature). What is the intuition behind the use of group membership supposedly leading to better performance?]

Response:

Addressing Sample-wise Reweighting In our previous research, we have observed that in scenarios involving long-tailed data with label noise, samples with small losses often correlate with major populations and clean labels. Conversely, samples with large losses could either belong to minor populations or be incorrectly labeled. Therefore, simply up-weighting high-loss samples could inadvertently amplify the influence of mislabeled data, leading to skewed model predictions. This insight influenced our decision to not solely rely on sample-wise reweighting.

Explaining the Absence of Class-level Consideration

We also steered clear of class-level reweighting, primarily due to the noisy nature of class labels in our dataset. Given this uncertainty in labeling, class-level adjustments could potentially introduce additional biases.

Justifying Sub-population Level Approach

Our focus on sub-population levels stems from their likelihood to be feature-dependent, circumventing the need to evaluate class or sample-wise performance directly. We aimed to illustrate this concept through Figure 1, which demonstrates how head sub-populations generally benefit from existing methods, while tail sub-populations could see improvements with the implementation of a fairness regularizer.

Clarifying the Novelty of Our Approach

Unlike the traditional fairness-accuracy tradeoff in resource-constrained settings, our findings, as presented in Figure 2 of the main paper, suggest that current methodologies for handling long-tailed noisy data fail to achieve Pareto optimality. This results in significant performance discrepancies across different classes/populations, hinting at an unfair distribution of model performance. We hypothesize that robust solutions might disproportionately affect sub-populations. Section 3 of our paper delves into this hypothesis by examining the influence of sub-populations on test data, revealing distinct impacts on model performance when dealing with noisily labeled long-tail data.

Presenting the Fairness Regularizer

Motivated by these observations, Section 4 introduces our Fairness Regularizer (FR). This novel approach is designed to prevent overemphasis on specific sub-populations and to minimize performance gaps across different groups. This is particularly crucial in the context of long-tailed, noisily labeled data, where certain populations might be significantly overlooked without fairness constraints. Our empirical studies (Section 5) and theoretical analysis (Appendix A) support the assertion that implementing FR can enhance the performance of tail sub-populations with minimal impact on head sub-populations.

Theoretical Underpinnings in Appendix A

In Appendix A, we provide a theoretical exploration of the effects of sub-populations in learning with long-tailed noisy data, using a binary Gaussian example. Our theorem suggests that by enforcing fair performance across specific sub-populations (i.e., equal training error probabilities), a classifier trained on noisy data with a fairness regularizer can approximate the optimal classifier for the clean data distribution.

We continue our response as below:

Here is a brief overview of the additional related works we intend to incorporate:

• Pre-Processing Techniques: These methods, like reweighing [1] and optimized pre-processing [2], aim to reduce bias in training data by adjusting training instance weights and modifying feature values for balanced representation.

• Post-Processing Techniques: Techniques such as those proposed by Hardt et al. (2016) [5] adjust model predictions after training to achieve fairness objectives, often focusing on equal error rates across groups.

• In-Processing Techniques: Central to our research, these strategies involve modifying the learning algorithm itself. Examples include fairness constraints within the optimization objective [3] and adversarial debiasing [4].

• Our proposed Fairness Regularizer aligns with in-processing methods but is unique in its pursuit of optimal performance on clean and balanced test data. It aims for a balance in performance across sub-populations, diverging from reductionist approaches like Agarwal et al. (2018) [6] by prioritizing overall efficacy alongside fairness. We hope this additional context addresses your concerns. We are also open to suggestions for any important related works in fairness machine learning that we may have overlooked.

References

[1] Kamiran, F., & Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1), 1-33.

[2] Calmon, F., Wei, D., Ramamurthy, K. N., Varshney, K. R., & Rosa, A. (2017). Optimized pre-processing for discrimination prevention. Advances in Neural Information Processing Systems, 30.

[3] Zafar, M. B., Valera, I., Gomez Rodriguez, M., & Gummadi, K. P. (2017). Fairness constraints: Mechanisms for fair classification. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS).

[4] Zhang, B., Lemoine, B., & Mitchell, M. (2018). Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society.

[5] Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems, 29.

[6] Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H. (2018). A reductions approach to fair classification. In International Conference on Machine Learning (ICML).

Weaknesses 3:

[I believe the points in the lower-left corner of Figure 2 and 6 are the underrepresented groups. However, Figure 6 and Figure 2 could provide a more explicit message to the reader by adding one more axis of information (e.g., the intensity of the color of the points) with the group representation.]

Response:

Thanks for this insightful suggestion. We will revise the figure by adding one more axis of information with the group representation

Question 1:

[Could the authors please highlight the main differences between FR and the reductions approach [1]?]

Response:

Please refer to our response to weakness 1.

Question 2:

[Why is the proposed method preferred compared to other fairness (statistical parity) improvement methods?]

[1] Agarwal et al. A Reductions Approach for Fair Classification. 2018. [2] Lowy et al. A Stochastic Optimization Framework for Fair Risk Minimization. 2022

Response:

Thank you for raising this insightful question. While the solutions proposed by Agarwal et al. (2018) and Lowy et al. (2022) are indeed efficient in targeting fair risk minimization, our approach diverges in its primary objectives. Our research is not focused on achieving fairness per se, but on demonstrating how fairness measures can be instrumental in learning from noisy, long-tail data. The goal is to enhance the model's overall performance, which incidentally also helps in reducing disparities across groups.

Furthermore, our methodology draws inspiration from one of the most recognized in-processing fairness solutions—implementing constraints based on fairness definitions. However, unlike these solutions, our emphasis is not solely on attaining the highest level of fairness but on leveraging these fairness measures as a means to improve overall model performance. This subtle but significant shift in focus distinguishes our method from the ones mentioned and aligns it more closely with our research objectives.

We appreciate your interest in our approach and hope this clarification helps in understanding the distinct positioning of our method in the landscape of fairness improvement strategies.

Thanks for your time for reviewing our work! Please refer to our following responses regarding your insightful suggestions!

Weaknesses 1:

[The proposed method, Fairness Regularizer, is similar to the reductions approach for statistical parity [1]. But I didn’t find a citation for the paper.]

Response:

Thanks for the great suggestions. In our revision, we will explicitly distinguish between our proposed Fairness Regularizer (FR) and the reductions approach for statistical parity, as outlined in Agarwal et al. (2018).

To briefly clarify, while both methods share a common goal of promoting fairness, they differ significantly in their implementation and outcomes. Traditional reduction approaches, including the one mentioned in your reference, predominantly focus on achieving statistical equality across different groups. This is often achieved by modifying the training process, specifically by addressing the Lagrangian:

$\mathbb{E}\_{(X, \widetilde{Y})\sim \widetilde{\mathcal{D}}}[\ell(f(X), \widetilde{Y})] +\sum\_{i=1}\^{N}\lambda\_{i} (\text{Dist}\_{i}-\delta),$

which aims to offer an alternative solution to the original optimization problem presented in Equation (2):

$\min\_{f: X\rightarrow [K]} \mathbf{E}\_{(X, \widetilde{Y})\sim \widetilde{\mathcal{D}}} [\ell(f(X), \widetilde{Y})], \qquad s.t. \qquad \text{Dist}\_{i}\leq \delta, \forall i \in [N], \text{[Eqn. (2)]}$

In contrast, our FR approach, while superficially similar, diverges in its core methodology. Specifically, it entails optimizing the Lagrangian in Equation (2) with uniform $\lambda_i=\lambda$ and $\delta=0$. This distinction is crucial as it results in different optimal solutions. Our approach, aimed at enhancing overall performance on clean and balanced test data, strategically avoids overemphasis on the worst-performing sub-populations. Unlike reduction approaches that strive for strict fairness across all sub-populations, our method prioritizes a more balanced improvement across the board.

We will ensure to elaborate on this point more clearly in our revised manuscript, including appropriate citations to Agarwal et al. (2018) to acknowledge their foundational work in this area. Thank you once again for your valuable feedback, and we look forward to incorporating these clarifications in our revision.

[1] Agarwal et al. A Reductions Approach for Fair Classification. 2018.

Weaknesses 2:

[There are multiple existing in-processing methods to ensure fairness (statistical parity) across groups in the population check [2] Table 1 for multiple references. The authors should, ideally, compare their results with other fairness improvement methods. At least, the paper should mention other fairness improvement methods and discuss the difference between the proposed and existing approaches.]

Response:

Thank you for your valuable feedback. We appreciate your insights and agree that a comparison with existing fairness improvement methods would enrich our paper. Please allow us to clarify our approach in light of your comments.

Clarification on Our Focus: Our primary objective is not solely to achieve fairness, but to demonstrate how fairness measures can enhance learning with noisy long-tail data, thereby improving the model's overall performance while reducing disparities across groups. Our method draws inspiration from popular in-processing fairness solutions, particularly those involving fairness definitions and constraints. However, our aim is distinct: we focus on enhancing overall performance rather than solely achieving the highest level of fairness.

Discussion on Methodological Differences: We recognize the importance of situating our work within the broader context of fairness in machine learning. To this end, we will include a detailed discussion of various fairness improvement methodologies in our revised manuscript. Here is a brief overview of the additional related works we intend to incorporate:

[More in Part 2 responses~]

Thanks for your time for reviewing our work! Please refer to our following responses regarding your insightful suggestions!

Weaknesses 1:

[How to tune lambda ? Is there any recommendation or theory to support this?]

Response:

Thank you for your insightful query regarding the tuning of $\lambda$. We appreciate the opportunity to clarify this aspect of our study.

In our research, we did not focus on providing a theoretical basis for $\lambda$ tuning, primarily because our approach does not necessitate strict fairness across sub-populations. In our experimental framework, we opted for a straightforward approach by selecting constant $\lambda$ values. This decision was driven by practical observations, where we found that using a constant $\lambda$ generally yields a simple yet effective means to manage the performance disparity.

For a more comprehensive understanding, we included an ablation study of $\lambda$ in Table 3, utilizing the Clothing 1M dataset. In this study, we explored various $\lambda$ values, specifically chosen from the set {0.0,0.1,0.2,0.4,0.6,0.8,1.0,2.0}. It’s important to note that a $\lambda$ value of 0.0 corresponds to the baseline methods' training without the application of our Fairness Regularizer (FR).

Our findings suggest that the default setting for FR ($\lambda=1.0$) consistently achieves competitive performance compared to other $\lambda$ values. This observation supports our recommendation to adopt $\lambda=1.0$ as a practical default in similar applications. Additionally, our results indicate that $\lambda$ values close to 1.0 generally outperform those nearer to 0.0. This trend not only underscores the effectiveness of our proposed fairness regularizer but also suggests a relative insensitivity of the model to hyper-parameter variations near the chosen $\lambda$ value.

Suggestion 1:

[Some theory presented in the appendix can be moved into main content]

Response:

Thanks for the suggestion. In our revision, we will consider summarizing main insights of our theoretical results in the appendix into the main content!

Question 1:

[Can the author make some comments on the distribution shift problem? If some sub-population in the test data does not appear in training data, how to guarantee its performance?]

Response:

This is a great question! When certain subpopulations $G_i$ does not appear in the training data, to improve the performance of $G_i$ at test time, it makes sense to assume that we have a held-out validation set for model selection/calibration. In that case, we could perform some post-hoc treatments on model weights scaling at test time, such weights could be inspired by referring to a validation set. Meanwhile, we tend to not intervene in the training process (except for FR) for the sake of fitting well on the existing sub-populations.

Thanks Reviewer yYZa for the efforts and time in reviewing our paper! Please see our response below regarding your concerns and mentioned weaknesses.

Weaknesses 1:

[The authors need to emphasize the differences and connections between dealing with long tail problems on noisy data and dealing with long tail problems under clean data (e.g., problem formalization or techniques).]

Response:

Regarding Problem Formalization: We appreciate the reviewer's point on differentiating between long-tail problems in clean versus noisy data contexts. In our perspective, addressing long-tail issues in clean data can be considered a subset of the broader challenge posed by noisy data. This relationship becomes apparent when considering the transition from clean to noisy labels, represented as $T_{i, j}(X)=\mathbb{P}(\widetilde{Y}=j | Y=i, X)$. In this framework, the scenario with clean data emerges when $T(X)$ simplifies to an identity matrix.

Concerning Techniques For illustrative purposes, we refer to the classical setting with class-dependent transition probabilities ($T_{i, j}(X)=T_{i, j}$, irrespective of $X$). Here, the learning or optimization tasks differ significantly between clean and noisy label scenarios. In the clean label context, the objective is to minimize:

$\mathbf{E}_{(X, Y)\sim \mathcal{D}} \left[\ell(f(X), Y)\right],$

which is equivalent to

$\sum_{i} \mathbf{E}_{X} ~ \mathbb{P}(Y=i) \left[\ell(f(X), Y=i)\right].$

In contrast, with noisy labels, the objective function shifts to minimizing:

$\mathbf{E}_{(X, \widetilde{Y})\sim \widetilde{\mathcal{D}}} ~\left[\ell(f(X), \widetilde{Y})\right].$

The objective under the label noise setting is equivalent to:

$\sum_j \mathbf{E}_{X} ~\mathbb{P}(\widetilde{Y}=j) \left[\ell(f(X), \widetilde{Y}=j)\right].$

Leveraging the transition matrix, we could then derive:

$\sum_j \sum_i \mathbf{E}_{X} ~\mathbb{P}(\widetilde{Y}=j|Y=i) \mathbb{P}(Y=i) \left[\ell(f(X), \widetilde{Y}=j)\right],$

It is worthy noting that the impact of class-imbalanced distribution is indicated in the formulation as well (varied $\mathbb{P}(Y=i)$). As we can see the differences, with the presence of label noise, the technique differences lie in how to further mitigate/correct the impact brought by $T_{i, j}=\mathbb{P}(\widetilde{Y}=j|Y=i)$.

Weakness 2:

[Why observation 3.1 holds？Specifically, if prediction model trained on noisy data has larger variance than the prediction model trained on clean data, the same pattern will be also observed. Therefore, observation 3.1 depends on the original variance of the prediction model. In addition, if we did not remove any sub-population and draw an Acc plot for clean data and noisy data, can we observe the similar pattern in Figure 4? Similar arguments are also held for observation 3.2.]

Response:

Thank you for your insightful comments and queries regarding Observation 3.1 in our manuscript. We appreciate the opportunity to clarify these points and ensure the accuracy of our findings. Firstly, we acknowledge your perspective on the potential variance of prediction models trained on noisy versus clean data. However, the focus of Observation 3.1 and Figure 4 in our study is not on the variance of model predictions but on the performance disparity, specifically the test accuracy drops at the sub-population level. Our analysis is centered on comparing the following metrics：

$\text{Acc}\_{\text{p}}(\mathcal{A},S, i, j)$ v.s. $\text{Acc}\_{\text{p}}(\mathcal{A}, \widetilde{S}, i, j)$.

In Figure 4, the broader distribution of the box plot for $\text{Acc}\_{\text{p}}(\mathcal{A}, \widetilde{S}, i, j)$ (pertaining to the noisy-label scenario) as compared to $\text{Acc}\_{\text{p}}(\mathcal{A},S, i, j),$ indicates a more pronounced influence of noisy data, thereby reinforcing Observation 3.1.

Regarding your question about the absence of sub-population removal and its effect on the accuracy plot for clean versus noisy data, we have addressed this in Figure 9 (located in the Appendix, lower right corner). It appears there may have been a misunderstanding regarding the definition of our influence measure depicted in Figure 4, which focuses on the accuracy drop. To elucidate, when calculating $\text{Acc}\_{\text{p}}(\mathcal{A}, \widetilde{S}, i, j)$ for each $i, j$, we assess the following difference:

$\mathbb{P}\_{f \leftarrow \mathcal{A} (\widetilde{S}), (X’, Y’, G=j)} (f(X’)=Y’) - \mathbb{P}\_{f \leftarrow \mathcal{A}(\widetilde{S}^{\backslash i}), (X’, Y’, G=j)} (f(X’)=Y’)$

Here, the upper term represents the scenario where no sub-populations are removed in training. Following your query, if no subpopulation is excluded, the lower term becomes identical to the first, resulting in a net influence of zero.

Weakness 3:

[The problem is handled by simply adding the distance of each class from the "center" to the original loss function, which is simple and lacks some novelty. For example, is it possible to calibrate clean data with noisy data or head with tail data to address this problem?]

Response:

Thank you for your insightful feedback. We appreciate your suggestion to calibrate clean data with noisy data or head with tail data. Indeed, this approach offers an intriguing perspective. However, our decision to not pursue this direction was driven by several pragmatic considerations.

Firstly, implementing such a solution would necessitate additional information, such as clean labels for training samples or a separate, clean validation set. Our study operates under the realistic assumption that these resources are not readily available. Estimating the necessary statistics without these could be particularly challenging in the context of long-tailed data, potentially leading to errors that might compromise robust learning.

Furthermore, our empirical observations highlighted the importance of controlling the performance disparity among different populations. Introducing a fairness regularizer aligned more closely with our research narrative and objectives, offering a more feasible solution under the constraints of our study.

Regarding the simplicity of our method, we believe that simplicity does not detract from novelty, particularly when it effectively addresses the complexities of long-tail distribution and label noise. The "distance" we refer to in our approach is not a traditional $L_{1}, L_{2}$ distance, but rather a measure of the performance gap across sub-populations. To our knowledge, our work is unique in employing fairness regularization to control this gap under such challenging conditions.

Weakness 4:

[How can we choose lambda for each sample in practice when it is not set to a constant?]

Response: Thank you for your insightful question regarding the selection of lambda for each sample in our proposed method. Your query touches on a fundamental aspect of our approach, which we appreciate. In addressing the optimization task presented in Equation (2):

$\min_{f: X\rightarrow [K]} \mathbf{E}_{(X, \widetilde{Y})\sim \widetilde{\mathcal{D}}} [\ell(f(X), \widetilde{Y})], \qquad s.t. \qquad\text{Dist}_i\leq \delta, \forall i \in [N], \qquad \text{[Eqn. (2)]}$

We could find an empirical solution utilizing the augmented Lagrangian form. This approach transforms the constrained problem into a series of unconstrained minimization problems, as detailed in Equation (R1):

$\min_{f: X\rightarrow [K]} \Phi(f_k):= \mathbf{E}_{(X, \widetilde{Y})\sim \widetilde{\mathcal{D}}} [\ell(f\_{k} (X), \widetilde{Y})]+ \mu\_{k} \cdot \sum\_{i\in[N]} g( \text{Dist}\_{i}-\delta), \text{[Eqn. (R1)]}$

where $g(x):=\max(0, x)^2$. The iterative process involves updating the solution from each iteration to serve as the initial guess for the subsequent one. This method ensures that the solutions converge asymptotically to the optimization task defined in Equation (2).

In the augmented Lagrangian form, with $\delta=0$, the formulation becomes:

$\min_{f: X\to [K]} ~\Phi(f_k):=\mathbb{E}_{(X, \widetilde{Y})\sim \widetilde{\mathcal{D}}}[\ell(f\_{k}(X), \widetilde{Y})] + \frac{\mu\_{k}}{2} \cdot \sum\_{i\in[N]}{\text{Dist}\_{i}}^2+ \sum\_{i\in[N]} \lambda\_{i} \cdot \text{Dist}\_{i}$

with $\lambda_i$ updating as $\lambda\_{i}\leftarrow \lambda\_{i} + \mu\_{k}\cdot \text{Dist}\_{i}$, and $\text{Dist}\_{i}$ takes the input classifier $f_k$ given by the solution to Eqn. (R1) at the $k$-th iteration. In such an augmented Lagrangian method, the variable $\lambda$ could be viewed as an estimate of the Lagrange multiplier, and the accuracy of this estimate keeps improving as iteration progresses. We adopt the augmented format instead of the penalty method for the reason that: it is not necessary to take $\mu \rightarrow \infty$ to solve Eqn. (2). And the use of the Lagrange multiplier term allows us to choose smaller $\mu_k$, hence avoiding ill-conditioning.

Regarding the adoption of a constant $\lambda$, our approach stems from the lack of a strict requirement for equal performance across sub-populations. A constant $\lambda$ offers a simpler implementation to manage performance disparities. Our ablation study (referenced in Table 3) supports this choice, demonstrating the efficacy of constant $\lambda$ values in Fairness Regularization (FR).