Fairness Under Demographic Scarce Regime

We thank the reviewer for detailed comments and constructive feedback.

Despite the fact that the scenario discussed in this paper is not entirely new, I would still suggest the authors provide a real example scenario when D1 has no demographic [...]

We have added in the problem formulation references to real-world examples where this setup applies. In particular, they can be found in healthcare [1, 2] or finance [3, 4] where patients/users self-report their sensitive information.

Q1: Results of "Ours" in Figure 2 were tuned for uncertainty threshold over [0.1, 0.7]. When you tune the threshold, are you using a validation set from the 0.7 training data or are you just selecting the best performing threshold on the 0.3 test data?

Thank you for bringing this up. The uncertainty threshold is a hyperparameter that can be tuned on the validation set. We used 10% of the training data for validation. We highlighted in the paper how we tuned the threshold. Furthermore, we observe the higher variation of the uncertainty threshold occurs in datasets with already higher uncertainty (e.g., LSAC and New Adult), from which models without fairness constraints provide lower bias. There are similar variations in the Pareto fronts over the validation set and the best-performing finetuned threshold also transfers to the test set.

Q2: In Section 3 Problem formulation, you mentioned: " However, to be able to estimate bias in label classifier f, we assume there exists a small set of samples drawn from the joint distribution X × Y × A, i.e., samples that jointly have label and demographic information." How was this reflected in the experiments? Does it refer to the 0.3 test data you used in the experiments? If so, I do not think this is a hard requirement in real world applications.

The joint distribution X × Y × A refers to test data used in the experiments. This assumption allows us to report the true fairness violation. Moreover, we could alleviate this assumption by using proxy-sensitive attributes for the evaluation, which might be a more realistic setup. But these proxies are likely noisy and there is a risk of overestimating or underestimating the true disparities in the model. We have added a limitation section where we discuss the limitation of such assumption in real-world applications and point out the possibility of using of bias estimation approaches that effectively capture the true disparities when the proxy demographic information is used [5] [6].

Q3: Please clarify: in Section 4.2, "the label classifier with fairness constraints is trained on a subset D′1 ⊂ D1". Does this mean the second phase classifier does train on any information except for D'1? Is that possible to train that label classifier on the entire training set D1 with fairness constraints on D'1--- fairness loss will be 0 for data do not belonging to D'1? This could potentially increase the accuracy of the label classifier when uncertainty threshold is low.

Great suggestion. The main objective of the paper is to provide evidence for the hypothesis we formulated. Modifying fairness-enhancing algorithms to account for the uncertainty is an interesting direction to explore. We have mentioned in the conclusion a similar perspective for future work where the fairness-enhancing model is trained on samples weighted by the uncertainty of predicting their sensitive attributes. Please note that all the changes in the revised version of the paper have been highlighted in blue.

We hope we have addressed most of your concerns. Please consider increasing the score if you find our responses satisfactory. We would be happy to answer any further questions.

[1] Brown, D. P., Knapp, C., Baker, K., & Kaufmann, M. (2016). Using Bayesian imputation to assess racial and ethnic disparities in pediatric performance measures. Health services research, 51(3), 1095-1108.

[2] Fremont, A. M., Bierman, A., Wickstrom, S. L., Bird, C. E., Shah, M., Escarce, J. J., ... & Rector, T. (2005). Use of geocoding in managed care settings to identify quality disparities. Health Affairs, 24(2), 516-526

[3] Zhang, Y. (2018). Assessing fair lending risks using race/ethnicity proxies. Management Science, 64(1), 178-197.

[4] Silva, G. C., Trivedi, A. N., & Gutman, R. (2019). Developing and evaluating methods to impute race/ethnicity in an incomplete dataset. Health Services and Outcomes Research Methodology, 19(2-3), 175-195.

[5] Chen, J., Kallus, N., Mao, X., Svacha, G., & Udell, M. (2019, January). Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the conference on fairness, accountability, and transparency

[6] Awasthi, P., Beutel, A., Kleindessner, M., Morgenstern, J., & Wang, X. (2021, March). Evaluating fairness of machine learning models under uncertain and incomplete information. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency.

I appreciate your response addressing my concerns.

Linked to W1, I acknowledge that legal regulations mandate discrimination-free decision-making while simultaneously prohibiting the use or collection of demographic information. However, I find this justification insufficient for supporting a practice that is both illegal and ethically dubious, involving the inference of sensitive information from individuals who have chosen not to disclose such details. Particularly considering the wealth of contributions in the field of differential privacy, it seems plausible to explore alternative approaches. I concur with reviewer PEk1 that the current approach appears somewhat basic, and the incorporation of privacy-preserving techniques could significantly enhance the paper's potential.

Concerning W2, I believe the comparison of the proposed method against state-of-the-art methods should be prominently featured in the main text to underscore the advantages of using this approach over existing methods. This emphasis is especially crucial given the reported results indicating a significant outperformance of the proposed method compared to the considered state-of-the-art approaches. Consequently, the comparisons involving Proxy-kNN and Proxy-DNN could be relegated to the appendix.

Furthermore, I'd like to take this opportunity to provide feedback on a couple of potential improvements in Tables from Appendix D and E. For instance, in Tables 3 and 4, Vanilla emerges as the most accurate method, but it is not highlighted in bold; instead, ARL and your method are highlighted. Additionally, it would be beneficial to maintain consistency in the number of decimals reported in the results. In Tables 5 and 6, most numbers have 3 decimals, but a few have 4 decimals.

We thank the reviewer for their comments and feedback. Below we provide clarifications for the concerns raised.

W1 I believe there's a significant ethical concern in constructing a classifier with the objective of predicting […] Instead, it would be preferable if this classifier incorporated desirable privacy properties, as outlined in Diana et al. (2022)

We acknowledge the ethical concern in inferring sensitive attributes. This represents a general concern for the group of methods relying on proxy-sensitive information. We believe the prediction of sensitive attributes with the objective of mitigating discrimination in high-stakes decision-making scenarios could be ethically acceptable. The inference of the sensitive attributes—using our proposed method or others, should not be used for a purpose different from bias assessment and mitigation. Our method provides insights into the benefit of uncertainty-awareness in predicting sensitive attributes for fairness of the downstream classifiers. Furthermore, the ethical concern in inferring sensitive attributes comes from the dilemma posed between privacy and fairness. In particular, laws or regulations enforce discrimination-free decision-making while they also prohibit the use or collection of demographic information, which is necessary for auditing and mitigating discrimination. While privacy guarantees for the sensitive attributes are out of the scope of the paper incorporating privacy aspects is a suggestion that we welcome. Moreover, methods designed under a privacy-preservation setup generally do not guarantee that an adversary cannot reconstruct the sensitive attributes, especially for methods relying on trusted third parties or secure multi-party computation. For example, [1] shows that one can exploit information about a fair model to reconstruct the sensitive attributes, even with black box access. We have added a limitation section in the revised version (please see Appendix A) where we discuss ethical concerns in inferring sensitive information.

W2 I find the comparison with respect to the state of the art to be lacking. The attribute classifiers chosen in this work, such as Proxy-kNN and Proxy-DNN, are rather simplistic and not well-documented in existing literature [...]

Some of the baselines considered also use proxy-sensitive attributes (Zhao et al., 2022; Liang et al., 2023; Yan et al., 2020;), and methods without relying on sensitive attributes such as ARL (Lahoti et al., 2020) can also improve group metrics such as equalized odds. Other baselines are considered for a comprehensive evaluation and we highlighted that the experimental setup might be different, in particular for methods targeting worst-case group improvement. We couldn’t perform comparisons with Diana et al, (2022) as the authors did not publish their code and we couldn’t reproduce their results within the short rebuttal period. Moreover, Awasthi et al. (2021) focused on postprocessing techniques for equalized odds, which generally highly impact accuracy, while we considered inprocessing techniques with controllable fairness-accuracy tradeoffs. Furthermore, the results reported using true sensitive attributes represent the optimal expected baseline for fairness, and comparison shows our method can achieve similar and in most cases better tradeoffs. It does not seem to be the case for Awasthi et al. (2021) and Diana et al, (2022) on the same datasets.

W3 The authors assert in the abstract that 'our framework outperforms models trained with constraints on the true sensitive attribute,' referring to the results from Figure 2. However, this result only considers a single [...]

Thank you for the suggestions. We have updated the statement accordingly and included privacy-related approaches in the related work section in the revised version. We have also highlighted in the limitation section and emphasized the need for further studies to explore whether our method and hypothesis extend to other fairness-enhancing methods, e.g., preprocessing or post-processing techniques.

[1] Ferry, J., Aïvodji, U., Gambs, S., Huguet, M. J., & Siala, M. (2023, February). Exploiting Fairness to Enhance Sensitive Attributes Reconstruction. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)

I appreciate your additional clarifications regarding Figure 3 and the superiority of your results, and thank you for rectifying my misunderstanding on Table 6. Furthermore, I find it noteworthy that Ours(uncertain) yields promising results.

I have an additional question: You assert that the superior performance of your results might be attributed to the utilization of a highly potent fairness-enhancing intervention (exponentiated gradient). Nevertheless, the remaining baselines employ logistic regression as their foundational classifier, a relatively simplistic approach that may not be the most suitable choice for these classification tasks. I'm inclined to think that this setup doesn't facilitate a truly fair comparison. Have you explored the use of other base classifiers such as SVM or even some straightforward neural networks?

Dear Reviewer EiNj,

Thank you for your time and constructive comments. We hope our last response provided a clarification to your question. As per your suggestion, we explored the use of a different base classifier such as a feedforward network. We could not consider SVM as all of the methods considered for comparison do not support SVM. We trained all the baselines using a multi-layer perception of one hidden layer with 32 units. We evaluated each method on the Adult dataset under the same setup to analyze its impact on the comparison. The results on other datasets are not provided due to the limited time and computational resources. The newly added Table 7 in the appendix shows that our method can still outperform other baselines under a more complex non-linear base classifier. We observed an increase in the accuracy of other baselines due to the increased capacity of the neural network, while our method still provides a significant gap in fairness.

We will be happy to address any remaining concerns within the remaining short time for discussion. We would like to ask you to kindly consider adjusting your score if you find that our discussion addressed your concern.

We thank the reviewer for the feedback on our work. Below we address the concerns.

Can authors clarify if there is anything I misunderstood about the technical contribution? Note that there is no point in merely repeating the details of the method. The constructive communication would be to point out if I am wrong when I say the method is just training a proxy classifier with transfer learning and thresholding predictions.

We apologize for the confusion. Our contribution is indeed more than just training a proxy classifier with transfer learning and thresholding. Our proxy classifier is trained with uncertainty awareness and not transfer learning. The threshold is later applied on the uncertainty of predictions and not on the predictions themselves. The teacher network has the same architecture as the student network and the teacher’s weights are updated using the moving average of the student’s weights. We use Monte-Carlo Dropout over the teacher for uncertainty estimation and the consistency loss to enforce the classifier to focus on samples with low uncertainty.

To the best of our knowledge, our proposed method is the first to draw connections between the uncertainty in sensitive attribute predictions and fairness. We have highlighted our contribution in the revised version of the draft.

Can authors explain the reason why no comparison to any of the methods in the literature of fairness without full access to sensitive attributes?.

We have indeed performed comparisons with six other baselines. In the experiment section, we mentioned the comparison with other baselines addressing fairness issues in a similar setup and we referred the reader to the appendix for results. We performed comparisons with the following baselines:

[1] Yan, S., Kao, H. T., & Ferrara, E. (2020, October). Fair class balancing: Enhancing model fairness without observing sensitive attributes. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management

[2] Zhao, T., Dai, E., Shu, K., & Wang, S. (2022, February). Towards fair classifiers without sensitive attributes: Exploring biases in related features. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

[3] Liang, Y., Chen, C., Tian, T., & Shu, K. (2023). Fair classification via domain adaptation: A dual adversarial learning approach. Frontiers in Big Data, 5, 129.

[4] Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., ... & Chi, E. (2020). Fairness without demographics through adversarially reweighted learning. Advances in neural information processing systems

[5] Hashimoto, T., Srivastava, M., Namkoong, H., & Liang, P. (2018, July). Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning

[6] Levy, D., Carmon, Y., Duchi, J. C., & Sidford, A. (2020). Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems

These comparisons are presented in the appendix due to the page limit and the main objective of the proposed method, which is to demonstrate the utility of uncertainty awareness in the proxy classifier in improving fairness-accuracy tradeoffs in the downstream classifiers. The results presented in the appendix (Tables 3-6) also show the effectiveness of the method in providing better fairness-accuracy tradeoffs compared to the considered state-of-the-art baselines. Moreover, our evaluations are performed over a testing set with the true sensitive attributes, i.e., we report the true fairness violation. We do not aim to improve bias estimation when proxy attributes are used which is why some of the baselines [2] [3] [4] proposed by the reviewer were not considered for comparison.

We hope our response clarifies the reviewer’s concerns and we will be happy to provide further clarifications where needed.

Dear Reviewer PEk1,

We Thank you for your time and for the constructive discussion, which has helped us to improve the manuscript. We have integrated all the suggestions you requested throughout our discussion and we will be happy to provide further clarifications and improvement where needed within the remaining rebuttal time. In case we have addressed all the concerns, we would like to request if you can adjust the score accordingly.