Beyond Demographic Parity: Redefining Equal Treatment

Dear reviewer, many thanks for the comments. Please consider the main points of our rebuttal. We address specific points below:

I.1. The main concern I had was the novelty of the paper, which I think is not sufficient. Specifically, using model attribution methods such as Shapley values to interpret model unfairness has been explored in prior works; the idea of using the two-sample test to examine the independence of two sets of variables has also been studied. While the settings are not the same, the techniques are somehow similar.

A.1. The intersection of fairness and explainable AI has been an active topic in recent years. We compare our approach with related work in Appendix B3. We would thank the reviewer for any additional paper to be considered.

The paper is not using the ICLR template.

We use the following in the latex preamble: \usepackage{iclr2024_conference}

I.2. It seems that the sources of unequal treatment can only be explained for linear models (as illustrated in Example 4.4 and Figure 3). How can the method be generalized to non-linear cases?

A.2. The approach is not restricted to linear models only. For instance, in Appendix E.3 non-linear models are experimented with. If the ET Inspector is linear, then the explanations are readily available, but, in general, the explanation of a non-linear ET inspector is proposed in the process from Figure 1.

I.3. Since ET can be much more difficult to achieve, can you provide a real example to illustrate why equal treatment is a more appropriate notion than demographic parity?

A.3 The main philosophical critique to the liberalism notion of equal of treatment as blindness, is that it is rarely achieved. Our paper confirms this classical intuition both, theoretically and experimentally.

In Figure 3, we provide cases where Demographic Parity is achieved but Equal Treatment is not, these are cases where demographic parity may flag wrongly a fair ML model. This figure is further extended in table 5 where we compare against distinct methods to measure demographic parity.

Dear reviewer, many thanks for the comment, please consider the main points of our rebuttal above and our specific points that follow now:

I.1. I disagree with some statements that the paper used to motivate the research question. For example, the abstract states “Related work in machine learning has translated the concept of equal treatment into terms of equal outcome and measured it as demographic parity (also called statistical parity)”. To my understanding, it is well-recognized in the fair ML literature that equal treatment and equal outcome are different concepts. While I agree that equal outcome is often measured with statistical measures, I think it is inaccurate to frame equal outcome as a convenient proxy of equal treatment. Instead, one simplified interpretation of equal treatment is “fairness through unawareness” or “colorblindness”. Rather than relying on the distinction between ‘equal outcome vs. equal treatment’, which can refer to much more high-level philosophical differences than what is captured in this paper, I would find it clearer to simply focus on equal treatment (new definition) vs. demographic parity.

A.5.1 We agree with the reviewer. We will compare equal treatment vs demographic parity instead of equal outcomes.

I.2 In Section 4, the theoretical analysis relies on assuming exact calculations of Shapley value are available. How realistic is this assumption in practice? When we do not have access to exact Shapley values, how will the theoretical results be affected?

I.2. The mathematical derivation of Shapley values in real cases, (non-iid data and non linear model), is not feasible. There are related works that focus on to improve the computation of Shapley values in those scenarios:

[1] Sebastian Bordt, Ulrike von Luxburg: From Shapley Values to Generalized Additive Models and back. pdf. AISTATS, 2023

[2] Interventional shap values and interaction values for piecewise linear regression trees, AAAI 2023

I.3. What are other explanation functions that can be used in the framework? In the appendix, another example is given, but I wonder is there a large set of options or is designing an effective explanation function an open question itself? If there are multiple candidate explanation functions, what makes one function better than another?

A.3. In this work, we have focused on feature attribution explanations that are well-developed for tabular data. We have selected LIME and Shapley because they comply with the efficiency property (equation 6), and this is required so the theoretical analysis hold. It is possible that alternative XAI explanation techniques, such as logical reasoning, argumentation, or counterfactual explanations, could be useful and offer their unique advantages. This remains a further work to explore. (This issue is discussed in the “Limitations” paragraph of the Conclusions.)

Dear reviewer, many thanks for the comments, above the main points of our rebuttal and our specific points that follow now:

I.1(a) I agree with the authors that in the case of exact demographic parity (independence), then this definition of equal treatment works (Lemma 4.2). However, my concerns arise in cases where the demographic parity is violated only by a small amount, which is the case in practice; no (useful) algorithm has a demographic parity of exactly zero, and most of the decision making algorithms usually have a small violation tolerance. Can the authors comment how equal treatment can be used on bounding demographic parity, or whether there exists any relationship there? This scenario is important for e.g., credit lending scenarios; in the U.S., the Equal Credit Opportunity Act [2] enforces no discrimination on the outcomes of the decision-making algorithm. From a law standpoint, one might not necessarily mind different explanations as long as the outcomes are not too dissimilar (i.e., low demographic parity).

A.1 Thank you for this comment. Lemma 4.2 shows that Equal Treatment implies Demographic Parity in the case of perfect independence. Apart from the trivial case where the turns out to coincide (Lemma 4.1), we have not explored in the paper bounds on DP given bounds on ET. This is something that we intend to do in future work, as it is a non-trivial property relating multivariate independence (ET) to univariate independence (DP). A promising approach will be to exploit the linearity property of Shapley values.

I.2. (b) First of all, unfortunately using AUC as a test statistic for classifier-two-sample test is not novel, see [1] for example (the good thing is that AUC is a relatively well behaved statistic, so that does not change the framework). By using a C2ST in the framework, we introduce (i) a data-driven algorithm to judge the level of equal treatment in the data but also (ii) an additional notion of uncertainty in our fairness definition. For (i), in practice this means that this approach is not necessarily low-sample-size friendly (as it does not use permutations), the complexity of the classifier directly affects type I and type II error and results may vary considerably according to which classifier is chosen (which the authors have actually explored in the Appendix). For (ii), we are rejecting the null-hypothesis with a certain probability threshold, as opposed to provide a single (deterministic) number as in demographic parity. That is, we are now guaranteeing that "up to a level 1-\alpha" the algorithm is providing equal treatment.

A.2. Many thanks for pointing to relevant related work; we will integrate it on our work as it helps to clarify the strength of our submission.

I.3.Citing again the equal credit opportunity act of 1961, such a definition of fairness would not be admissible in a credit lending scenario, which puts into question once again the practical feasibility of this new definition of equal treatment.

A.3. May we ask in which section of the ECOA of 1961 does the regulation state this?

Note that equal opportunity, if understood by the Rawlsian "luck" egalitarianism, may fall closer to our notion than to TPR differences.

In the more recent regulation discussion [3,4] Wachter advocates for demographic parity. It is important to note the EU and US law are different. And there may be other worldwide incoming regulations from other countries that might be also different.

The applicability of AI regulation is an ongoing discussion with recent workshops specifically targetting this area (https://regulatableml.github.io/)

In [4]""In particular, some of the tests used by the European Court of Justice and Member State courts to measure indirect discrimination match the metric of demographic parity from algorithmic fairness.""

[1] Model-independent detection of new physics signals using interpretable SemiSupervised classifier tests, Chakravarti, Purvasha and Kuusela, Mikael and Lei, Jing and Wasserman, Larry, The Annals of Applied Statistics, 2023 [2] https://www.justice.gov/crt/equal-credit-opportunity-act-3#:~:text=prohibits%20creditors%20from%20discriminating%20against,under%20the%20Consumer%20Credit%20Protection

[3] Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI https://www.sciencedirect.com/science/article/abs/pii/S0267364921000406

[4] Bias Preservation in Machine Learning: The Legality of Fairness Metrics Under EU Non-Discrimination Law https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3792772

I4. The word "natural data" sounds a bit weird, as usually the machine learning community uses "real data".

A.4 The opposite to real data, is unreal data. While the contrary of natural data, is synthetic. We made this choice as we believe it's an improvement wrt terminology precision.

Dear reviewer, many thanks for the comments, see the main and specific rebuttal:

I1.1 [Major] I remain unconvinced that 'equal treatment' is a superior notion of fairness.

A.1.1.In the main rebuttal, we have argued why Equal Treatment is a superior notion to Demographic Parity. In Appendix B1, we have provided an illustrative example regarding blind reviews of papers

I1.2 A notable limitation of this fairness notion is its potential indirect correlation with protected attributes like Z....

A 1.2.In example 4.4 we have studied the indirect correlation with protected attributes, which is derived from the Shapley value theoretical propertie (Appendix A). Here, we adapt the example to the case that the reviewer proposes. We will predict a fictitious variable “weight” based on “age” and “place of birth” (both independent of “weight”) and “height” which is related to “weight”.

Let $X = X_1,X_2,X_3$ be independent features, and $X_1, X_2 \perp Z$, and $X_3 \not \perp Z$. Where $X_1$ and $X_2$ are “place of birth” and “age” respectively, and $X_3$ is “height” which correlates with gender $Z$ and “weight” $Y$.

We now have a linear model $f_\beta(x_1, x_2, x_3) = \beta_0 + \beta_1 \cdot x_1 + \beta_2 \cdot x_2 + \beta_3 \cdot x_3$ with $\beta_1, \beta_2, \beta_3 \neq 0$.

We can calculate the Shapley values by $S(f_\beta, x)_i = \beta_i \cdot x_i$. (Aas,2021)

Now we measure Equal Treatment by $g_\psi(s) = \psi_0 + \psi_1 \cdot s_1 + \psi_2 \cdot s_2+\psi_3 \cdot s_3$, which can be written in terms of the $x$'s as: $g_\psi(x) = \psi_0 + \psi_1 \cdot \beta_1 \cdot x_1 + \psi_2 \cdot \beta_2 \cdot x_2+\psi_3 \cdot \beta_3 \cdot x_3$. By OLS estimation properties, we have $\psi_1 \approx cov(\beta_1 \cdot X_1, Z)/var(\beta_1 \cdot X_1) = cov(X_1, Z)/(\beta_1 \cdot var(X_1)) = 0$ and analogously $\psi_2 \approx 0$. Finally, $\psi_3 \approx cov(X_3, Z)/(\beta_3 \cdot var(X_3)) \neq 0$.

The coefficients of $g_\psi$ provide information about which feature contributes to the dependence between the explanation $S(f_\beta, X)$ and the protected feature $Z$.

Using the example the reviewer proposed $\psi_3$ shows how related “height” is to the protected attribute in contributing to the prediction, thus violating Equal Treatment.

Many thanks to the author for the question, which helps clarify the paper's contributions. Does the reviewer find this clarification helpful?

I.2[Major] The paper omits a critical discussion on related work. There appears to be a study, specifically on individual fairness [1], that resonates with the motivations of this paper.

A.2 Individual fairness can not often be applied and is therefore not a popular metric [2]– unlike group fairness metrics. For example, when a company hires for one position, it obviously has to discriminate even if two individuals are identical wrt non-protected characteristics. Talking about unfainress is not plausible in this situation. Only if the company shows a systematic, statistically significant pattern of discriminating a group of people, one can talk about unfairness. Some Individual fairness metrics may have the challenge of achieving a precise mathematical definition of “similarity”. See [1] for a summary of critics on fairness metrics.

[1] Salvatore Ruggieri, José M. Álvarez, Andrea Pugnana, Laura State, Franco Turini: Can We Trust Fair-AI? AAAI 2023

[2] Fleisher, W. What's fair about individual fairness?. AIES 2021

I.3[Medium] The motivation presented within the paper is somewhat unclear, and it is concerning that significant discussions related to the work are relegated to the appendix. This decision diminishes the visibility and importance of such discussions.

A.3. We will adapt the introduction based on the feedback of the reviewers, but, due to space constraints, we decided to privilege the main contributions in the main text, and the detailed discussions in the appendix.

I.4 [Medium] I cannot agree with the authors that equal opportunity could lead to reverse discrimination and overcorrection.

A.3 See first answer for clarification (A.1).

From a technical perspective, Equal Opportunity requires labeled data which in practice, which after deployment is hard to obtain. Equal Opportuntiy forces to meet some quotas (True Positive Rate), this introduces a second order discrimination. Equal Treatment has no quotas, it is fine to “hire” the best, while the decision is taken independently from the protected attribute.

From a philosophical perspective much is discussed on the literature, the commentary at https://link.springer.com/content/pdf/10.1057/palgrave.cpt.9300060.pdf about the book [1] provides some hints.

[1] Matt Cavanagh, Against Equality of Opportunity. Clarendon Press, 2002.

Dear reviewer, in the following, we relate further metrics to philosophical requirements, as you have asked for. Please let us know if further questions or doubts remain.

Individual fairness is specified according to the original definition by requiring that $d (f(x),f(x')) \leq L \cdot d(x,x')$ where L > 0 is a Lipschitz constant and d(·, ·) is a distance function.

From a philosophical perspective, individual fairness is closest to the liberal point of view. However, it fails the requirements of liberalist arguments. Liberalism argues for meritocracy, i.e. disparate treatment is okay if it is based on varying efforts or preferences of individuals but not fair if it is based on characteristics that individuals did not choose. E.g. it's fair to hire someone because of better grades, but it is unfair if these grades depend on ethnicity. These varying treatments of attributes is not captured by the definition of Individual Fairness above - but it is captured by our notion of Equal Treatment.

Also, from the technical perspective, the philosophical requirements are not matched: (i) a definition of distance including continuous and discrete attributes leaves a lot of leeways and leaves decisions to the choice of hyperparameters. (ii) Individual fairness is not a statistical measure and will fail to account for situations, such as hiring decisions, where one needs to decide even if the differences between individuals are minimal (also our rebuttal answer A.2)

Fairness through un-awareness An algorithm is fair according to this definition of fairness if all the protected attributes $Z$ are not explicitly used in decision-making. Any learned function $f: X \rightarrow Y$ that excludes $Z$ satisfies this. Fairness by unawareness has the shortcoming that elements of $X$ can depend on $Z$. Such features are called proxy features. They can be used to train $f$ to make discriminatory decisions.

Counterfactual Fairness Another metric iscounterfactual fairness[1]. In the literature, it has been observed that this definition is “basically demographic parity”[2]. Thus, counterfactual fairness exhibits the same drawbacks that we have elaborated about Demographic Parity

Causal Fairness More commonly, causal fairness refers to metrics that require causal structure to be known for computing fairness or its violation. For example, Definition 1 ( taken from [3]) includes interventions to define fairness. Even if this definition may mirror what the liberal school of thought asks for. It remains a huge challenge to model or learn causal structures - and most often they remain unavailable.

To conclude, our notion of equal treatment is a practical and useful measure for judging the amount of unfairness, following the liberal school of thought in philosophy. Dealing with philosophical and practical requirements is indeed a superior fairness metric. We acknowledge that we should have elaborated on these metrics more broadly. However, as the reviewer may see from our rebuttal, we have concise arguments as to the superiority of our metric and are happy to include them in the paper - most likely in the appendix due to length restrictions.

[1] Counterfactual Fairness.Matt J. Kusner, Joshua R. Loftus, Chris Russell, Ricardo Silva: NIPS 2017

[2] Counterfactual Fairness Is Basically Demographic Parity. Lucas Rosenblatt, R. Teal Witter. AAAI 2023

[3] Causal feature selection for algorithmic fairness. Galhotra, S., Shanmugam, K., Sattigeri, P., & Varshney, K. R. In Proceedings of the 2022 International Conference on Management of Data.

Dear reviewer, many thanks for the comments. Please consider the main points of our rebuttal. We address specific points below:

I.1Operationalized tool:

A.1 First of all, we would like to thank the reviewer for trying out the code. We really appreciate it. We have now made a software release `explanationspace==0.0.2, addressing the reviewer’s concerns.

We would like to distinguish between the open-source software, which we will disseminate and maintain, and the repository of the experimental results of this paper.

We have now fixed the mentioned issues so the installation and tutorial of https://explanationspace.readthedocs.io/en/latest/index.html works accordingly.

I.2. While the authors propose an interesting perspective, the paper's structuring makes it hard to appreciate their contributions.

A.2 Due to space constraints, we decided to privilege the main contributions in the main text, and the detalled discussions in the appendix - bade based on what we believe it is more adequate to ICLR. We are open to any restructuring suggestions.

I.3. The authors provide insufficient related work in the area and problem background....

A.3. Both papers are distinct from ours. They do not focus on the intersection of equality notions and ML, nor on the techniques used (Classifier Two Sample Test), proposing methods that don’t have the theoretical background solidity of our approach.

We will be happy to acknowledge the recent work reported in the two arxiv papers. To the best of our knowledge, the first paper is still under review, while the second, we are unaware of its publication status. However, we hope that their absence in the submission is not held against us.

I.4. Some proofs and examples seem limited and don't explore corner cases.....

A.4 This basic example serves us to compare with the benefits of measuring bias on the input data. We agree that in most cases the model might not be linear. It's in this situation where the Shapley values become of use, we have studied non-lineal models in Appendix E.2 and E.3 Often in tabular data problems, ML models are fed with many more features than the model actually use. This method helps to deal with this problem.

I.5 Given that one might not have access to test data, would it be better to apply the ET inspector as a diagnostic on the train/val data instead?

A.5 We agree with the reviewer on this point. ET inspector can be used to detect bias (in train/val data), and to monitor bias in the absence of labeled data. We will point out this in the paper.

I.7Authors could have compared their experimental results to other related works and shown the impact on ET inspector and explanations on fairness on the different groups (something similar to table 5 in the appendix)

A.7. We kindly ask the reviewer which other related works our approach should be compared to.

I.8. For novelty, authors use AUC rather than accuracy in their C2ST instead of accuracy as previously done. This is a bit of a tradeoff, and while the scale invariance might be good, it is damaging when inspecting other cases of fairness where one might, for example, care more about false positives than false negatives.

A.8. Even if we use AUC, other metrics are also possible as far as a test statistics is available (e.g., accuracy in Lopez-Paz and Oquab 2017)

I.9Reliance on Z as a binary variable is restrictive, especially since there are lots of intersectionalities.

A.9 The inspector $g$ can be a multi-class classifier. In the experiment of Figure 3, there are several ethnicities. In this case we have used a 1vsAll approach, further methods can be extended even to continuous protected attributes using regression instead of classification

I.10 The explanation highly relies on f_{\theta}. It might be informative to also look at features independent of the model.

A.10 This is done in Example 4.3 in the Theoretical section and in the “Fairness Input” analyses in the Experimental section (e.g., Figure 2).

I.11 While the proposed method has several similarities with Explanability for fair machine learning and Model Explanation Disparities as a Fairness Diagnostic paper, operationalising their model has positively influenced my score. However, issues in the writeup and code documentation negatively influenced my score. Authors should please address these issues in the weakness section.

A.11 We have aimed to address the reviewer’s concerns in the above discussion.

We do not think that not comparing to recent work that is still currently under review should be held against us. We will add a comparison against this work in the updated version of the manuscript.