Counterfactual Fairness With the Human in the Loop

• A1. We want to emphasize that we treat $U’$ as a hidden variable. However, it is not an exogenous variable. $U’$ is the effect of $U$ and decision $\hat{Y}$. On the other hand, $U$ is an exogenous variable and does not have any parents. In this case, SCM is not violated.

• A.2 $\eta$ is not a hyper-parameter. We consider it as a constant. It is a value that reflects how the individual response to the current prediction. In our paper, we assumed it is a given since we don't have a real dataset about the interaction. If we measure $Y$ and $Y’$ before and after decision $\hat{Y}$, respectively, $\eta$ can be estimated.

• A.3 There are other forms for strategic classification. For example, when a cost function is defined on an attribute $U_{i}$ as $c(u_{i}, u_{i}’)$ (which means how much the individual need to cost by changing the attribute), the response function of this attribute is the $\arg \max_{u_{i'}} \hat{Y}(u_{i}’) - c(u_{i}, u_{i'})$.

  The advantage for the response function form we used in our paper is that it is simple yet can reflect a general kind of people’s response. It means that people try to take action to change their status along the direction that will change the prediction mostly. The disadvantage is that when the prediction model is not differentiable, the response function could be out of work.

• B.1 $p_{1}$ is a hyper-parameter that controls the trade-off between the LCF and accuracy. For example, if $p_{1} = \frac{1}{2\eta(||w\odot \alpha||^{2} + \gamma^{2})}$, LCF (eq.4) is satisfied and we have perfect fairness. If we choose smaller $p_1$, we improve accuracy but the LCF is satisfied approximately (I.e., the left hand side in eq.4 is close to the right hand side).

  On the other hand, $\eta$ is fixed in our model, we cannot and don't need to optimize it.

• B.2 Yes, we need the knowledge about the prior distribution $P(U)$.

• B.3 The Eq.5 is deterministic meaning that $\alpha, \beta, \gamma$ and $w$ are fixed. However, we do not know the specific value of $U$ and it is a random variable.

• B.4 Yes, when we used MCMC to get the conditional distribution of $U$, we need to know $P(U)$. It is a common requirement for counterfactual fairness. For the experiment, we used the same prior as the one used in the experiment of Kunser et al. 2017 .

• C.1. $A$ is the sensitive attribute which determines different demographic groups. For example, it can be the gender of the person. $X$ are the features, for example, the income and credit score. We are trying to predict whether the person will default or not, and the credit score it is also included in $X$. Note that credit score is not the same as default probability.

• C.2 We can imagine a case, we can observe the income of a person. For simplicity, assume that the income dictates the qualification of an applicant. And there is a hidden attribute U represents how hard the person works. Both $U$ and $A$ can impact income. So, in the counterfactual world where the person has the same $U$ with different gender may have a different income. If that income is low, the counterfactual person would not be qualified.

  Mathematically, $Y_{A\leftarrow a}(U)$ and $Y_{A\leftarrow a’}(U)$ do not have the same distribution necessarily. However, counterfactual fairness implies that the predictions $\hat{Y}_{A\leftarrow a}(U)$ and $\hat{Y}_{A\leftarrow a'}(U)$ should have the same distribution.

• C.3 We assumed that SCM is known in our problem which is a common practice (e.g., Kusner et al. 2017)

• D.1 Sorry, you are correct. We meant the distribution of \hat{Y}. We have corrected it in the newly updated version.

• D.2 The assumption is implicitly included in Eq.5. We also state it explicitly in theorem 3.1.

• D.3 Thank you for your reminding about the clarity. We added it to our updated version.

If you mean the prior distribution of $U$ by $P(U)$, even with the bijective assumption, it is still impossible to infer it without knowing the distribution of all the observable variables.

You might think it is inconsistent because we said we need $P(U)$ at first response but not now. It is because we thought you are talking about a general counterfactual inference. Without bijective assumption, we can use methods like MCMC to infer the counterfactual values ($\check{X}$, $\check{Y}$), which needs the prior distribution $P(U)$. But with bijective assumption, we can apply the method in [1] to estimate $\check{X}$ or $\check{Y}$ without $P(U)$. Then we can use what we described in Appendix B to solve the following problems.

We appreciate your help. Your valuable comments can help us to improve the paper during the discussion period. If you see any inconsistency, we are going to address it.

For weakness 3:

We want to emphasize that LCF takes into account $\hat{Y}$. As we mentioned earlier, $Y'$ and $Y$ are two separate variables, and $\hat{Y}$ is a cause for $Y'$. Therefore, by imposing a constraint on $Y'$, we are imposing certain constraints on $\hat{Y}$ as well.

The whole point of this paper is to make sure that $\hat{Y}$ is designed in a way that equation 4 is satisfied. Even though $\hat{Y}$ does not appear in equation 4, it is a cause for $Y'$. Therefore, we need to design $\hat{Y}$ very carefully to satisfy equation 4.

For question 1:

Yes, we use human in the loop to emphasize that a human is interacting with an AI model and responds to it through a response function $r$.

We appreciate your opinion about the title. We are willing to change the title to Lookahead Counterfactual Fairness if the ICLR policy allows us to change the title.

For question 2:

Thank you for your comment. We modified the section to clearly indicate the three steps in the updated version. Question 3: Yes, causal sufficiency is required. Generally speaking, counterfactual variables are defined based on the complete causal structure. So, we need to measure all the hidden variables. Both counterfactual fairness (Kusner et al. 2017) and lookahead counterfactual fairness can be studied when the causal structure is known. There are various methods for finding a causal model, see e.g., Peters et al. (https://arxiv.org/pdf/1309.6779.pdf).

For weakness 1:

CF considers both factual and counterfactual worlds. However, it imposes constraint on the prediction without considering the consequence of the decision in factual and counterfactual worlds. We are not claiming that counterfactual fairness is a bad fairness measure and has limitations. Just we are saying LCF and CF have different goals. In particular, we want to emphasize that while \hat{Y} is the same in the factual and counterfactual world under CF constraint, true Y and future Y’ may not be the same in both worlds. If Y is not the same in both worlds, and we want to make sure that Y’ will be the same in the factual and counterfactual world, then CF fairness constraint cannot handle this situation. You can see more about why CF does not imply LCF in theorem A.1 we added in the updated version.

In our assumption, the individual is labeled (or at least we can approximate the $Y$ with the causal graph). So, when the individual in the factual world and counterfactual world belongs to the different demographic group (which means different value for the node representing this attribute in the causal graph), it is possible to get $Y = 1$ and $Y^{CF} = 0$ under LCF.

For weakness 2:

As we know, strategic classification did not always mean cheating. There are two kinds of strategic behavior: manipulation and improvement (see in https://openreview.net/forum?id=W98AEKQ38Y). In this paper, we are considering the case for improvement.
We want to emphasize that $U$ and $U'$ are not the same variables. $U'$ is the hidden variable after the individual receives a decision. On the other hand, U is the exogenous hidden variable (without any parents) before decision \hat{Y}. U and U' are completely separate variables. U’ can be impacted by $U$ and $\hat{Y}$. As you can see in our model, $U' = r (U, \hat{Y})$.

If you think of $U$ and $U'$ as two separate variables, then it is easy to see that counterfactuals are non-backtracking in this paper. You can interpret U as an inherent hidden variable (similar to what we have seen in literature) and $U'$ as the hidden variable but it is the effect of $U$ and $\hat{Y}$.

Similarly, $Y$ and $Y'$ are two separate variables. Consider a loan approval problem. The qualification of an applicant at the current time is denoted by $Y$. On the other hand, the qualification of an applicant in the future is denoted by $Y'$. $Y'$ will depend on current circumstances (i.e., $X, U$) as well as loan approval/rejection at the current time ($\hat{Y}$). In other words, $\hat{Y} X$, and $U$ are the cause for $Y'$. However, $\hat{Y}$ is not a cause for $Y$ because $Y$ is the qualification before loan decision. Given this explanation, I think our experiment in section 4.2 makes sense. We consider two separate variables $K$ and $K'$. In our setting, the decision $\hat{Y}$ cannot change $K$. $K$ does not change over time because $K$ is the knowledge at the current time. $K'$ is a separate variable and its value is different from $K$.

We respectfully ask the reviewer to look at the paper one more time. We strongly believe that if the reviewer reads the paper one more time and considers that $(U, X, Y)$ and $(U', X', Y')$ are two separate variables, the story of our paper would make more sense. You can also see the strategic classification does not cause any issue as $U$ and $U'$ belong to separate time and are different in nature. $U$ is exogenous and no variable is a cause for $U$. While $U'$ is not exogenous and other variables can be a cause for $U'$.

For weakness 1 and question 2:

We do not make any assumption on whether $A$ is a cause for $Y$ or not. As you can see in figure 1a and example 2.3, $A$ is indeed a cause for $Y$.

For weakness 2 and question 1:

Note that LCF is not a minor extension of CF. We cannot simply rename $Y$ and treat $Y'$ as the new label. $Y'$ is a consequence of the decision $\hat{Y}$ and depends on both $Y$ and $\hat{Y}$. As discussed in our paper, satisfying LCF is much more challenging, and we cannot achieve LCF using the same approach as for CF. In the new version of our paper, we have also included Theorem A.1 in the appendix to demonstrate that a predictor satisfying CF does not necessarily satisfy LCF.

Furthermore, we want to emphasize that imposing a fairness constraint on $\hat{Y}$ does not necessarily mitigate harm if we do not consider the future outcome/consequence of $\hat{Y}$. For example, in a loan approval problem, having the same selection rate among different demographic groups may seem like a plausible fairness constraint. However, it is possible that the majority of people in one demographic group are unable to repay the loan and default. Consequently, having the same selection rate across different demographic groups may adversely affect one demographic group. Therefore, we must consider the consequences of our decision. In our model, $Y'$ represents the future outcome/consequence of the predictor $\hat{Y}$.

For weakness 3 and question 3:

We have already included Theorem 3.3 for the nonlinear causal model in our paper. Additionally, in the new version, we have added Theorem A.2 to demonstrate that our results can be extended to the nonlinear case.

For weakness 1:

We are considering the probabilistic counterfactual. In our model, we can only infer the distribution of $U$ conditioned on the observed features. We utilized $U$ as the input for our LCF predictor, similar to Kusner et al. 2017 (Section 4.1), for training a counterfactually fair predictor. We sample the value of $U$ from the conditional distribution and take it as the input. Under the distribution of $U$, our algorithm minimizes the expected error over $U$, $X$, and $Y$. For inference, we can also find the expectation of the output over $U$ (similar to Kusner et al. 2017).

For weakness 2:

We aim to emphasize that our method can be extended to nonlinear causal models. In particular, we consider a nonlinear causal model in Theorem 3.3. Additionally, in Appendices A.3 and A.4, we provide a full illustration and proof of the theorem. We also introduce another type of nonlinear causal model in Theorem A.2. In Appendix A.6, we present the formal theorem and a comprehensive proof.

In all the causal models discussed in our paper, we have included hidden variables. For example, in our linear model, we incorporate $U_X$ and $U_Y$ as hidden variables that affect the features, as shown in Eq. 5. These are expressed in the form of additive noise.

In general, CF does not imply LCF. We added theorem A.1 in Appendix A.6 showing that when LCF could be violated even though CF was satisfied.

For weakness 3:

We implemented the method used in Jing et al., and the results are as follows:

We reported the MSE, which reflects the distance between $\hat{Y}$ and $Y$, and the AFCE, which reflects the average distance between $Y'$ and $\check{Y}'$. Compared to our methods, this method achieves slightly better accuracy, but like other counterfactual fair predictors, it cannot improve LCF at all. UIR represents the ratio by which LCF has been improved. It is 0% for this method, while we can achieve 100%.

Note that the method proposed in Depeng et al. is only applicable to a classification setting, whereas in our experiment, we are working with a regression problem. We also want to emphasize that the method proposed in Yongkai et al. is for synthetic data generation under intervention, not for training a predictor. Moreover, we could not find the GitHub repository for the Yongkai et al. paper, and their experiments are not reproducible.

For question 1:

Generally speaking, counterfactual variables are defined based on a causal structure. Both counterfactual fairness (Kusner et al. 2017) and lookahead counterfactual fairness can be studied when the causal structure is known. There are various method for finding a causal model, see e.g., Peters et al. (https://arxiv.org/pdf/1309.6779.pdf).

There is a $\hat{Y}$ in the first step. Our goal is to train $\hat{Y}$ which can estimate $Y$ under LCF. There is no pre-defined relationship between $\hat{Y}$ and $Y$, but we try to make $\hat{Y}$ and $Y$ as close as possible.

We used $Y'$ in Definition 3.2 instead of $\hat{Y}$ because we want to make the future outcome $Y'$ fair. Imposing fairness constraints on $\hat{Y}$ has been studied in the literature. However, in this work, we aim to take into account the future impact of our decision. For example, consider a loan approval problem. Having the same selection rate among different demographic groups may be a plausible fairness constraint. However, it is possible that the majority of people in one demographic group are not able to repay the loan and default. As a result, having the same selection rate across different demographic groups may hurt one demographic group, and we have to take into account the consequences of our decision. In our model, $Y'$ represents the future outcome/consequence of the predictor $\hat{Y}$.

For question 2:

We are not doing intervention on the noise $U$. We are using the law of total probability with respect to U to find the probability distribution for counterfactual random variable $Y_{Z\leftarrow z}(U)$.