Understanding Unfairness via Training Concept Influence

We thank the reviewer for considering our work to be "well-written," "intriguing," "practical," "intuitive," "easy to understand", and "reasonable design."

We address the following concern.

1. Theoretical Understanding

We have dedicated an entire section on the theoretical insights in Appendix D. We did not include it in the main paper due to the page limit.

We briefly summarize here why intervening training data attributes using CIF framework can improve fairness. For simplicity, we focus on accuracy disparity as the fairness measure.

We base the analysis on the data generation model adopted in [1] to capture the impact of data patterns generated with different frequencies and the impact of label errors. This setup is a good fit for understanding how counterfactual data interventions can change the data frequency of different groups (majority group with higher frequency vs. minority group with lower frequency) and provides insights for CIF.

Intervening labels $Y$ is relatively straightforward. If we are able to intervene on a training label of a disadvantaged group from a wrong label to a correct one, we can effectively improve the performance of the model for this group. Therefore the label intervention can reduce the accuracy disparities. Our analysis also hints that the influence function is more likely to identify samples from the disadvantaged group with a lower presence in the data and mislabeled samples. This is because, for a minority group, a single label change would incur a relatively larger change in the influence value.

Intervening sensitive attributes $A$ improves fairness by ``balancing" the data distribution. In the experiments, we show that the influence function often identifies the data from the majority group and recommends them to be intervened to the minority group. In the analysis, we also show that this intervention incurs positive changes in the accuracy disparities between the two groups and therefore improves fairness.

Please let us know if there is still anything unclear.

[1] Feldman, Vitaly. "Does learning require memorization? a short tale about a long tail." Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 2020.

We thank the reviewer for considering our problem to be important. We address the concerns one by one.

1. Applicability

We thank the reviewer for the concern. Although we test binary classification models in our experiments, our method is flexible enough to be extended to multi-class fairness - the only difference is the definition of fairness, which is easy to plug in.

We admit that extending to the regression or generative models would require significant change, but almost all works in influence function (we mean general influence function, not even the influence function for fairness) only target the classification models, and therefore we think our scope is reasonable.

2. Relationship between "Concepts" and "Fairness"

We thank the reviewer for asking for clarification.

In our case, the concept can be sensitive attribute $A$, feature $X$, or label $Y$. Consider group fairness like Demographic Parity (DP): $\big| \mathbb{P}(h_\theta(X) = 1 | A = 0)-\mathbb{P}(h_\theta(X) = 1 | A = 1) \big|$, we can see it depends on feature $X$ and sensitive attribute $A$, and therefore the concept $X$ and $A$ are closely related to the fairness. Or consider Equality of Opportunity (EOP): $\big| \mathbb{P}(h_\theta(X) = 1 | A = 0, Y=1)-\mathbb{P}(h_\theta(X) = 1 | A = 1, Y=1) \big|$, it depends on all $X$, $Y$, $A$.

However, the relationship cannot be derived precisely since, for example, the change of a training sample's $A$ would impact the trained model weights $\theta$ and then impact the test fairness on $\theta$. How $A$ would impact the final fairness is complex and therefore we need the influence function and our whole approach of generating counterfactual samples to estimate it.

3. "override(.)" Operator

We thank the reviewer for the clarification. As the reviewer already noticed our explanation in Section 2.3, we will try our best to further clarify.

The "override(.)" operator is conceptually similar to the do(.) operator in the causal inference; but it is broader. In general, it represents the change of data attributes (feature $X$, label $Y$, or sensitive attribute $A$) of training samples. Since $X$, $Y$, and $Y$ are interdependent, we need to find out, for example, how the change of $A$ would impact $X$. If we already know the underlying causal relationship between variables, then it is the same as the do(.) operator. However, it would only work on synthetic data.

In practice, given non-synthetic data, the causal relationship is unclear (and discovering it is a notoriously hard problem in causal inference), which is our focus where the traditional causal methods are not applicable. To this end, we propose a set of empirical methods to estimate it (i.e. our W-GANs).

4. Baseline Comparison

We thank the reviewer for raising the concern.

To the best of our knowledge, the only prior works that study influence function on fairness are [1, 2]. However, both are not comparable to our work because they only consider removing or reweighing whole training samples while we decompose the fairness influence into features, labels, and sensitive attributes. We do not know other works that also study the fairness influence function of modifying features, labels, and sensitive attributes independently. If the reviewer knows, please let us know.

In addition, the "Removal" baseline in our experiment considers the effect of removing a training sample, which can be thought of as the method commonly used in the literature [1,2].

Furthermore, we have compared our mitigation to the baseline of in-processing mitigation with fair training [3] in Figure 5.

Last but not least, our work is mainly an explainable AI tool and the primary goal is to explain fairness to AI practitioners. The main objective is not to outperform other bias mitigation methods but rather attributing and explaining the observed fairness.

[1] Wang, Jialu, Xin Eric Wang, and Yang Liu. "Understanding instance-level impact of fairness constraints." International Conference on Machine Learning. PMLR, 2022.

[2] Li, Peizhao, and Hongfu Liu. "Achieving fairness at no utility cost via data reweighing with influence." International Conference on Machine Learning. PMLR, 2022.

[3] Agarwal, Alekh, et al. "A reductions approach to fair classification." International conference on machine learning. PMLR, 2018.

We thank the reviewer for considering our work to be "generally useful." We address the concerns one by one.

1. Incremental Approach

We thank the reviewer for the concern. We understand that it might look like our approach is incremental: influence function and W-GAN are adapted and modified from the prior works. However, our novelty does not lie in the details of measuring the influence of samples or generating counterfactual samples; it rather lies in the proposal of a general framework that explains and measures the training samples' influence on fairness.

To the best of our knowledge, almost no work in the area of influence function is based on a completely new technical approach. For example, influence function itself [1] is merely a revival of the concept in robust statistics that can be traced back to as early as 1974.

In addition, our framework is far from something technically trivial. None of the prior works can be directly applied in our setting. For example, the original influence function does not target fairness, and we adapt it to the fairness context. Furthermore, W-GAN in [2] only works on sensitive attributes, and we adapt it to also consider features and labels.

Most importantly, the concept of our framework is entirely new. The limited prior works on understanding fairness via influence function [3,4] so far only focus on removing or reweighing whole training samples. Our framework decomposes the fairness influence into features, labels, and sensitive attributes. This distinction is vital in fairness because each attribute can significantly impact the underlying fairness. And it is feasible because of our novel framework by generating counterfactual samples.

In short, our framework is more flexible, powerful, and more suitable for the nature of AI fairness and can give practitioners a wider scope of understanding of fairness. It also leads to a wider range of applications, e.g. detecting mislabeling, fixing imbalanced representations, and detecting fairness-targeted poisoning attacks, as articulated in the paper.

We think if we all insist on an entirely new invention of all methodological details, then almost no work on fairness, influence function, or causal inference can be published on ICLR. We believe most of the papers in those areas do not propose a completely new technique.

We will clarify it in the paper. Please let us know if we still have not addressed the concern. If the reviewer disagrees, feel free to let us know the reasons as well as some concrete and technically detailed examples.

[1] Koh, Pang Wei, and Percy Liang. "Understanding black-box predictions via influence functions." International conference on machine learning. PMLR, 2017.

[2] Black, Emily, Samuel Yeom, and Matt Fredrikson. "Fliptest: fairness testing via optimal transport." Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 2020.

[3] Wang, Jialu, Xin Eric Wang, and Yang Liu. "Understanding instance-level impact of fairness constraints." International Conference on Machine Learning. PMLR, 2022.

[4] Li, Peizhao, and Hongfu Liu. "Achieving fairness at no utility cost via data reweighing with influence." International Conference on Machine Learning. PMLR, 2022.

2. Details of "Overriding X" Experiments

We thank the reviewer for asking for clarification. Due to the space limit, we have included the settings of overriding $X$ in our experiments in Appendix E.1. We understand that it is easy to be omitted. We recap it here.

"Overriding X" means counterfactually changing samples as if their features were changed to different values, and then measuring how the underlying fairness would be impacted. For example, in an image task, if we want to understand how skin color would affect fairness (say gender fairness), we can generate counterfactual features (image pixels) as if they come from a different race, i.e. how would fairness change if those images change their racial group.

In COMPAS, $X$ is the binary feature that indicates age $> 45$ or not while the sensitive attribute is race. In Adult, $X$ is race while $A$ is sex. In CelebA, $X$ is the binary label ``Young'' (i.e. age) while $A$ is gender.

Please let us know if this is still not clear enough.