Fairness-Aware Domain Generalization under Covariate and Dependence Shifts

We sincerely appreciate the reviewer's valuable and insightful suggestions, which have significantly enhanced the clarity of our writing. We are committed to carefully considering all recommendations to further refine and improve the quality of our paper.

[W3] Some domain generalization papers concern the shift in fairness, such as [a, b]. For example, in [a], they gave an example using the Waterbird dataset, where the backgrounds (i.e., sensitive subgroups, land and water) are spuriously correlated with class labels (waterbirds and landbirds). This spurious correlation causes sub-par performance in the smallest subgroups. However, [a] only takes the shift in fairness as domain variations. In our paper, we state the distribution shift across domains due to covariate shifts (data features) and changes in fair dependence. Therefore, the dependence shift is proposed. Since prior research, such as [a, b], has been proposed, we do not conclude the dependence shift is a major contribution to our paper.

[W5] In our setting, an image has two labels, a class label and a sensitive label. For example, an image of a “dog in grass” has a “dog” class label and a “grass” sensitive label. Therefore, the features extracted from this image can be perceived as encapsulating information about both the "dog" and "grass". Another example is given in the empirical study using the ccMNIST dataset where the digit color is used as domain variation due to covariate shift and the background color is used to describe the sensitive label, dark and black. This is similar to traditional fairness learning using tabular datasets. For example, gender is a sensitive label of instance, “height” and “has a beard” could be features used to describe gender information.

[W6, Q2, Q5] This paper focuses on group fairness and uses DP or DDP (difference of DP) as the fairness notion. However, in our Eq.(1), it can be easily changed to the difference of equalized odds when setting $p_1=\mathbb{P}(Z=1, Y=1)$. And Eq.(1) can be generalized to take the maximum of $g$ across different classes for the multi-class setting. For multi-dimensional sensitive attributes, Eq.(1) can be replaced by corresponding group fairness metrics, such as an extended version of DP or EO proposed in [e].

[W8, Q3] GDRO, Mixup, and CORAL are used as baseline methods in many multi-source domain generalization works, such as [c] and [d]. Such methods have been included in a public code repository developed by FacebookResearch, named domainBed (see https://github.com/facebookresearch/DomainBed) with 1.2k stared. Moreover, in our Problem 1, the max operator requires domain labels $e$. However, the domain labels are expensive or even impossible to obtain in part due to privacy issues. Therefore, under the disentanglement-based invariance and domain shift assumptions, Problem 1 can be approximated to Problme 2 with the max operator eliminated.

[W9, W10] Thanks for the head-up. Certainly, addressing the related work as highlighted by the reviewer and making the necessary corrections regarding citations will be a part of our discussion and subsequent revisions.

[W11, Q4, Q6, Q7] Some limitations could be the assumptions that we made (1) instances can be encoded into three latent factors (2) such factors are independent of each other, and (3) each training domain shares the same content space. Our proposed method may not work well when data are generated with more than three factors, and each is correlated to the other. Besides, our model is built based on domain augmentation. Although results showcase the effectiveness of our model, it may not work well when content spaces are not fully overlapped across domains. In this case, it would be better to first augment data by minimizing semantic gaps for each class across training domains and then conduct domain augmentations. To address such limitations, studies on causal learning could be a solution, and we will save it for our future works. Besides, our model is a bit expensive as the transformation model is learned using GAN. The detailed training process is discussed in the Appendix.

[a] Creager et al., Environment Inference for Invariant Learning, ICML 2021

[b] Oh et al., Learning Fair Representation via Distributional Contrastive Disentanglement, KDD 2022

[c] Zhang et al., Towards principled disentanglement for domain generalization. CVPR 2022

[d] Robey et al., Model-based domain generalization. NeurIPS 2021

[e] Yang et al., Fairness with Overlapping Groups, NeurIPS 2020

[W7] Some additional results on the trade-off between fairness-accuracy are conducted. We will update it in a later version.

In our algorithm, because $\lambda_2$ is the parameter that regularizes the fair loss, we conduct additional experiments to show the change between accuracy and fairness. Our results show that the larger (smaller) $\lambda_2$ , the better (worse) model fairness for each domain as well as in average, but it gives worse (better) model utility.

Moreover, we show additional experiment results based on choosing different $\gamma_1$ and $\gamma_2$. We observe that (1) by only increasing $\gamma_2$ , the model towards giving unfair outcomes but higher accuracy; (2) by only increasing $\gamma_1$, performance on both model fairness and accuracy decreases. This may be due to the failure of disentanglement of factors.

[W1] Our example in Figure 1 is motivated by [a]. In [a], they gave an example using the Waterbird dataset, where the backgrounds (i.e., sensitive subgroups, land, and water) are spuriously correlated with class labels (waterbirds and landbirds). This spurious correlation causes sub-par performance in the smallest subgroups. Similar to Figure 2 in our paper, the spurious correlation between classes (dogs and cats) between image backgrounds or other objects (grass and couches) leads to untrusted predictions. For example, if a model learned from all training images “dogs in the grass” and “cats in the couch”, as a consequence the model face challenges accurately classifying “dogs” and “cats” during inference with images “dogs in the couch” and “cats in the grass”. This indicates sensitive features “grass” and “couch” are implicitly correlated with model prediction.

[W2] In this paper, we use a binary sensitive attribute and classes just as a simple case for idea illustration, as many fairness works did, such as [a]. This can be easily extended to multinary cases by either changing the group fairness notion or taking the maximum of difference of DP over each class/subgroup. Similar settings are considered in [b].

[W3] The reviewer seems to forget to give the [cite] reference, but we assume it refers to [b]. The difference between [b] and our works, with respect to handling fairness (dependence shift), is that [b] does not implicitly or explicitly assume fairness levels (dependence between sensitive attributes and classes) change across domains. Since test domains are inaccessible during training, in our works training domains are augmented with new variations and fairness levels. The greater the diversity of domains encompassing various variations and fairness levels encountered during training, the more effectively the model can generalize to different domains during inference. With limited fairness levels in training domains, in contrast to [b], our model has a better capability of generalization in fairness.

[W4] Some other papers in domain generalization based on maximizing the Evidence Lower Bound (ELBO) do not make this assumption, such as [c]. We hence consider it as an assumption but not a definition.

[W5] Theorem 1 provides an upper bound on the fairness adaptation achievable using the proposed algorithm. Theorem 2 gives the empirical duality gap, as the proposed algorithm using duality that iteratively updates model primal parameters and dual variables (lines 11-12).

[W6] The goal of domain generalization problems is to find an invariant classifier in the classifier space of $\mathcal{F}$, theoretically. But in experiments, with given datasets, this goal is required to approximate to find model parameters of the classifier in a parametric space. Moreover, in Problem 1, the max operator requires domain labels $e$. However, the domain labels are expensive or even impossible to obtain in part due to privacy issues. Therefore, under the disentanglement-based invariance and domain shift assumptions, Problem 1 can be approximated to Problme 2 with the max operator eliminated. We will clearly explain this in an updated version.

[a] Creager et al., Environment Inference for Invariant Learning, ICML 2021

[b] Pham et al., Fairness and accuracy under domain generalization. ICLR 2023

[c] Liu et al., Learning Causal Semantic Representation for Out-of-Distribution Prediction, NeurIPS 2021

[W1] (1) Concept shift in Figure1 is more related to semantic shift. Semantic shift as illustrated in [a] can be viewed as a type of concept shift. In [a], concept shift is defined as semantic changes, and in their experiments CIFAR100 is split into 10 sub-datasets with increasing conceptual differences from CIFAR10 classes. For example, pickup truck (CIFAR100) is much closer to truck (CIFAR10) than sunflowers (CIFAR100) semantically. (2) Our example in Figure 1 is similar to [b]. In [b], they gave an example using the Waterbird dataset, where the backgrounds (land and water) are spuriously correlated with class labels (waterbirds and landbirds). Notice that when using images for fairness learning, since inputs are images followed by class and sensitive labels, some aspects of the image features may inadvertently encapsulate information related to the sensitive label.

[W2] The fairness level in Figure 2 is calculated using demographic parity (DP) using the evaluation metric used in experiments. The equation of DP is proposed in Appendix C.3.

[W3] In this paper, we use a binary sensitive attribute and classes just as a simple case for idea illustration, as many fairness works did, such as [b]. This can be easily extended to multinary cases by either changing the group fairness notion or taking the maximum difference of DP over each class/subgroup.

[W4, W5] The AUC (defined in Appendix C.3) in this work refers to a statistically consistent measure of predictive strength by comparing outcomes between sensitive subgroups. Notice that this is different from the AUC used for classification and this metric is commonly used for fairness learning, see [c,d,e].

[W6] Dependence between $Y$ and $Z$ is to reveal the bias with respect to the dataset unfairness. Dependence between $\hat{Y}$ and $Z$ refers to algorithmic fairness with respect to the model prediction. Dataset unfairness is used as a reference to compare with the algorithmic fairness for performance improvement.

[W7] Such works will be discussed in an updated version.

[Q1] In Problem 1, without the constraint, it is equivalent to a domain generalization (DG) problem stated in some DG works, such as [f,g]. A fair constraint can be perceived as the pursuit of optimal model parameters within a fair parameter space, which is derived from the real domain after removing sensitive attributes.

[Q2] DP as a commonly used group fairness metric refers to the likelihood of a positive outcome being the same across different groups. We use DP only because this metric is well-known in fairness machine learning and is considered in many related works. Some other fair metrics, such as equalized odds, can also be applied.

[Q3] The reduction of AUC indicates the sum ranks of predicted outcomes between sensitive subgroups are similar. This AUC metric (defined in Appendix C.3) is not the same as the one commonly used in binary classification based on TPR and FPR. The intuition behind this AUC is based on the nonparametric test, Mann–Whitney U test, in which for randomly selected values $\mathbf{x}_1$ and $\mathbf{x}_2$ from two sensitive subgroups, a fair condition is defined as the probability of $\mathbf{x}_1$ being greater than $\mathbf{x}_2$ is equal to the probability of $\mathbf{x}_2$ being greater than $\mathbf{x}_1$, i.e. AUC=0.5.

[a] Tian et al., Exploring Covariate and Concept Shift for Out-of-Distribution Detection, NeurIPS 2021

[b] Creager et al., Environment Inference for Invariant Learning, ICML 2021

[c] Calders et al., Controlling Attribute Effect in Linear Regression, ICDM 2013

[d] Zhao et al., Rank-Based Multi-task Learning For Fair Regression, ICDM 2020

[e] Ling et al., AUC: a statistically consistent and more discriminating measure than accuracy. IJCAI 2003

[f] Zhang et al., Towards principled disentanglement for domain generalization. CVPR 2022

[g] Robey et al., Model-based domain generalization. NeurIPS 2021