Adversarial Latent Feature Augmentation for Fairness

More complex dataset

Please see the global response, addressing more general scenarios beyond binary classification with binary-sensitive attributes.

Overfitting problem in data augmentation with adversarial training

In the proposed framework, overfitting is not a matter, because unlikely typical adversarial training, we use different objective functions in the attacking step and training step. Moreover, we can interpret the 'unfair region' described in Figure 1(a) as an overfitted or underfitted area. We directly recover the unfair region making the misclassification rates for each subgroup fair, and alleviate the overfitting and underfitting of original prediction.

About the L2 perturbation and hyperparameters

In fact, our framework is not limited to $L_\infty$ perturbation. $L2$ perturbation is also applicable because the perturbation vector $\delta$ is trained by the fairness attack objective function Eq.(5). Especially, the Sinkhorn distance regularizes the perturbated dataset has a similar distribution with the original dataset.

The reason we adapt $L_\infty$ is the hyperparameter $\epsilon$ is not straightforward if we use $L2$ perturbation. However, we may be able to omit the hyperparameter $\epsilon$ by computing the proper $\epsilon$ given dataset and decision boundary. In detail, we can measure the Euclidean distance between each subgroup and the pre-traind decision boundary. And we can set the mean distance as $\epsilon$ for the $L2$ perturbation.

For example, if we use $L2$ perturbation by setting $\epsilon$ as the measured distance for COMPAS dataset and MLP with $\alpha=1.0$ and $\lambda = 0.5$, we obtain the range of $\delta\in[-0.2722,0.2552]$. Following the measured perturbation, we obtain a similar result with $L_\infty$ by setting $\epsilon=0.27$. Therefore, we can conclude that the choice of $L2$ and $L_\infty$ doesn't affect the performance. If a practitioner wants to control the hyperparameter $\epsilon$, $L_\infty$ is suitable. On the other hand, if one wants to simplify the setting, the $\epsilon$ measurement with $L2$ perturbation could be a wise choice. We will add this ablation study in the revision.

When it comes to $\lambda$, the selection of $\lambda$ is significant to the accuracy-fairness trade-off since it works as the weight between the original and perturbed dataset. Therefore, we vary $\lambda$ and report the 'line' of the performances in the Pareto Frontier as well as varying $\alpha$ in Eq.(5). We suggest using $\lambda=0.5$ since it empirically shows the best accuracy-fairness trade-off for all datasets. Still, the hyperparameter tuning is needed for $\alpha$. Here we report the training time for experiments for each combination of hyperparameters. Each fine-tuning is conducted in 50 epochs. We measure the training time for a single run for baseline, and the average of 10 runs for fine-tuning.

Therefore, we can omit the hyperparameter $\epsilon$ and $\lambda$ by using $L2$ perturbation taking the mean Euclidean distance between the latent feature and the decision boundary as $\epsilon$ and setting $\lambda=0.5$ to make the original dataset and perturbed dataset equally contribute. However, even though only one hyperparameter, $\alpha$, is need to be tuned, the hyperparameter tunig itself takes less time compared to the original ERM because fine-tuning trains only the last layer of the model.

Storage cost

ALFA is effective in terms of storage or memory cost. In detail, we can think about two types of augmentation, offline augmentation (storing augmented data in the drive) and online augmentation (augmentation on the data loader, stored in the memory temporarily).

For the offline augmentation, more storage cost may be required compared to the baseline ALFA. However, it requires less storage cost compared to the data augmentation on the input space. For example, for $N$ samples, the storage cost for $N$ images might be huge, while ours requires a single $N \times d$ matrix where $d$ is the dimension of the latent feature.

For the online augmentation, the data augmentation on the input space may require more training time and memory space. However, as ours is a fine-tuning method using a latent feature from original images and a perturbed latent feature, we only require two $N \times d$ matrices in the memory space removing the dataloader from the input images in the memory.

Beyond binary $Y$ and $A$

Please see the global response, addressing more general scenarios beyond binary classification with binary-sensitive attributes.

Linear separability assumption

We agree with the reviewer's point that the linear separability assumption is needed in our framework. However, linear separability is a widespread assumption [1] even though the size of the dataset is small or the number of features is large even in the input space. Moreover, deep neural networks transform the input data into well-separated latent features, making the classification easier.

[1] Moshkovitz, Michal, Yao-Yuan Yang, and Kamalika Chaudhuri. "Connecting interpretability and robustness in decision trees through separation." International Conference on Machine Learning. PMLR, 2021.

Label flipping strategy

Although flipping labels might generate samples in which $\Delta EOd$ and $\Delta DP$ are worse, training on them doesn't necessarily improve fairness. For example, we consider the latent distribution in Figure 1 in the paper. The augmented dataset may have the same distribution as the original data but flipped labels. Training on the original and augmented (flipped) dataset together may produce poor decision boundaries, exacerbating the accuracy.

On the other hand, ALFA aims to produce a biased dataset while maintaining the data distribution of the original feature, using Sinkhorn distance, which ensures the generated data has a similar data distribution to the original one and is useful to improve fairness while maintaining accuracy.

Sample repetition

Yes, the data sampling is repeated to maintain the size of each subgroup to be the same. Although this repetition is not a mandatory condition empirically, we make the subgroup size equal to ensure the efficiency of the fairness attack as shown in Theorem 3.2.

Contribution of $\lambda$

The selection of $\lambda$ is significant to the accuracy-fairness trade-off since it works as the weight between the original and perturbed dataset. Therefore, we vary $\lambda$ and report the 'line' of the performances in the Pareto Frontier in Figures 2 and 3 in the paper, as well as varying $\alpha$ in Eq.(5).

Multiple protected attributes

Please see the global response, addressing more general scenarios beyond binary classification with binary-sensitive attributes.

I would like to thank the authors for their rebuttal work. For most parts, my questions have been answered. However, I do have some additional comments on them which seems to be important -

1. Linear separability: The original comment should be read as separation of data wrt different sensitive attributes which seems to be essential to be explanation that rotation of linear boundary would help cover unfair regions. Even though data with different classes (labels) would be linearly separable [1], it is not guaranteed the same for data with different sensitive attributes. Consider Figure 1, if the two sensitive attributes were inter-mixed in latent space, I think a rotation would be insufficient to fix unfairness. This issue even worsens when there are more classes of sensitive attributes.

2. Label flipping - the original question Q1 was to generate augmented samples using maximizing FPR for one of the classes. This could in theory generate samples to cover the unfair region as well. How would this direct approach do in comparison to covariance loss used in the paper.

3. Contribution of $\lambda$ - I was of the understanding that the figures 2, 3 are plots between EOd or DP vs Accuracy. Q3 was regarding the absolute value of $\lambda$ vs EOd and Accuracy.

4. New results on multi-class classification - The new results seem to indicate that proposed method leads to significant drop in acc with not much gain in DP. Also why are EOd not reported for this experiment?

Minor typo in definition Differential Fairness DF in the rebuttal. Please check indices i,j.

The statement about GAN in tabular dataset

The statement in the paper reviewer mentioned is based on our empirical observation of the existing work, FAAP [1] which is applicable only to the image dataset. As shown in Figures 1 and 2 in the paper demonstrating the experimental result on tabular datasets, FAAP shows poor performance in tabular datasets and we suspect it is because the GAN-based perturbation might not be converged well.

Application to vision dataset

We reported the experimental results for the image dataset, CelebA as shown in Figure 4. In this case, FAAP shows comparable results to ours, and W1 is not a matter in this case. Therefore, it shows that FAAP works only for image datasets, while ALFA (ours) works for both tabular and image datasets. Moreover, we conduct additional experiments applying ALFA to the NLP dataset, as shown in the global response in this rebuttal.

Missing notations

Thanks for pointing out the typos. We will revise them and screen again the entire notations thoroughly.

The novelty of adversarial fairness

ALFA and [2] have different perspectives on adversarial training. [2] uses an adversary to predict sensitive attributes, while the classifier predicts the label. The objective function for the classifier is designed to minimize the binary cross-entropy (BCE) loss on the label while maximizing BCE loss on the sensitive attribute, which makes the classifier not use the demographic information during the prediction. However, no mathematical derivation is introduced as to how maximizing BCE on the sensitive attribute makes the classifier fair. Moreover, [2] is designed only for the COMPAS dataset.

We contain recent adversarial training methods in the paper such as [1] and [3], and describe how they are different from ours, in Section 1 and Section 2, respectively.

Beyond binary labels and sensitive attributes

Please see the global response, addressing more general scenarios beyond binary classification with binary-sensitive attributes.

[1] Zhibo Wang, Xiaowei Dong, Henry Xue, Zhifei Zhang, Weifeng Chiu, Tao Wei, and Kui Ren. Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10379-10388, 2022b

[2] Christina Wadsworth, Francesca Vera, Chris Piech. Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction. FAT/ML Workshop, 2018

[3] Peizhao Li, Ethan Xia, and Hongfu Liu. Learning antidote data to individual unfairness. In International Conference on Machine Learning, pp. 20168–20181. PMLR, 2023.

The benefit of the proposed method compared to data reweighting.

Existing data reweighting methods [1], and [2] assign weights for all samples according to their importance, and re-train the entire model using the weighted data. On the other hand, ALFA freezes the encoder and fine-tunes the last layer only using the augmented data. Therefore, ALFA is more effective than existing reweighing methods in terms of time and memory cost. Moreover, according to [3] data reweighting always overfit, so there's no guarantee that the worst-group test performance to be better than ERM. In contrast, we directly consider both the overestimated (high false positive rate) and underestimated group (high false negative rate), seeking a fair misclassification rate for them.

Description of the Pareto Frontier

Thanks for pointing out. We will revise the explanation for the figure. In detail, the Pareto Frontier shows the trade-off between accuracy and fairness metric, while the black solid line shows the optimal trade-off. The closer a point to the black line, the better the accuracy-fairness trade-off is. In most cases, ours is located in the upper-right of the figure, which means it can achieve fairness while minimizing the compromising of accuracy.

In fact, most of the existing frameworks shown in the figure are designed to improve fairness while maintaining accuracy. And only a few cases the fairness frameworks can improve accuracy too. It is hard to observe either decreasing or increasing accuracy in the experimental result. Moreover, we exclude some experimental results if they do not converge well, or are worse than the base rate.

Comparison to reweighting and baseline

We mainly focus on augmentation-based methods or some methods conducted in the latent space. However, as the reviewer asked, we can further compare the baseline and reweighting methods. In particular, we implement the state-of-the-art reweighting methods [2]. The revised Pareto Frontier, Figures 2 and 3 in revised paper's PDF file shows that ALFA outperforms the baseline and reweighting method. Moreover, the improvement in [2] is not consistent, whereas ALFA shows promising improvement.

The details about "data augmentation in latent space" and "unfair region"

In the binary classification scenario, a model might be biasedly trained due to data imbalance in the demographic group, producing inconsistent misclassification rates across the subgroup, which we define as an 'unfair region' in the paper.

Please see the Appendix K in the paper for the figures.
Here's an example of synthetic data explaining the concept of unfair region and how the decision boundary is rotated. We simply the binary classification task with a 2D Gaussian mixture data, consisting of two classes $y \in \\{0, 1\\}$ and two sensitive attributes, $A \in \\{0, 1\\}$ (indicating unprivileged and privileged groups.)

$

x \sim \begin{cases}
    group1: **N** (\begin{bmatrix}
       \mu \\\\
       \mu
     \end{bmatrix} , \sigma^2 )& \text{if} \\: y=1, a=1 \\\\
    group2: **N** (\begin{bmatrix}
       \mu \\\\
       \mu^\prime 
     \end{bmatrix} , \sigma^2)& \text{if} \\: y=0, a=1 \\\\
    group3: **N** (\begin{bmatrix}
       0 \\
    \mu
     \end{bmatrix} ,(K\sigma)^2) & \text{if}\\: y=1, a=0 \\\\
    group4: **N** (\begin{bmatrix}
       0 \\\\
       0
     \end{bmatrix} , (K\sigma)^2)& \text{if}\\: y=0, a=0
\end{cases}

$

where $\mu^\prime =r\mu, 0<r<1$ and $K>1$, where the number of samples in each group is $N_1 : N_2: N_3: N_4$. We arbitrarily set $K=3$, $r=0.7$, $\mu = 1$, $N_1 = N_2 = 100$, and $N_3=N_4=400$.

From the given synthetic data, we have a decision boundary like Figure 1 in the rebuttal pdf. Due to the imbalance in the dataset, the subgroup $a=1,y=0$ is overestimated as label $y=1$, and the subgroup $a=0, y=1$ is underestimated as label $y=0$. The difference in misclassification rate is shown in Figure 2. We define them as 'unfair regions' where the misclassification rate is disproportionately high.

Our fairness attack shifts the data distribution in a more biased way. In the synthetic data, as the demographic group $a=1$ is privileged to be predicted as $y=1$, group $a=1$ shifts towards the positive area, while group $a=0$ towards the negative area. Although this perturbation is counter-intuitive, the perturbed data directly recover the unfair region, producing a new decision boundary.

Figure 2 shows that when we vary the amount of perturbation $\delta$ from 0 to 0.2, the misclassification rate for each group changes accordingly. As the fairness evaluation metric $\Delta EOd$ is defined as the summation of the True Positive Rate (TPR) gap and False Positive Rate (FPR) gap between the demographic groups, we plot the TPR gap, FPR gap, $\Delta EOd$, and the overall misclassification rate. The lower subfigure in Figure 2 shows that the TPR gap and FPR gap drop significantly achieving fairness while maintaining the overall misclassification rate.

More general scenarios beyond binary labels and sensitive attributes

Please see the global response, addressing more general scenarios beyond binary classification with binary-sensitive attributes.

Novelty of the proposed method

Existing data augmentation methods are conducted on the input space. Although some of them utilize adversarial training to generate augmented data, the transparency and efficacy of transformation in the input space are not obvious because of the non-linearity of the deep models. Moreover, the objectives of adversarial training in the existing methods are different from ours. In detail, FAAP [1] adopts adversarial training to align the data to the auxiliary decision boundary predicting sensitive attributes rather than labels. AntiDRO [2] generates antidote data produced by Distributionally Robust Optimization (DRO) to select the worst-performing data. The generated data is used as data augmentation while having analogous features from the original dataset but different sensitive attributes. However, AntiDRO is designed only for individual fairness, and training on these antidote data cannot improve group fairness. In contrast, our combination of data augmentation and adversarial training is not trivial. In detail, we define 'unfair regions' where a demographic group is disproportionately overestimated (high false positive rate) and underestimated data (high false negative rate). The fairness attack automatically finds the unfair regions and directly makes the perturbed data recover the unfair regions. In this step, the perturbed data itself could be a biased dataset, but we utilize them as data augmentation which is a counter-intuitive way to mitigate bias by making the misclassification rates for each subgroup fair.

Computational complexity and training time

When it comes to computational complexity, ALFA requires the same memory space as the original training during the attacking step. In the fine-tuning step, as it stores perturbed latent features as a matrix form and is directly applied to the final classifier, only the final classifier needs the additional computational cost for the single latent feature matrix $Z^\prime \in \mathbb{R}^{N \times d}$ where $N$ is the number of samples and $d$ is the dimension of the latent feature. For example, let's assume we use $L$ layers of MLP, and each layer has the same dimension $d$ for simplicity. Then, the computational complexity is $\mathcal{O}(pdN + (L-1) d^2N + dN) )$ where $p$ is the number of features of the input. During the attacking step, we freeze the encoder and classifier, only computing the covariance and the perturbation update, where each of them takes $\mathcal{O}(dN)$ complexity. Moreover, during the fine-tuning step, it also takes complexity of $\mathcal{O}(dN)$. Therefore, the additional computational complexity for an iteration is $\mathcal{O}(3dN)$ including covariance calculation, perturbation update, and fine-tuning, which is significantly light compared to the ERM.

Stability of the proposed method

In typical adversarial training, the min-max optimization or bi-level (alternative) optimization may not be stable. However, it doesn't happen in ALFA since it requires a one-time attacking step to maximize the fairness constraint, and the augmented data is used in the training to minimize binary cross-entropy loss. Therefore, our framework has the same stability as traditional empirical risk minimization (ERM).

Extension to more complex dataset and model

Please see the global response addressing more complex situations.

Extension to diverse data types and scenarios

We adopt the fine-tuning strategy of freezing the encoder part, our experimental results show that this strategy effectively achieves fairness even for the image dataset (CelebA) and text dataset (Wiki), as shown in the manuscript and the global response.

We also further research the usage of ALFA in the multi-class classification, as it can be conceptualized as multiple One-Vs-All binary classifications as presented in the global response. Though ALFA can address multi-class classification, the improvement is not significant, and the dataset used in the experiment is not large enough in terms of its size and label space.

We summarize our future works here. First, we will find or construct a large dataset having fairness concerns since the existing benchmark datasets for fairness research are relatively small in terms of size and label space. As we have verified the effectiveness of finding and directly recovering unfair regions, we expect that ALFA can be applied to large label space. Second, we are interested in the long-tail classification scenario, where the label space is large and extremely imbalanced. Although this topic is not directly related to the fairness problem, we motivate this scenario because it might suffer from highly overestimated or underestimated classes, which our framework can address.

Comparison to existing perturbation methods in latent space

We have investigated the literature about latent perturbation methods for fairness and identified two existing works [1,2]. Both use GAN for the perturbation, and the direction of the perturbation is regarding the auxiliary decision boundary, produced by the sensitive attribute. In detail, [1] generates a perturbed sample having opposite sensitive attributes but the same target labels locating it in a symmetric point. Therefore, the role of perturbation is to make the number of samples and distribution similar across the subgroups. However, [1] is based on the strong assumption that each subgroup's data distribution should be symmetric about the decision boundaries. [2] also uses the sensitive attribute decision boundary and generates the perturbation towards that decision boundary. We show the conceptual image showing that this perturbation doesn't necessarily lead to the fair decision boundary, as shown in Figure 5(b) in the paper. Moreover, both [1] and [2] are designed only for the image dataset. In our paper, we empirically show that FAAP [2] produces poor results in tabular datasets as shown in Figure 2 and 3.

The novelty of our method can be summarized:

1. the adaptiveness to diverse types of data,
2. defining unfair regions where a demographic group is disproportionately overestimated (high false positive rate) and underestimated (high false negative rate),
3. directly recovering the unfair regions using fairness attack which is counter-intuitive usage of fairness constraint, generating biased dataset to mitigate bias.

Although we have identified only two existing methods, we are open to further discussion about the perturbation method in latent space for fairness. In particular, we will study further how to build interpretable latent perturbation for fairness.

Multi-class classification

For the multi-class classification, the decision boundaries are not linear, so our framework might not be directly applicable. However, multi-class classification can indeed be conceptualized as multiple binary classifications in a certain strategy called One-Vs-All. In this approach, for a problem with $N$ classes, we can create $N$ different binary classifiers. Each classifier is trained to distinguish between one of the classes and all other classes combined.

As each classifier can be seen as a binary classification task, we can utilize ALFA for the multi-class classification scenario by detecting unfair regions and recovering the region by fairness attack. The evaluation metric for multi-class fairness takes maximum Demographic Parity across the classes [4]. In details, $$ \Delta DP_{\text{multi}} = \max_{k \in [K]} \bigl \vert P(\hat{Y} = k \vert a=1 ) - P(\hat{Y} = k \vert a= 0 )\bigr \vert

where $\hat{Y}$ is the predicted class, and $k\in[K]$ denotes each class $k$ in the multi-class classification. Among existing datasets for fairness research, Drug dataset can be used for multi-class classification. In fact, the original labels of the Drug dataset are multi-class settings, from 'CL0' to 'CL6' indicating the frequency of drug abuse. We have binarized them as 'never used' and 'ever used' regardless of the frequency in the main paper. However, for the multi-class classification setting, we adopt the original multi-class setting and report the mean accuracy and $\Delta DP_{\text{multi}}$ with MLP. | Drug-multi| Accuracy | | $\Delta DP_{\text{multi}}$ | | | --- | --- | --- | --- | --- | | Model | mean | std. | mean | std. | | MLP | 0.5196|0.0032|0.1930|0.0132| | MLP + ALFA | 0.4960|0.0219|0.1733|0.0287| ## Extension to complex cases ### Complex model In our current paper, we used four tabular datasets and one image dataset with Logistic Regression, Multi-Layer Perceptron (MLP), and ResNet-18, respectively for experiments. To verify the effectiveness of our framework on the tabular dataset, we extend the experiment using 1) a ResNet-like backbone, and 2) a Transformer-like backbone designed for tabular datasets both proposed in [5]. The revised Pareto Frontier, Figure 6 and 7 in Appendix I.4 in the revised manuscript show that ALFA is applicable to more complex model. ### Complex dataset Moreover, we further explore the adaptability of the proposed method to the Natural Language Processing (NLP) dataset. We utilize Wikipedia Talk Toxicity Prediction [6] which is a comprehensive collection aimed at identifying toxic content within discussion comments posted on Wikipedia's talk pages, produced by the Conversation AI project. In this context, toxicity is defined as content that may be perceived as "rude, disrespectful, or unreasonable." It consists of over 100,000 comments from the English Wikipedia, each meticulously annotated by crowd workers, as delineated in their associated research paper. A challenge presented by this dataset is the underrepresentation of comments addressing sensitive subjects such as sexuality, religion, gender identity, and race. In this paper, the existence of sexuality terms such as 'gay', 'lesbian', 'bisexual', 'homosexual', 'straight', and 'heterosexual' is used as the sensitive attribute, 1 for existing, and 0 for absence. For the NLP dataset, we use a pre-trained word embedding model, Glove [7] to produce input data and use MLP and LSTM network [8] for the experiments. | Wiki | Accuracy | | $\Delta DP$ | | $\Delta EOd$ | | | --- | --- | --- | --- | --- | --- | --- | | Model | mean | std. | mean | std. | mean | std. | | MLP | 0.9345|0.0043|0.1735|0.0066|0.0734|0.0145| | MLP + ALFA | 0.9349|0.0002|0.1684|0.0019|0.0500|0.0049| | LSTM | 0.9294|0.0007|0.1922|0.0053|0.0806|0.0095| | LSTM + ALFA | 0.9280|0.1001|0.1872|0.0001|0.0607|0.0004|