Fair Adversarial Training: on the Adversarial Attack and Defense of Fairness

Thank you for the detailed comment.

[Weakness 1 (W1): Experimental setup]

Sorry for the confusion. Detailed experimental setup can be found in Section A of the appendix.

[W2: Source code]

We 'll provide the source code in final version.

Thank you for the detailed comment.

[Weakness 1 (W1): Explanation of Figure 1]

Sorry for the confusion. As shown in Fig. 1, under the fairness attack, the black TPR increases to 1, while the black TNR decreases to 0. This means all the black people are predicted as "high risk". On the contrary, the white TPR decreases to 0, while the white TNR increases to 1. This means all the white people are predicted as "low risk". Consequently, such disparity leads to crucial injustice problems.

[W2: Presentation issue]

"ny" should be "by" in the content. We are sorry for the typos and we 'll fix them in final version.

Thank you for the detailed comment. For a better clarification of our framework, we refer to the revised version of our paper in the following discussion. Specifically, in the revised version, Thm1&2 are about guaranteeing fairness robustness using accuracy robustness, and Thm3 is about guaranteeing accuracy robustness using fairness robustness.

[Weakness 1 (W1), W2: Explanation of Theorem 1 and 2]

We refer to Theorem 2 in the revised paper to show the practical implication of Theorem 1 in the revised paper regarding the fair adversarial training framework. Specifically, in the revised version, the upper-bound of fairness robustness regarding samples in the "Opposite Direction" in Theorem 1 allows us to upper-bound changes of EOd and DI under the fairness attack using static fairness, i.e., fairness issues stemming from data or algorithmic biases without the adversarial perturbtion, and accuracy robustness of the model. Consequently, Theorem 1 and 2 in the revised version provides theoretical guarantee to our fair adversarial training framework.

[W3: Discussion of the experiments]

Thank you for the suggestion. We 'll inlcude more analysis regarding the experimental results in final version.

[W4: Motivation of the fairness attack]

We agree with the reviewer that static fairness, namely fairness issues stemming from data or algorithmic biases is an important problem, and therefore our fair adversarial framework takes into account both static fairness and fairness robustness, as shown in Eq. 5. Similar to that of the accuracy attack, in terms of pracitcal applicability, it is peculiar to intentionally depreciate a model's accuracy or fairness, unless out of some egregious concern, as dicussed in Fig.1 of the main paper. Instead, the fairness attack can be considered as means of measuring of generalizability of model in terms of fairness on some unknown data under the worst-case scenario. As discussed in Section 6.2 of the main paper, under a successful accuracy attack, the accuracy of the model decreases to 0, while the DI or EOd does not show a monotonic increase as the perturbation level of the accuracy attack increases. This shows that the accuracy attack alone does not provide adequate information in terms of the generalizability regarding the fairness of the model under dynamic secnarios. Therefore, as with the accuracy attack, we propose the fairness attack to quantify the generalizability of a model in terms of fairness regarding the worst-case scenario.

[W4: Results of AutoAttack]

Thank you for the suggestion. We show results using AutoAttack [1] to further validate the effectiveness of our method:

The table above shows results of AutoAttack (the objective of white-box attacks are replaced by $L_{\text{DI}}$ the objective of fairness attack) on Adult dataset under perturbation level $\epsilon=0.2$. Compared with the baselines, our methods show remarkable improvement in terms of fairness robustness. We 'll include full results on other datasets in final version.

[W5: Notational issues]

Thank you for the detailed suggestion. Please refer to the revised version for a more comprehensive version of notations.

[Question 1 (Q1): Accuracy in the experiments]

The accuracy in the experiments refers to both standard accuracy and robust accuracy. As shown in Fig. 2 and 3, when the perturbation level $\epsilon$ equals zero, it corresponds to the standard accuracy results. Under non-zero perturbation levels, it gives the robust accuracy results.

[Q2: Calculation of $D(x_{\text{FN},female})$]

The values in Table 2 are calculated based on the group-wise average. For example, $D(x_{\text{FN},female}) = \frac{1}{|\{\text{FN},female\}|}\sum_{x_i \in \{\text{FN},female\}} D(x_i)$.

[Q3: Setting of $\gamma$]

The $L$ term in Eq. 5 corresponds to the fairness measures, i.e., DI or EOd in our discussion. It is hard to directly set $\gamma$ as some constant during training, as the fairness measures are intractable and non-differentiable. Instead, during training, $\gamma$ is integrated as a Lagrangian multiplier as a fairness regularizer where the multiplier value varies by the methods, and the hyperparameters of different methods (preprocessing, in-processing and post-processing) are tuned as suggested by the authors in the original paper to find the best performance in terms of fairness.

[1] Croce, F. and Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks, In ICML, 2020

Thank you for the detailed comment. For a better clarification regarding the notation and our framework, we refer to the revised version of our paper in the following discussion. Specifically, in the revised version, Thm1&2 are about guaranteeing fairness robustness using accuracy robustness, and Thm3 is about guaranteeing accuracy robustness using fairness robustness.

[Weakness 1 (W1): Formulation of the fairness attack]

We would like to clarify that both the fairness attack and the accuracy attack employ collective adversarial objectives, rather than point-wise objectives. Regarding the poisoning attacks, the adversarial objectives take into account the statistical properties of the classifier at a collective level, such as accuracy or DI/EOd subject to the specified targets, rather than the point-wise objectives as in the evasion attacks.

Moreover, similar to that of the accuracy attack, while the adversarial objective is targeted against the statistical properties of the model, i.e., DI and EOd of the model, the fairness attack on different data points are independent. This is further discussed in Eq. 2 and Eq. 3 of the main paper, which is a instance-wise reformulation of Eq. 1. While the estimation of DI and EOd relies on group information, regarding gradient-based fairness attack, the adversarial objective $L_{\text{DI}}$ only takes into account the soft predicton of $x_i$, and information from all the other points can be treated as constants, as they do not affect the update of the fairness adversarial samples by $x_i$. Therefore, the fairness attack applied to one single instance does not influence the subsequential attacks for the others, and the adversarial attack is performed based on purely the gradient information of the trained model w.r.t. the very instance.

[W2: Definition of fairness robustness]

The definition of fairness robustness is presented in the introduction of main paper: "Here we similarly define fairness robustness as a model’s ability to resist adversarial perturbation by an fairness attack and remain same predictions on clean and fairness adversarial samples.", i.e., $\mathbb{P}\_{x \sim \mathcal{D}\_{\mathcal{X}}}[\exists \|\epsilon^{\text{Fair}}\| \le \epsilon\_0$ s.t. $f(x+\epsilon^{\text{Fair}}) \neq f(x)]$ where $\epsilon^{\text{Fair}}$ is specified by the fairness attack. This reflects the consistency of the classifier to maintain the same predictions both with and without the adversarial perturbation under the fairness attack. Similarly, in the introduction, we define accuracy robustness as "a model's ability to resist adversarial perturbation by an accuracy attack and remain same predictions on clean and accuracy adversarial samples.", i.e., $\mathbb{P}\_{x \sim \mathcal{D}\_{\mathcal{X}}}[\exists \|\epsilon^{\text{Acc}}\| \le \epsilon\_0$ s.t. $f(x+\epsilon^{\text{Acc}}) \neq f(x)]$ where $\epsilon^{\text{Acc}}$ is specified by the accuracy attack. This reflects the consistency of the classifier to maintain the same predictions both with and without the adversarial perturbation under the accuracy attack.

[W3: Alignment of robustness]

Regarding the effect of adversarial attack, both the magnitude and the direction of the attack matter. As discussed in Section 4 of the main paper, The fairness attack and the accuracy attack acts upon different directions regarding different samples. Under the same step size and the same perturbation level up to the same iterations, the fairness attack and the accuracy attack act equivalently upon samples in the 'Same Direction' category. For samples in the 'Opposite Direction' category, the fairness attack and the accuracy attack act towards the opposite directions. We cannot assert that "weaker attacks on cross-entropy" leads to trivial alignment between fairness robustness and accuracy robustness, owing to the misalignment in the direction of the two attacks. Therefore, we show in Theorem 1 in the revised paper that the fairness robustness, measured by the change in cross-entropy loss before and after the fairness attack, regarding $x_{\text{FN},1}$ is upper-bounded by the accuracy robustness of $x_{\text{FN},0}$, i.e., the fairness robustness guarantee by accuracy robustness. Moreover, our discussion in Theorem 3 in the revised paper also points to the non-trivialness of the problem: if the fairness robustness amounts to weaker robustness than the accuracy robustness, we cannot have the accuracy robustness guarantee by the fairness robustness.