Should We Attend More or Less? Modulating Attention for Fairness

Thank you for your detailed review and engagement in our work. We are delighted to hear that you consider our work interesting and with thorough evaluations. We address your main points below.

Results are difficult to parse and not contextualized. Even in Appendix B, there is just a lot of variety in the results.

We are sorry about this confusion. The results in Figure 2 and Appendix B demonstrate that fairness is not always maximized by attending to a wider context, as previously claimed by Attanasio et al., (2022), and that the architecture and dataset determine whether the model needs to focus on a wider/narrower context to improve fairness. We will clarify this confusion in the manuscript.

It is unclear why and when minimizing versus maximizing attention entropy is preferred

Thanks for bringing up this important point. If the model attends to stereotypical tokens in a narrow context, attention entropy maximization makes the model focus on a wider context, leading to less bias. However, if the model already attends to a wide context containing stereotypical words, attention entropy minimization makes the model focus on a narrower context, which also reduces bias.

Examples of the impacts of applying EAT would also be helpful. Show the attention outputs pre- and post-EAT to demonstrate that the bias is being mitigated.

This is a great suggestion. We will add more qualitative examples, as shown below.

For toxicity detection using BERT and Input “Hello Dan, please fix LanguageRo.php first, it has explicit references to Wikipedia.”:

Input “Freedom will and must prevail, your tyranny will come to an end, Nicholas.”:

Using RoBERTa and Input “Emilia, do you find more information about this Swedish diva? If you do so, help complete this article!”:

I'm having a hard time seeing the significance of this work.

As the reviewer had rightly pointed out in their summary, our main contribution is showing empirically and conceptually that the previous work by Attanasio et al., (2022), claiming that maximizing attention entropy always improves fairness, is not general. We will make our main contribution clearer in the revised manuscript.

Thank you for your constructive feedback and thoughtful engagement in our work. We are motivated to hear that you consider our work well-presented, intuitive, and with thorough analyses. We address your main points below.

the paper would benefit from including more experimental results using other popular debiasing methods such as Mable [1] and [2].

This is a very interesting suggestion. We added MABLE as a baseline and launched the experiments comparing it to EAT. If we get the results before the rebuttal deadline, we will discuss them, but we will add them to the manuscript in any case. Given that MABLE and Nullspace Projection are in-processing and pre-processing methods, respectively, they are both complementary to our proposed post-processing method (and hence can be combined with it).

GLUE benchmarks or any other downstream classification tasks, are necessary.

We launched experiments to measure the effect of attention modulation on downstream tasks. We will discuss the results if we get them before the rebuttal period ends, and we will add them to the manuscript in any case.

the choice of beta can be applied to different layers, implying the knowledge of social bias or spurious correlation is stored among various layers.

This is an excellent comment. Previous work has shown that the first layers are more critical as they learn more abstract concepts [1], but intuitively any layer could learn bias. This motivates varying the attention modulation based on the layer, which comes at the cost of rendering the hyperparameter search more difficult. We kept the attention modulation factor the same in all layers for simplicity.

[1] Sajjad et al. On the effect of dropping layers of pre-trained transformer models. Computer Speech & Language, 2023.

Do the authors have any case studies regarding how the choice of beta is relevant to the linguistic characteristics of a sentence?

This is an interesting question. Based on our understanding, the choice of beta does not depend on the linguistic characteristic of the sentence per se, it rather depends on the model’s attention to stereotypical tokens. If the model attends to stereotypical tokens in a narrow context, the optimal beta would be smaller than 1, making the model focus on a wider context. However, if the model already attends to a wide context containing stereotypical words, the optimal value of beta would be larger than 1, to make the model focus on a narrower context.

Thank you for your detailed review and comments. We are encouraged that you find our work interesting, empirically useful, and computationally efficient. We address your main points below.

The method is arguably not a great advance over previous work with temp scaing.

We agree that the idea of temperature scaling exists already in the literature, as Reviewer q9VN and czv4 had rightly pointed out. Our main contribution lies in demonstrating, both empirically and conceptually, that attention entropy maximization does not always lead to fairer models, as previously claimed by Attanasio et al., (2022).

It is not clear how to trade off the decreased performance and improved AUC equality score.

Thanks for bringing this up. As mentioned in Section 5.4, we maximize fairness while ensuring that the performance drop does not exceed 3% on the validation dataset.

Optimal parameters vary a lot across datasets, so it would not be useful with data that is not labeled for the sensitive attributes.

We agree that the optimal value of the temperature scaling hyperparameter varies based on the setting. For different architectures and datasets, models memorize different shortcuts to solve the task by focusing on an overly narrow/wide context, which leads to bias. The optimal beta mitigates this bias by modulating the model’s attention.

The datasets and metrics used seem specific -- the choices could have been motivated more and compared to alternatives.

Thanks a lot for your comment. We chose the datasets and metrics that are widely used in the fairness literature [1-4]. We will add this information to the manuscript to make it clear.

[1] Zhang et al. Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting. ACL, 2020.

[2] Reddy et al. Benchmarking bias mitigation algorithms in representation learning through fairness metrics. NeurIPS Datasets and Benchmarks Track, 2021.

[3] Shen et al. Optimising Equal Opportunity Fairness in Model Training. NAACL, 2022

[4] Dixon et al. Measuring and mitigating unintended bias in text classification. AIES, 2018.

Thank you for your time in providing a thorough and complete review. We appreciate your interest in understanding all the details in our work and address your main points below.

The temperature scaling hyperparameter indicates that this approach needs to be configured for different data distributions/architectures.

We agree that one of the limitations of our method is that it assumes having access to validation data from the same distribution as the test data to choose the optimal temperature scaling.

The technical novelty in this paper is limited as the approach is an application of temperature scaling

We agree that the idea of temperature scaling exists in the literature, as the reviewer rightly pointed out. The main contribution of our work is challenging the existing claim that maximizing attention distribution entropy always improves fairness, by showing empirically and conceptually that this is not a general observation.

Qualitative insights are completely missing.

Thanks for pointing this out. Table 3 in Section 5.4 provides some qualitative results, but we will add more examples to strengthen the paper, as shown below.

For toxicity detection using BERT and Input “Hello Dan, please fix LanguageRo.php first, it has explicit references to Wikipedia.”:

Using RoBERTa and Input “Emilia, do you find more information about this Swedish diva? If you do so, help complete this article!”:

How confident are you that a single bias/fairness metrics supports the findings in this paper

In line with related works [1-3], we use four bias assessment metrics for text classification. For gender bias, we use demographic parity, equality of odds, and equality of opportunity; and for general social biases, we use pinned AUC. Due to space limitations, some results are shown in Appendix B. We also use BOLD benchmark for bias assessment in text generation.

[1] Reddy et al. Benchmarking bias mitigation algorithms in representation learning through fairness metrics. NeurIPS Datasets and Benchmarks Track, 2021.

[2] Shen et al. Optimising Equal Opportunity Fairness in Model Training. NAACL, 2022

[3] Dixon et al. Measuring and mitigating unintended bias in text classification. AIES, 2018.