Finetuning Text-to-Image Diffusion Models for Fairness

Response [1/2]

Thank you for the constructive feedback, which has helped us improve the quality of our paper. We respond below to your questions and concerns:

W1: This paper kind of mixing fairness and bias and uses both terms interchangeably, especially for the experiment evaluation part, they define the metric for bias by themself which reduces the credential of the evaluation. I wonder if any other metrics from other research papers have been used for evaluation. Is it possible to use well-defined fairness metrics like demographic parity/ equal opportunity, etc?

We apologize for the confusion arising from our uncareful use of the terms fairness and bias. They indeed have different meanings in our paper. We consider fairness as the distributional alignment with a user-defined target distribution, a flexible notion. Since our main experiments use a perfectly balanced target distribution, we define the metric bias to measure how far the generated images are from the perfectly balanced target distribution. When evaluating fairness for age, where we consider 75% young and 25% old as the target distribution, we directly report the percentage of age=old. We added clarification in the second paragraph of section 5.1, highlighted in blue.

We note that demographic parity (DP) and equal opportunity (EO) are defined for prediction problems, where every individual has a protected attribute $S$, a true label $Y$, and a prediction $\hat{Y}$ (chapter 3 of [1]). For T2I generative models, every generated image has only a protected attribute $S$. Therefore, DP and EO do not exactly apply here. In a way, the fairness problem in T2I generative models is simpler than that in prediction problems. Our objective is simply to control the generated images’ marginal distribution in terms of $S$ towards a target distribution.

We understand the reviewer has concern over whether the bias metric is truthful. In response, we have added plots of gender and racial representations by each prompt in Appendix Fig. A.2, A.3, A.4, and A.5, specifically at page15 & 16. These results provide a more detailed analysis than those presented in Tab. 1.

Q1: for the equation 4, what is the u?

We have significantly revised Section 4.1 to enhance the clarity of our presentation. Eq. 4 in the initial paper corresponds to Eq. 4 and 5 in the updated paper. $\\{\boldsymbol{u}_1,\cdots,\boldsymbol{u}_N\\}$ are $N$ iid samples drawn from the target distribution $\mathcal{D}$. Say we use the one-hot vectors [0,1] and [1,0] to denote male and female, and consider a balanced target distribution over these two classes. Then approximately half of $\\{\boldsymbol{u}_1,\cdots,\boldsymbol{u}_N\\}$ will be the vector [0,1] and the other half will be [1,0]. We construct $\\{\boldsymbol{u}_1,\cdots,\boldsymbol{u}_N\\}$ in this way so that it represents the target distribution and the optimal transport finds the most efficient modification of the current images towards the target distribution.

Q2: As for the alignment loss, have you tried some other metrics other than the optimal transport loss? And why did you choose the optimal transport metric?

We naturally arrived at the optimal transport (OT) formulation and didn’t experiment with other losses. We start by considering the following problem: suppose we have a set of 10 images, composed of 2 female and 8 male images, and our goal is to reconfigure this set to contain an equal number of female and male images, specifically 5 of each. The key question is determining which gender class every image should be reconfigured to. Additionally, it is essential to minimise alterations to the images when striving to attain an equal distribution of 5 female and 5 male images. This is naturally the OT (or linear assignment) problem, identifying which class every image should be transported to while minimising the transport distance. Finally, the OT problem considered here is low dimensional (2 for gender, 4 for race, and 8 for gender$\times$race) and therefore efficient to solve.

We sample $\\{\boldsymbol{u}_1,\cdots,\boldsymbol{u}_N\\}$ from the target distribution and compute the expectation of OT because it (1) acknowledges even a perfectly balanced distribution can still produce unbalanced finite samples, (2) takes into account the sensitivity of OT, and (3) produce a confidence measure for the target class assignment.

Response [2/2]

Q3: For the face classifier, how accurate it is? and if the combination of face detector and face classifier is not performed well on some datasets, how does it affect the debias experiment result?

We generally require the face detector to be accurate but we did not find it a significant problem in our experiments. Mistakenly identifying other regions as faces can lead to incorrect application of the distributional alignment loss and result in erroneous edits. Therefore, we use a relatively high detection threshold for the face detector. The failure to recognize faces is less problematic. It means some faces might not be considered in our fairness objective. To reduce the frequency of this oversight, we use two different face detectors, the face_recognition [2] and the insightface [3] packages.

We do not require the face classifiers to be highly accurate, such as over 95%. In our experiments, the gender classifier has 98% validation accuracy, the race classifier has 86% validation accuracy, and the age classifier has 90% validation accuracy. A few points are worth noting. Firstly, predicting certain attributes like race and age from visual cues can be inherently challenging. Secondly, the Distributional Alignment Loss (DAL) actually steers images towards those for which the classifier shows high confidence in distinguishing between male and female. These high-confidence classifications can still provide a reliable visual guidance of gender, even when the classifier might have lower accuracy for the low-confidence classifications.

References

[1] Solon Barocas, Moritz Hardt, and Arvind Narayanan, Fairness and Machine Learning: Limitations and Opportunities, 2023, https://fairmlbook.org

[2] https://github.com/ageitgey/face_recognition

[3] https://github.com/deepinsight/insightface

Finally, we note that many other improvements have been made in the revised paper. We are happy to consider more suggestions and respond to any additional questions you may have. We would appreciate it if you could increase your score should you find our revision satisfactory.

Response [2/2]

Q6: The optimal transport approach in Eq. (4) was a bit confusing to me at first because it differs from the usual setting in which the optimal transport scheme between two distributions is considered. By contrast, the authors consider an expectation over the optimal transport schemes between two vectors here. I believe everything is actually described correctly, but maybe there is a way to make this section even easier / more intuitive for readers to grasp? (Sorry, possibly not very helpful.)

Your observation is correct: we consider the optimal transport (OT), or the linear assignment problem, between two sets of vectors here, followed by computing the expectation of OT w.r.t. randomness of the target set of vectors $\\{\boldsymbol{u}\_1,\cdots,\boldsymbol{u}\_N\\}$. We are particularly grateful that you pointing out our presentation of Equation 4 was not clear. We have thoroughly revised Section 4.1 to enhance its clarity. Eq. (4) in the initial paper corresponds to Eq. (4) and (5) in the revised paper.

Q7: It is a little bit confusing that y denotes both the generated target labels (normal font) as well as the images generated by the frozen model (boldface).

We have changed $\boldsymbol{y}^{(i)}$ to $\boldsymbol{o}^{(i)}$ to denote the original images generated by the frozen model.

Q8: What does it mean that CLIP-ViT-bigG-14 and DINOv2 vit-g/14 are "more performative than the ones used in training"?

By “more performative”, we meant the CLIP-ViT-bigG-14 used in evaluation has higher ImageNet zero-shot acc than the CLIP ViT-H/14 used in training. Similarly, the DINOv2 vit-g/14 used in evaluation also has higher ImageNet zero-shot acc than the dinov2-vit-b/14 used in training. In the revised paper, we have removed this sentence due to its lack of clarity.

Q9: The authors might want to consider citing Lester et al. regarding soft prompt tuning?

Thank you for highlighting relevant previous research on soft prompt tuning. We have acknowledged it in the second-to-last paragraph of the introduction, highlighted in blue.

References

[1] Shtedritski et al., What does CLIP know about a red circle? Visual prompt engineering for VLMs, https://arxiv.org/abs/2304.06712

[2] Wang et al., FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralization, https://arxiv.org/abs/2210.14562

[3] Wolfe et al., Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias, https://dl.acm.org/doi/abs/10.1145/3593013.3594072

[4] Zhang and Ré, Contrastive Adapters for Foundation Model Group Robustness, https://arxiv.org/pdf/2207.07180.pdf

Finally, we are happy to incorporate any additional feedback and provide further clarification as needed. Thank you very much for the time and effort you invested in reviewing our work.

Response [1/2]

We are grateful and humbled by your highly positive reception of our work. We have diligently incorporated the suggestions from all reviewers into the revised paper. We respond below to your questions and concerns:

Q1: CLIP is not without its own issues; see e.g. Shtedritski et al., Wang et al., Wolfe et al., or Zhang and Ré. Could the authors discuss how this might affect the efficacy of their proposed method?

Thank you for pointing out our oversight regarding CLIP's own biases. We added acknowledgement in Section 4.1, page 4, highlighted in blue. We did not observe significant negative impact on our debiasing method arising from CLIP and DINO’s own social impacts. But we did observe DINO’s ViT architecture that splits an image into fixed-size patches can cause faces to appear rectangular in finetuning during our initial experiments. This observation is one of the factors that led us to use both CLIP and DINO.

The inherent image-image biases of CLIP and DINO, as opposed to text-image biases, could potentially affect our debiasing methods, given that we use CLIP and DINO to minimize image-image similarities. These biases might vary from the text-image biases documented in Shtedritski et al. [1], Wang et al. [2], Wolfe et al. [3], and Zhang and Ré [4]. We hypothesise that CLIP could be more biased than DINO in our context because CLIP is trained with text supervision while DINO is trained using image self-supervision. CLIP's training process, which aligns image features with text captions, is likely to introduce social biases. Our use of both CLIP and DINO might help mitigate the impact of these biases from CLIP.

Q2: How do the authors implement using different lambda values for different regions of the same image, as they describe at the end of section 4.1? ("We use a smaller weight for the non-face region ... and the smallest weight for the face region.") L_img is per image, not per pixel, no?

$\mathcal{L}\_{\textrm{img}}$ can differ between the detected face region and non-face region in the same image, and therefore is per pixel. We implement $\mathcal{L}\_{\textrm{img}}$ by adding a gradient manipulation layer right before inputting the generated image $\boldsymbol{x}^{(i)}$ to CLIP & DINO. By scaling the gradients at the level of the generated image $\boldsymbol{x}^{(i)}$, we can effectively implement a per-pixel $\mathcal{L}_{\textrm{img}}$.

Q3: I was confused about the training of the gender and race classifiers for the evaluation. Were separate classifiers trained for the training and the evaluation stage? If yes, why? …

We trained separate gender and race classifiers for evaluation. We trained them for twice longer iterations, using the same architecture and training data. We do so in an effort to make the evaluation fairer. The debiasing method should not overfit to the classifier used during training and generates images that the training classifier views as fair but not the separately trained evaluation classifier. We added clarification in the revised paper.

Q4: It was unclear to me which of (soft prompt, text encoder, U-Net) were actually finetuned for all of the main experiments in section 5.1?

For all of the main experiments, except explicitly stated otherwise, we finetune a LoRA adaptor with rank 50 applied on the text encoder. We added clarification in the first paragraph of Section 5.1, page 6, in the revised paper.

Q5: Can the authors think of (and comment on) any drawbacks of applying their method at scale and for many fairness-relevant properties simultaneously? Could there be any negative consequences (e.g., in terms of image quality or biases) when e.g. generating very different images (bears, tables, groups of people, ... )? Recent observations by Qi et al. could also be interesting to discuss in this regard.

Thank you for pointing us to consider the potential negative impacts of our debiasing finetuning. In response, we added an evaluation and discussion of the debiased SD’ generations for general prompts in Appendix page 17-22. We identify the debiased SD can occasionally and to a mild extent decrease the naturalness and smoothness of the generated images. Additionally, there are limitations in the generalizability of the debiasing effect. As illustrated in Figure A.11 on page 23, for the prompt "A beautiful painting of woman by Mandy Jurgens, Trending on artstation", the debiased SD increases the representation of Asian women, but does not similarly enhance the representation of Black and Indian women. We refer the reviewer to the more detailed discussion at Appendix Section A.5, specifically at page 14 & 17, highlighted in blue.

Response [2/2]

Q2: In Table 2, on unseen prompts, how good is the proposed method compared with the baselines? Does the fine-tuning have an overfitting problem?

We have included baselines comparisons in Table 2, page 7, highlighted in blue. We generally do not find overfitting, as demonstrated by the additional evaluation on general prompts (Appendix Section A.5, page 14 & 17~22). We did identify some negative impacts on image quality for the debiased SD, including occasional reduced naturalness and smoothness in the generated images. We discuss these in Appendix page 14 & 17, highlighted in blue.

Finally, we note that many other improvements have been made in the revised paper. We are happy to consider more suggestions and respond to any additional questions you may have. We would appreciate it if you could increase your score should you find our revision satisfactory.

Response [1/2]

Thank you for the constructive feedback, which has been very helpful in enhancing the quality of our paper. We respond below to your questions and concerns:

W1: The paper only tested the method on single-face generation, limiting the applicability of the proposed method.

We have updated our manuscript accordingly. (1) We've included an evaluation of multi-face generation in Table 3, page 7, highlighted in blue. The results show that the debiasing effect generalizes to images with multiple faces. (2) This evaluation also includes a bias metric calculated for all faces in the generated images, corroborating the original bias metric calculated using the most prominent face in each generated image. (3) Examples of generated images with multiple faces are shown in Appendix Figure A.13 & A.14, at page 29 & 30.

We have also considered directly debiasing all faces in generated images but find it is more appropriate to focus on debiasing the most prominent face in each image. This is because there may be legitimate correlations among individuals within the same image, and those at the forefront should not be simply equated with those in the background. As a result, defining an ideal fair outcome for all faces in the generated images is a more intricate task. But technically, our method is capable of directly debiasing all faces in the generated images.

W2: It is unclear whether fine-tuning with the fairness loss affects the quality and diversity of the generated images on general prompts.

We have incorporated an evaluation of general prompts in the revised paper, on Appendix Section A.5, page 14 & 17~22. In short, the quality of the generated images remains largely unchanged but we do identify occasional and relatively minor reduction in naturalness and smoothness of the generated images. We observed no reduction in diversity of the generated images in general. We observed increased diversity for the prompt “beautiful painting of a woman by Mandy Jurgens, trending on ArtStation”, even though the model was debiased for templated occupational prompts and never for the term "woman". We refer the reviewer to the more detailed discussion at Appendix page 14 & 17, highlighted in blue. It analyzes the quality, diversity, and fairness of images generated from general prompts.

W3: Although experiments in the paper show better performance than baseline methods, it is unclear how expensive the fine-tuning is compared with the baseline methods.

Debiasing SD may require a one-time training cost or additional compute at every inference time. Our fine-tuning method generally requires more training compute than baselines. But it effectively scales to a large number of prompts and does not incur additional inference compute. In the second paragraph of the introduction (highlighted in blue), we recognized existing methods are more lightweight.

In terms of specific comparisons, Ethical Intervention is a prompting method and does not incur additional training or inference compute. Based on our experiments, DebiasVL requires around 1 GPU-hour (using an A100) training. UCE requires around 24 GPU-hours to debias 50 prompts. But we find the required compute time increases drastically when we increase the number of prompts. Concept Algebra does not require training but doubles the inference cost, which can be undesirable. Our method requires 8 GPUs * 48 hours for debiasing 1000 prompts but does not increase inference compute.

We have included the training losses plot in Appendix Figure A.1, page 14. This plot indicates that although we train for 10k iterations for best performance, the finetuning achieves debiasing quite early, within 1k iterations, and can potentially be early stopped at 5k iterations. Therefore, the computational requirements for our method might not be as extensive as reported.

Q1: In Equation (4), how to obtain the expectation in practice?

We have significantly revised Section 4.1 to improve its clarity. Eq. 4 in the initial paper corresponds to Eq. 4 and 5 in the revised paper. In general, the expectation is approximated by repeatedly drawing $\\{\boldsymbol{u}\_1,\cdots,\boldsymbol{u}\_N\\}$ from the target distribution $\mathcal{D}^{N}$, computing the optimal transport from $\\{\boldsymbol{p}\_1,\cdots,\boldsymbol{p}\_N\\}$ to $\\{\boldsymbol{u}\_1,\cdots,\boldsymbol{u}\_N\\}$, and computing the average of these OTs. When the number of classes $K$ as well as $N$ is small, we can enumerate all possible samples from $\mathcal{D}^{N}$, compute OT for all of them, and analytically compute the expectation by weighing different OTs according to the target distribution.

Dear Reviewer T8Ls, as we are approaching the end of the author-reviewer discussion period, could you please let us know if our responses and additional experiments have adequately addressed your concerns? If there are any remaining comments, we are committed to providing prompt responses before the discussion period concludes. Thank you for your time and consideration.