A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks

Confidence interval in Text-to-Image Generation

Thank you for pointing out the lack of confidence intervals in our experimental results.

We acknowledge that text-to-image generation is not deterministic. This is why we conduct text-to-image generation 10 times with different seeds and use a unified evaluation metric to measure the skewed distribution across 10 different generations for each neutral prompt:

$

Skew = \frac{1}{|\mathcal{P}|} \sum_{p \in \mathcal{P}} \frac{\max(N_{p,m}, N_{p,f})}{10},

$

where $N_{p,m}$ and $N_{p,f}$ are the numbers of detected genders for each profession, $\mathcal{P}$ is a profession set. For example, if a model generates the same gender for a class 9 times out of 10, the Skew value for this profession becomes 90%. Although this metric does not include a confidence interval, it accounts for the randomness in text-to-image generation.

On the other hand, a metric used in [1] measures gender discrepancy as follows:

$

Discrepancy=\sqrt{\Bigl(\frac{N_{p,m}}{\vert \mathcal{P}\vert}-0.5\Bigr)^2 + \Bigl(\frac{N_{p,f}}{\vert \mathcal{P}\vert}-0.5\Bigr)^2},

$

This metric can be used in a single generation, so it may produce a confidence interval when conducted on multiple seeds. However, this metric does not effectively reflect bias in text-to-image generation. For example, assume we have a set of four gender-dominated professions: {nurse, dancer, doctor, engineer}. If a biased text-to-image model always produces female images for nurse and dancer, and male images for doctor and engineer, the evaluation metric becomes 0, even though the bias is prevalent. This metric fails to demonstrate bias, even with a confidence interval over 10 runs. Therefore, our evaluation metric, which computes Skew over 10 runs, more effectively reflects the bias in text-to-image generation, accounting for randomness.

On the other hand, demonstrating a confidence interval for the mismatch rate for gender-prompt is possible as it can be measured by a single experiment. As we have already executed the experiment with 10 different seeds, we report the confidence interval for mismatch rates in Table 1 of the rebuttal PDF file.

[1] Chuang, Ching-Yao, et al. "Debiasing vision-language models via biased prompts." arXiv preprint arXiv:2302.00070 (2023).

Confidence interval for all downstream tasks

We agree with the importance of including confidence intervals in experimental results, even when the model is deterministic, as is the case with pre-trained models used in zero-shot classification, text-to-image retrieval, and image captioning.

For zero-shot classification and image captioning, we conducted experiments on the full test dataset. We utilized the bootstrapping technique by generating 1000 datasets with replacement and reported the confidence intervals in Table 1 of the rebuttal PDF file for zero-shot classification and image captioning.

In the case of text-to-image retrieval, the test set is a subset of the entire Flickr30K dataset, created by randomly sampling 1000 samples. We extended the experiments by using 10 different test sets to obtain the confidence intervals. These results are also included in Table 1 of the rebuttal PDF file.

Robust superiority of SFID

Overall, when we include confidence intervals for all downstream tasks via bootstrapping or additional experiments, SFID consistently outperforms the comparison methods in all cases. This robust performance across various scenarios highlights the effectiveness and reliability of SFID in mitigating bias.

Comparison with Prompt-debias (Chuang et. al.)

Thank you for recognizing a similar approach and the novelty of our work. We agree that while their goal in debiasing VLM embeddings is analogous to ours, our methodologies differ significantly.

We include the experimental results of Prompt-debias in Table 1 of the rebuttal PDF file. It turns out that while Prompt-debias can mitigate bias in various downstream tasks, our method, SFID, performs significantly better. Moreover, in text-to-image generation, Prompt-debias works only with neutral prompts but deteriorates the bias in the case of mismatch rates for gender-prompts.

The reason our method works better than Prompt-debias is due to the bias amplification in the decoder or image encoder. Prompt-debias's application is limited to the text embedding, which is the output of the text encoder. In contrast, SFID mitigates bias in either the encoder or decoder output, or both, providing a more comprehensive approach to debiasing.

Visualization of embedding translation by SFID

We include a visual aid for the embedding to indicate the bias-free representation achieved by SFID in Figure 2 of our rebuttal PDF. As SFID aims to obscure all the features in the important index by imputing ambiguous values, all the data points are gathered at a single point to mute the information related to the sensitive attributes, while maintaining the features in other indices as they are. It is important to note that although this translation is enforced to a single point, it still remains within the distribution of the original samples, as shown in Figure 3 of our main paper.

Grammar and Structure

Thank you for pointing out the organization of our paper. We will address the minor grammatical errors and ensure clarity throughout. In Section 3, it is true that it serves both as preliminaries and motivation section. Experimental setups are included in this section to provide readers with context regarding each downstream task and experimental setting, ensuring consistency. We will revise and reorganize our content to enhance readability.

Limitation and Future work of SFID

Thank you for pointing out the limitations section. We have identified potential limitations and areas for future exploration in our global rebuttal.

Novelty of the proposed method

We appreciate the reviewer's opinion regarding the simplicity and novelty of utilizing RandomForest in our framework.

While the RandomForest algorithm itself is well-known and simple, the novelty of our work lies not in the use of RandomForest alone, but in the innovative way we integrate it within our Selective Feature Imputation for Debiasing (SFID) framework. The SFID methodology introduces several novelties:

• Selective Feature Imputation: Our approach combines feature pruning with low-confidence imputation (LCI) to replace bias-causing features with bias-free representations. This novel technique maintains the semantic integrity of the embeddings while effectively reducing bias.
• Efficiency and Cost-Effectiveness: SFID eliminates the need for costly retraining of pre-trained VLMs and does not require paired text-image datasets, making it a cost-effective and efficient solution for debiasing.
• Versatility Across Modalities and Tasks: Unlike existing methods that focus on specific modalities or tasks, SFID is designed to be seamlessly integrated into various components of VLMs, including both encoders and decoders, across a wide range of tasks such as zero-shot classification, text-to-image retrieval, image captioning, and text-to-image generation. We emphasize that none of the existing work can debias such a seamless number of downstream tasks without further training or fine-tuning.
• Empirical Validation and Effectiveness: Our experimental results demonstrate the practical effectiveness of SFID in mitigating biases across various tasks without compromising performance. This is evidenced by significant improvements in fairness metrics across multiple benchmarks. The comparative analysis with other debiasing methods, such as DeAR, CLIP-clip, and Prompt-debias (newly added), highlights the superior performance of SFID, further validating the novelty and utility of our approach.

While RandomForest is indeed a well-known algorithm, the novelty of our work lies in the innovative application and integration of this algorithm within the SFID framework. This novel approach offers a significant advancement in the field of debiasing VLMs, as demonstrated by our empirical results and comparative analysis. Furthermore, we highlight the necessity for a unified debiasing framework, such as SFID, to address the challenges posed by the emergence of diverse VLMs applications.

Multiple sensitive attributes in racial bias

Thanks for pointing out the details how to define $\Delta DP$ in multiple attributes and how SFID mitigates such bias.

SFID is free from the type of bias when the attribute labels are given such as gender and race in FairFace dataset, even in scenarios involving multiple attributes. We define $\Delta DP$ for multiple attributes by adopting the evaluation metric from [1] and [2], which measures the maximum discrepancy across the sensitive attributes for each class. This metric is defined as follows:

$

DP_c = \max_{i \in S} \max_{j \in S \setminus {i}} \Bigg( \bigl| P(y=c \mid a=i) - P(y=c \mid a=j) \bigr| \Bigg)

$

where $i, j \in S$, $c$ is a class, and $S$ denotes the set of multiple sensitive attributes. We consider the mean and maximum values of $DP_c$ to evaluate the bias across the classes. By applying this metric, we can effectively measure and demonstrate the reduction in bias achieved by SFID, despite the differences in sensitive attributes between the training and test sets.

To support the adaptability of SFID in addressing different types of bias with multiple attributes, we report experimental results for racial bias in our global rebuttal. The table in the global rebuttal demonstrates that SFID can effectively mitigate racial bias even when dealing with multiple sensitive attributes. This showcases SFID's versatility and robustness in reducing bias across various scenarios.

[1] Foulds, James R., et al. "An intersectional definition of fairness." 2020 IEEE 36th International Conference on Data Engineering (ICDE). IEEE, 2020.
[2] Denis, Christophe, et al. "Fairness guarantee in multi-class classification." arXiv preprint arXiv:2109.13642 (2021).

Impact of the number of important features

Thank you for pointing out how to choose the number of important features. Indeed, the number of important features significantly affects performance, but we have an easy way to determine the optimal number, $k$, as shown in Appendix A.1 of the main paper.

In SFID, selecting $k$ is crucial for performance. As shown in Figure 5 of Appendix A.1, we determine an appropriate value for $k$ by identifying the elbow point in the feature importance curve, obtained by sorting the indices by their importance. According to Figure 5, $k=50$ is a reasonable elbow point.

We also analyze the impact of $k$ on performance. As expected, we achieve the best trade-off between utility and fairness when $k=50$, as shown in Figure 6 and Figure 7 in Appendix C of the main paper.

Overall, we have a solid guideline for choosing the number of important features, $k$, and acknowledge its impact on performance. The observed trend aligns with our intuition.

Visualization: Correlation of identified features to social biases

Thank you for suggesting this valuable enhancement to our paper.

We visualize how the important features are correlated to social biases by showing GradCAM visualizations, as presented in Figure 1 of our rebuttal PDF file. For example, the more important features highlight human faces, while the least important features are correlated with the image background.

SFID imputes the representations at important indices with the values from low-confidence samples, making the face-related features ambiguous. Despite this imputation, the replaced values remain within the distribution of the original samples, as shown in Figure 3 of our main paper.

How SFID Mitigates Bias in Various Attributes, Including Racial Bias

Thank you for your detailed observations on how SFID mitigates different types of bias, which is even more complex.

SFID effectively addresses biases when attribute labels are provided such as gender and race in the FairFace dataset. The RandomForest attribute classifier is trained to identify important features within the frozen image embedding. For instance, as illustrated in Figure 1 of our rebuttal PDF, GradCAM highlights that the 'important features' pertain to human faces, allowing for the recognition of identity in both gender and race attribute cases. SFID aims to impute the values in these important features with those of low-confidence samples, rendering the final embedding ambiguous in terms of the targeted attribute.

To demonstrate the adaptability of SFID to different types of attributes, we have included experimental results addressing racial bias in our global rebuttal. Both Figure 1 in the PDF and the Table in the global rebuttal illustrate SFID's capability to mitigate more complex biases, showcasing its overall effectiveness.

Effectiveness in using RandomForeset

We appreciate the reviewer's insightful feedback regarding the potential limitations of using RandomForest for feature importance. However, we argue that using RandomForest for feature importance is both theoretically and empirically sufficient and effective.

• Contrary to some concerns, RandomForest is capable of capturing complex, non-linear relationships due to its ensemble nature. By combining multiple decision trees, each considering different feature subsets and data splits, RandomForest can effectively model complex interactions and dependencies among features.
• Specifically, given the ample representation of images or text extracted by pre-trained neural networks, RandomForest's role is to identify important features while considering the dependencies between them.
• In short, RandomForest is an interpretable method for determining feature importance, robust against overfitting, and efficient in handling large datasets.
• Moreover, the empirical results in the paper demonstrate the effectiveness and versatility of using RandomForest within our framework.

While we have demonstrated the effectiveness of RandomForest in our current work, we acknowledge the importance of exploring additional techniques that might further enhance the modeling of complex relationships. Future research could incorporate advanced methods for identifying important features while improving the quality of representation.

Comparison beyond DeAR and CLIP-clip

Thank you for your feedback. We acknowledge that many debiasing methods have been proposed recently, but most focus on specific downstream tasks, with only a few demonstrating versatility across various tasks, such as DeAR and CLIP-clip. The paper suggested by the reviewer, Prompt-debias [1], is limited to scenarios where the VLM input is text, making it inapplicable to tasks such as image captioning.

For the other three tasks, we have reported the results in our rebuttal PDF file. The results in Table 1 show that while Prompt-debias mitigates bias in zero-shot classification and text-to-image retrieval, our method, SFID, is superior in mitigating bias in these tasks compared to Prompt-debias. In the case of text-to-image generation, Prompt-debias shows slightly better performance with neutral prompts with CoDi, but significantly deteriorates the bias with gender prompts.

In conclusion, few methods are versatile across various downstream tasks. SFID demonstrates superior performance in mitigating various types of bias across all the downstream tasks we evaluated.

[1] Chuang, Ching-Yao, et al. "Debiasing vision-language models via biased prompts." arXiv preprint arXiv:2302.00070 (2023).

Notation in Section 3

Thank you for pointing out the readability issues. We will revise the notations to be simpler and will add an appendix to provide more detailed explanations and visualizations for each evaluation metric to increase readability.

Computational Cost

We are happy to address the computational cost for each component and data type. The overall computational cost is significantly low since SFID only utilizes frozen embeddings from a pre-trained network, without involving any training or fine-tuning of the neural network. Regardless of the network size, we break down the computational cost into two parts: a) identifying feature importance and b) inference. Both tables demonstrate that applying SFID does not increase the computational cost in practice.

Identifying Feature Importance (One-time for each component)

Inference

Visualization of embedding translation by SFID

We include a visual aid for the embedding to indicate the bias-free representation achieved by SFID in Figure 2 of our rebuttal PDF. As SFID aims to obscure all the features in the important index by imputing ambiguous values, all the data points are gathered at a single point to mute the information related to the sensitive attributes, while maintaining the features in other indices as they are. It is important to note that although this translation is enforced to a single point, it still remains within the distribution of the original samples, as shown in Figure 3 of our main paper.

Limitation and Future work of SFID

Thank you for pointing out the limitations section. We have identified potential limitations and areas for future exploration in our global rebuttal.