CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?

Thank you for your thoughtful review of our paper. We greatly appreciate your positive remarks on the definitions of various types of bias and our approach of testing across different datasets and models. We acknowledge your constructive criticism regarding the clarity of our figures and the need for more explicit interpretation of our results. Please see our detailed response below.

Addressing the concerns about the interpretability of our figures, in Figures 2 and 3, the bars that we compare are for "baseline", "balanced", and "proxies" within each experimental configuration. For example, we compare them when using 1B examples on ViT-B backbone with 768 representation length. But, as you correctly mention, we are also interested in comparing across other dimensions, such as the size of the training data to show that even with extra training data, bias persists. We will clarify this in the revised version of the paper.

In Figure 4, both “balanced” and “proxies” have lower AB than the baseline in most of the experiments, with $p$-values of less than $10^{-6}$. The error bars are for the three random seeds used for each experiment, while the $p$-values are calculated across all experiments (e.g. seeds, model size, representation length).

Answers to Questions

• We believe our findings are more nuanced. For example, it might turn out that training on large datasets helps CLIP learn a more accurate representation of sensitive attributes and labels, thereby separating them. In our experiments, however, we answer that question by showing that a bigger dataset doesn’t fix the bias. Also, it was not clear how effective fine-tuning would be for AB and RB. Our experiments provide a detailed answer: RB is sensitive to the last distribution it sees on the data so fine-tuning is as effective as training on balanced data from the start, but in AB the effectiveness of fine-tuning is more gradual and depends on the length of its duration. We also study the impact of using proxies as well; e.g. proxies help RB but hurt AB, and we offer an explanation. We also study in detail the impact on the model's quality, and provide a detailed explanation in Appendix A.6 for why classification and retrieval metrics are impacted differently. We hope this clarifies our contribution.

• We agree with your assessment. We hope that since our proposed algorithm is designed to be adaptable to various sensitive attributes and labels by relaxing many assumptions (e.g. attributes and labels can overlap), it is flexible enough to incorporate different scenarios that may be pertinent in real-world applications.

• We acknowledge the complexity of this issue. As you mentioned, real-world disparities can continue to have an influence on the model even after balancing the data, and we observe evidence for this in our experiments. For example, AB is not removed entirely even after decorrelating perceived gender with occupation. But, we believe that understanding the extent of how effective balancing the data is for multimodal learning, such as CLIP, would be a step in the right direction.

We hope these responses address your concerns and clarify our approach. Again, we are grateful for your constructive feedback, which has highlighted areas for further exploration and improvement in our research.

If there are any other questions or concerns, we would appreciate it if you let us know so we can respond to them during the discussion and improve the manuscript.

Thank you

Dear reviewer,

We would like to kindly inquire if our responses have satisfactorily addressed your concerns and questions. We are open to further clarification if needed. If we have addressed your concerns, we would appreciate it if you consider revising your review/score accordingly. Otherwise, please let us know so we can respond to your inquiries during the discussion period.

Sincerely

Thank you for your constructive feedback. We appreciate your recognition of the empirical evidence provided, the principled approach of our data balancing algorithm, and the relevance of our study to real-world applications.

Addressing your concerns regarding the limited number of sensitive attributes in our experiments, we acknowledge that our current focus has been primarily on perceived gender and occupation. This decision was intentional. First, we did not have skin tone annotations in the downstream evaluation datasets. For example, FairFace and UTKFace use race/ethnicity, not skin tone. Second, our goal was to determine how biases in the data (both RB and AB) transfer to their corresponding biases in the model. We believe that using perceived gender and occupation would still provide useful insights that can illuminate this relationship.

About the experiments of the data balancing algorithm, those comparisons with classical preprocessing methods are available in Appendix A.2. We prefer to keep them in the appendix since the primary scope of our work is to study the impact of balancing the data on CLIP models.

Answers to the Questions

• We treat proxies as labels, so they are included in the set of variables $**y**_r$ in Equation 3. The algorithm identifies the set of weights that satisfy all constraints, including the correlation constraints between the sensitive attributes and proxies. For instance, we remove any correlation between the proxy “computer” and perceived gender.

• This phenomenon might be attributed to the complex interactions between different variables in the dataset. In [1], for example, the authors find that many factors influence the model’s prediction of sensitive attributes, such as gender, including non-obvious ones such as beliefs and communion. We will elaborate on this in the revised version of the paper. This is why we do statistical analysis to confirm the significance of our findings.

• $\alpha$ and $\beta$ are Lagrange multipliers associated with the constraints $q\le Q$ and $q\ge 0$ respectively (in our experiments $Q=1$). One way to interpret Lagrange multipliers is using the “sensitivity analysis” method [2]. In brief terms, a large value of either $\alpha$ or $\beta$ means that the training example has a bigger “influence” on the final solution (set of weights). More precisely, a large value of $\alpha$ means that the training example will have a bigger impact on the optimal objective function if we allow weights to be larger than $Q$. We will elaborate on this in the revised version of the paper. On the other hand, $s$ is a binary indicator for the sensitive attribute. For example, if we have two attributes for perceived gender, then $s=(1, 0)$ for perceived women and $s=(0, 1)$ for perceived men.

Again, we are grateful for your constructive feedback, which has highlighted areas for further exploration and improvement in our research.

If there are any other questions or concerns, we would appreciate it if you let us know so we can respond to them during the discussion and improve the manuscript.

Thank you

[1] Fraser, Kathleen C., Svetlana Kiritchenko, and Isar Nejadgholi. "A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?." arXiv preprint arXiv:2302.07159 (2023).

[2] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge University Press, 2004.

Dear reviewer,

We would like to kindly inquire if our responses have satisfactorily addressed your concerns and questions. We are open to further clarification if needed. If we have addressed your concerns, we would appreciate it if you consider revising your review/score accordingly. Otherwise, please let us know so we can respond to your inquiries during the discussion period.

Sincerely

Thank you for your thoughtful evaluation of our paper and for your insightful questions and suggestions. We appreciate your recognition of the strengths of the paper, including the motivation, clarity of presentation, and the comprehensive experimental results we provided. Your feedback is invaluable in enhancing our work.

Addressing your question regarding the mitigation of representation bias (RB) versus association bias (AB), we suspect that the reason AB is easier to mitigate is because of its closed-vocabulary nature. RB, on the other hand, deals with an open-vocabulary problem so it is harder to mitigate. Nevertheless, we still observe a positive message. As shown in Figure 2: (1) balancing the sensitive attributes generally helps, and (2) using proxies make balancing more effective. However, these do not eliminate RB entirely as acknowledged in the paper. We will elaborate on this more in the revised version.

Your suggestion of developing a suitable augmentation strategy that might help RB is well-taken. Another open research question is to identify (from the statistics of the data) the minimal set of proxies that can have the biggest impact on RB. We plan to explore these directions in future work.

About viewing AB and RB as distribution shifts, this is an interesting angle to view both problems. In your notation, $y$ would be the sensitive attribute and $x$ would be the label (e.g. occupation or proxy) if we understand it correctly. So, a shift in $p(y)$ would correspond to RB (balancing → uniform distribution) white a shift in $p(y|x)$ corresponds to AB (balancing → $p(y|x)=p(y)$, i.e. independence). It’s unclear, however, if a shift in $p(x)$ (i.e. changing the distribution of labels) would correspond to a notion of fairness/bias. We would appreciate it if you could elaborate on this.

Again, we are grateful for your constructive feedback, which has highlighted areas for further exploration and improvement in our research.

If there are any other questions or concerns, we would appreciate it if you let us know so we can respond to them during the discussion and improve the manuscript.

Thank you

Dear reviewer,

We would like to kindly inquire if our responses have satisfactorily addressed your concerns and questions. We are open to further clarification if needed. If we have addressed your concerns, we would appreciate it if you consider revising your review/score accordingly. Otherwise, please let us know so we can respond to your inquiries during the discussion period.

Sincerely