Causal analysis of social bias in CLIP

We thank reviewer mYoV for their insightful remarks.

W1: The reviewer's observation that our paper employs ``attributes from social psychology instead of a self-defined list of words" may not fully acknowledge the depth and scientific rigor associated with using psychometrically validated scales; We refer to Furr (2021) for an introduction to the field of Psychometrics. Moreover, the widespread adoption of the SCM and ABC models underscore their practical relevance in measuring stereotype perception. Thus, our decision to utilize validated scales in our study is not a mere preference for a different list of words but a deliberate choice to enhance the contribution of our research to the field.

W2: Yes, it is important to check for biases, artifacts, and confounds. If the referee has specific concerns, we will be the first ones to want to hear about them. The reviewer asks for "evidence of absence of confounders". Unfortunately, that is typically impossible to come by. However, the evidence for the presence of confounders is precious. We selected the Liang (2023) dataset precisely because it was carefully controlled. They verified that attribute manipulations were equally effective across different demographics using the ratings of five annotators per image (Fig. 3, Liang 2023). They checked for visible image artifacts (Sec 4.1) and excluded all images containing artifacts from their dataset. We have added to the manuscript a sentence discussing this (please also see response to referee 3zN2). Are we sure that all possible confounds and biases have been eliminated? Of course not. That would be foolish. In the history of science, there are many examples of important experiments where confounds and biases were hammered out year after year by generations of scientists. The crucial point is adopting the scientific method, which is based on experimentation, and experiments may be refined over time. We argue that measurements of bias in AI models now can, and thus must, move beyond correlations. Generative methods offer a practical way forward toward controlled experiments and causal conclusions. Thanks to progress in generative AI and constructive scrutiny from an engaged community of scholars, we expect our measurements to become more accurate with time.

W3: We have thoroughly revised our paper in light of the reviewer's critique. We concur on the need to clarify the definition of cosine similarity, therefore we introduced equation (1) in section 3.1. Although our paper also utilizes metrics like WEAT, mean cosine similarities, and markedness, we have chosen to keep their detailed descriptions in the appendix. This decision aligns with common academic practices of detailing mathematical methodologies in supplementary materials, rather than in the main text, especially since these metrics are well-established and not novel contributions of our work. Section 4.1 was moved to the appendix, now appendix C, as we think it is not part of the main results of our paper but rather a supporting analysis that helps interpreting the results in Fig. 2.

Q1: Despite variations in national laws, global human rights standards set by the United Nations—like UDHR, ICCPR, ICESCR, and CEDAW—mandate non-discrimination based on race, age, and gender. While some countries may not legally protect these attributes, the majority do. Hence, we refer to these variables as legally protected, aligning with the scholarly consensus in various fields.

Q2: Our study compares FairFace to CausalFace; The latter however only includes three races: Asian, White, and Black. Some researchers might be interested in analyzing social perceptions across all FairFace races without comparing them to CausalFace. We will open-source our code, which allows those researchers to adapt our experiments to all racial categories in FairFace.

Just wanted to chime in here that I think reviewer mYoV raises a good point about the possibility of confounders in this generated dataset making this signal less clean. I think this is actually fairly essential to this paper and something I hadn't thought of in my review. The authors response that this is something considered in the original dataset is good and I appreciate that it was added to the discussion section - in my opinion, given how important this is, the evidence that these confounders are minimal should be moved up to 3.2 where the CausalFace dataset is introduced.

Extra thought: the authors are quite dismissive of this point in their rebuttal and the response around the "history of science" comes off as somewhat condescending imo. In fact, this is an extremely reasonable and important critique of the paper on which the legitimacy of its contribution rests, and it was something which was not addressed at all in the original draft. In the "history of science", I agree that experiments are not guaranteed to come without confounds, but this mostly applies to observed phenomena (e.g. natural experiments): here, we are dealing with entirely controlled settings and should be aiming for a higher bar, particularly given the (strong) claims of the paper.

The reviewer thanks the authors for their response. The reviewer acknowledges the time and effort invested in elucidating the raised concerns. While the reviewer values the comprehensive insights provided in the response, the reviewer has concerns about the choice of tone in response W2. The reviewer understands that discussions in the scientific community merit a respectful and collaborative tone. The reviewer's intentions were only to raise constructive criticism to improve the scientific contribution of the reviewed work.

In the following, the reviewer addresses the specific points raised in the response:

W1: The initial review did not intend to overlook the significance and scientific rigor linked with the utilization of psychometrically validated scales. On the contrary, the original review explicitly recognized this aspect as a primary contribution of the paper. The central query posed in the initial review pertains to whether this particular contribution, in isolation, constitutes a sufficient basis to warrant submission to ICLR. Regrettably, the authors have not yet addressed this specific concern in their response.

W2: The reviewer remains unconvinced regarding the assumption that synthetic datasets do not introduce artifacts or confounds. This concern is fundamental, given the paper's assertion of studying causality rather than correlation. It is incumbent upon the authors to provide a robust justification to assure readers that the study indeed warrants claims of causality. Notably, ample evidence suggests the presence of gender artifacts in visual datasets across various image levels [1]. Consequently, it remains unclear whether the utilization of a synthetic dataset eliminates these artifacts or if they are learned and transferred within the generative model.

W3: Thanks for reviewing the paper to make it self-contained.

Q1 and Q2: Thank you for answering the questions.

Reference: [1] Meister et al. Gender Artifacts in Visual Datasets. ICCV 2023.

We thank reviewer qwQ3 for their positive assessment and insightful remarks.

W1: You are inviting us to make our title more specific and we agree: "A causal analysis of social bias towards faces in CLIP."

W2: We agree with the reviewer that the figure caption (now Figure C.3) was unclear. The interpretation of the figure was moved to the appendix and a mathematical definition was added: "Specifically, let $a^+_1$ and $a^+_2$ be two text embeddings of attributes of the positive valence agency dimension, and $a^-_1$ and $a^-_2$ be text embeddings of attributes of the negative valence agency dimension. The within-valence similarity is given by calculating the cosine similarities $\cos{(a^+_i, a^+_j)}$ for all positive attribute pairs $(i,j)$. The cross-valence similarity is determined by calculating $\cos{(a^+_i, a^-_j)}$ for all mixed attribute pairs $(i,j)$."

W3a: We agree that mean cosine similarities were not defined sufficiently. For clarification, we moved Eq. 1 from the appendix to the main paper (see. Sec. 3.1.). In addition, we added further explanation in the results section when discussing Table 1, which reads as follows: "Mean cosine similarities are calculated as the mean of cosine similarities between an image category and all attribute categories of SCM and the positive attribute categories of ABC (see Equation (1))".

W3b: We added appendix A.3 to further explain and mathematically define markedness.

W4: We show the positive correlation of positive and negative valence attributes in Figure 2. The high correlation is however only visually assessed. We agree with the reviewer that we should calculate a correlation coefficient. We did so for each dimension (agency, belief, communion) and for each intersecting group separately. We plot and interpret a heatmap of correlations in Figure C.4.

W5: Indeed, it was not obvious what the term "intensity" means in the context or study. Therefore, we added the following explanation when it was first introduced: "In this context, we use valence as the direction of change in cosine similarity (increasing or decreasing) and intensity as the magnitude or absolute difference in change."

SN2: To maintain consistency with Liang (2023), we kept the original scales for smiling (-2.5 to 4) and age (0.8 to 4.4), as shown in Figures D.2 and D.3. For smiling, the non-equidistant levels range from -2.5 to 4. We selected a threshold of 0.7 to distinguish between sufficiently different smiling faces, deliberately excluding the 0.5 increments. A similar approach was applied to age. To avoid confusing a future reader, we introduced the following footnote: "The effect of the thresholds can be better understood by looking at the scale of Figures D.2 and D.3"

SN3: To avoid redundancy between the title and the x-axis label in Figure 3a, the x-axes are unlabeled. To clarify, we have added to the figure caption: 'The three panels depict the variations in smiling, lighting, and pose on the x-axis.'

SN4: We agree that we should have been more precise with the use of language at this point. The patterns of all gender-race intersections look similar, with one exception: Male-blacks. Only for this group does smiling lead to an increase in positive social perception. The interpretation needs to be more nuanced. There is a subtle but existing difference between stating a "glass is half full" versus "a glass is half empty". Similarly, there is a subtle but existing difference between describing someone with less negative terms or more positive terms. Black males are the only group that is described predominantly with more positive terms as the smile increases. We changed the corresponding description in the paper to: "For "belief", only black males deviate from the pattern consistently observed for the remaining subgroups."

Q1: We appreciate the reviewer's insightful feedback on our discussion of variations in legally protected versus non-protected variables. We acknowledge that the original phrasing might have led to misunderstandings. We concur with the reviewer that gender, race, and age are socially significant factors, meriting the focused research they receive. Their importance indeed underscores our call for greater precision in measurement and a thorough examination of potential confounds in statistical analyses. To clarify our position, we have revised the statement at the end of section 4.4: "Our observations suggest that non-protected attributes are important and should be taken into consideration to obtain a clear measurement of the socially relevant age- and gender-induced variations."

We thank reviewer 3zN2 for their positive assessment and insightful remarks.

W1: We appreciate the reviewer's suggestion regarding the selection of face datasets for our analysis. In response, we have thoroughly evaluated each of the proposed datasets and would like to share our rationale behind their consideration: MIAP does not have face crops and, therefore, would introduce a significant amount of noise from contextual elements. These are additional confounding factors that could result in visible differences compared to FairFace. It is not the main goal of our paper to control for these types of confounds; instead, we focus on face-related confounds. MIAP would, therefore, not directly support our findings, and thus, we decided to focus on face-cropped datasets. CelebA meets our criteria but we do not consider it ideal for assessing biases in race, gender, or age groups for the following reason: It is likely that celebrities occur in the training data of CLIP. Therefore, we would assume that embeddings are overly specific to individuals rather than representative of broader demographic groups. Unlike FairFace, which includes only a limited selection of celebrities, CelebA is comprised entirely of celebrity individuals. We recognize UTKFaces as a viable alternative to FairFace for our analysis. We are in the process of extending our study to include this dataset and plan to integrate these results before the end of the rebuttal period.

W2a: Thank you for highlighting the need for clarity in the definition of markedness. To address this, we have added Appendix A.3 which succinctly reviews related works, illustrates the concept with examples, and provides a precise mathematical definition.

W2b: We do not generate synthetic images ourselves but the dataset was pre-generated and made available to us by Liang (2023). Smiling, lighting, and pose are dimensions generated through latent space traversals using a GAN to create face images. The specific manifestations of these dimensions in the images are illustrated in Appendix D. Upon reviewing our paper's section on image datasets, we acknowledge a potential source of confusion. Initially, we mentioned "we generate variations," which we have now revised to "we sample variations" to better reflect our methodology.

W3: The reviewer hypothesizes that cosine similarities between warmth/competence-related text prompts and images might fluctuate more significantly due to varying smile/pose levels than those between job type-related prompts and images with similarly varying smiling/pose levels. This is an interesting hypothesis and we do not investigate the relation of smiling/pose to occupation in generative AI and we are not aware of literature that does so. However, research investigating humans suggests that facial attributes do systematically vary with occupation: Facial appearance matters for leader selection (Stocker 2016). More specifically, the width-to-height ratio seems to distinguish CEOs (Lewis 2013) . Furthermore, men and women smiled more in response to a low-status job than to a high-status job (Vrugt 2002)--suggesting that smiling may be a more significant predictor of job type than gender. More generally, smiling has been found to differ across age and gender (Morse 2010, LaFrance 2003,Otta 1998, Doff 1999, DeSantis 2005). Therefore, we assume that occupation is also sensitive to smiling/pose. Consequently, there is no clear answer to the question of which outcome metric is more sensitive to varying levels of smiling/pose. This is an interesting question to ask therefore, we add the following paragraph to the discussion: "Our study uses social perception as the primary outcome metric, revealing that smiling and pose significantly affect this measure. Research in human contexts shows that smiling in particular, influences not only social but also job-related perceptions. Future research could focus on practical metrics such as occupation, investigating how varying degrees of smiling and pose impact these aspects in generative AI."

Thanks for the reply. Adding UTKFace would strengthen the findings of the paper. Also, thanks for clarifying the misunderstanding about generating the samples, and fixing it in the paper. I agree that adding the last paragraph to the discussions section would be useful to the reader.

By "neutral lighting," I meant a value of -1 in Figure 3(a), which you have now clarified.

We thank reviewer SWyU for their positive assessment and insightful remarks.

W1: To address the reviewer's concerns concerning clarity, we have made the following revisions: Firstly, we revised section (formerly) 3.2/ now 3.1 and additionally include mathematical notation. Additionally, we improved the caption of Figure B1, which now conveys our methodology more transparently. Secondly, recognizing the constraints of paper length, we have introduced Appendix A.4, which contains a detailed discussion of social psychology's view on stereotypes, and add additional references. This should provide an in-depth background for readers with a specific interest in the theoretical underpinnings of our study. We also made concise changes to section 2.2 to logically connected both models by stating: "A more recent framework, the ABC Model, Koch(2016), proposes beliefs as alternative central dimension, and subdivides the Warmth and Competence differently into two categories." Furthermore, we introduce both theories by stating that "social psychology has long been recognized for delineating the primary dimensions of beliefs about stereotypes." Thirdly, we justified the integration of social psychology frameworks within our study as follows in 2.2: "Given the reflection of human-like biases in Generative AI, our study employs two leading theoretical frameworks from social psychology—long recognized for quantifying cognitive biases—to measure stereotypes systematically". In all sections, we have striven for consistency in terminology, using "dimensions" and "attributes" to describe the components of the SCM and ABC models consistently. We hope these changes address your concerns, and we invite your further feedback to refine our manuscript.

W2: Thank you for pointing this out. Yes, we should have commented on this. We have included the following statement in the discussion: "Generative models, in particular GANs and Diffusion Models, can now generate face images that look realistic to human observers, and one can effectively control some of the facial attributes. There are three caveats. First, manipulating one attribute may induce unwanted changes in other attributes; Balakrishnan (2021) show how to address this issue systematically, and it should be checked by visual inspection, as was done for the dataset we use (Liang 2023). Second, it is possible that some physiognomies are generated more frequently than others. E.g., within the European group, it is possible that more Mediterranean-looking and fewer Scandinavian-looking physiognomies are generated. If this were a concern for a study, one would have to come up with a fine-grained definition of race and control this attribute directly, as our dataset does for the main racial groups. Third, it is possible that facial attribute manipulations are more or less effective for some or the other demographic groups. For the dataset we use, this was excluded by direct verification with human annotators (Liang 2023, Fig. 3.). Generative methods and techniques to validate their output are progressing fast. Thus, the experimental method we advocate, based on carefully controlled synthetic data, will become an increasingly attractive option for researchers."

Q1: One could expect that bias stems from training data (i.e., text-image pairs) rather than the model architecture. Therefore, we would expect that our findings apply to other models if they are trained on similar datasets. We present our method primarily as a tool for investigating biases in such models. Following the reviewer's question, we have updated the discussion section to emphasize the applicability of our approach to future research on a broader range of vision-language models.

Q2: Besides WEAT, we considered max-skewness and NDKL as alternative bias metrics but eventually omitted these results. Responding to the reviewer's interest, we are now reintegrating these findings into the paper and will present them before the rebuttal period concludes.

We wish to express our sincere thanks to all four reviewers. We believe that our paper has significantly improved, and we hope the reviewers share this perception.

We uploaded a final revision of our manuscript. Below, we summarize the main changes between the current and final revision and the initial version of the paper:

1. We introduced three additional bias metrics, defined them mathematically, added a new figure in Section B1, and interpreted the results, which align with the initially chosen bias metrics.

2. We evaluated a second observational dataset, namely UTKFace, leading to changes in Table 1, Figures 1 and 2. The interpretation of this dataset is in line with the interpretation of FairFace.

3. We have substantially expanded the discussion section of the paper, which now includes a discussion of the evaluation procedure and the causal interpretation of the dataset. It also addresses several points raised by the reviewers that are not covered in the main paper and should be pursued in future research.

4. The interpretation of Figures C4 and C5 were moved to the appendix. Equation (1) was shifted from the appendix to the main paper.

5. The entire paper was revisited with striving for consistency in the terms describing social perception dimensions and their comprising items. The word "attribute" is now exclusively used for legally protected (like race and gender) or non-protected attributes.

6. We have added a section on the correlation between positive and negative valence dimensions to the Supplementary Materials (Appendix C), including a newly introduced Figure C5.

7. We have included a detailed definition of Markedness in Section A3 and a comprehensive review of social psychology literature in Section A4.