A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models

We thank the reviewer for the positive assessment and great support of our work. We are particularly happy that the reviewer finds our experiments “extensive and technically sound” and that our results are “significant and interesting” to the reviewer. We also appreciate the suggestions provided in the review to improve the clarity and reading flow, as well as the interesting reference to work on synthetic data and bias amplification.

We included these in our updated version, as detailed below. Additionally, we highlight again our main contributions, and we are happy to further discuss any remaining unclear aspects of our paper.

The following are our individual responses.

---

The paper, while being strong, sound and interesting, is not particularly innovative and does not present unique or creative ideas.

We greatly appreciate that the reviewer finds our paper significant, and we respect their judgment regarding the novelty of our ideas. We would still like to highlight our innovations:

Our study features an improved experimental methodology, and we expect that future work will adopt this improved analysis pipeline. Furthermore, we investigate gender bias beyond occupations and link it to bias that exists in real-world contexts.

In addition to the empirical contribution this provides, researchers can use our prompts and categories for their analyses. We also link results from human studies in social science with statistical results on gender distribution in generated images. We think that this combination of research areas is novel and will facilitate further interdisciplinary studies.

Furthermore, we illustrate how biases are amplified beyond real-world contexts, which may be linked to the models (as opposed to training data). For this, we use gender-concept cooccurrence frequencies from LAION-400m. This is another valuable contribution of our work that will inspire future research, because it allows much more detailed analyses regarding the relation between bias in pretraining data and model bias. It also reduces the reliance of bias analysis on demographic workforce statistics. This contribution enables the establishment of new baselines for bias amplification, and it enables research on better understanding how dataset bias translates to model bias.

---

Lines 224-225 are a bit confusing, since it is not clear why the parent-son prompts are not gender-neutralized (e.g., parent-child)

Thank you for pointing this out. We decided to keep the activities in their original formulation as provided by Wilson and Mihalcea [A], save for syntactical adaptation to our prompt format. However, a number of prompts indeed contain gendered expressions, such as “son” or “mom”. Our rationale was that this could allow for an analysis of gender bias in interpersonal interactions. For example, would the prompt “attend a meeting at one's son's school” mainly generate male-gendered images, while the prompt “go shopping with one's son” (both are actual examples from our prompts) would generate images that show women? However, we didn’t include experiments on this in the current paper and leave this for future work. We hope that our released data, including images, will inspire future work on this.

[A] Wilson and Mihalcea: Measuring semantic relations between human activities. In IJCNLP 2017

---

For clarity reasons, it would be great if the authors could briefly explain the difference between "bias analysis" and "open bias detection" at the beginning of Section 2.

We agree that this point should be clarified. We have added the following explanation, which will be included in our updated version:

“Here, bias analysis means understanding and quantifying known biases (e.g., gender bias) of T2I models, while open bias detection means discovering biases that the T2I model exhibits for a given set of prompts, without any prior information what kind of bias (e.g. gender/race/age bias or other attribute correlations) we are looking for.”

---

In the part in which they mention the utilization of generated images for training other models, they might consider adding some reflections on the impact, considering the results shared in the following paper: Chen, T., Hirota, Y., Otani, M., Garcia, N., & Nakashima, Y. (2024). Would Deep Generative Models Amplify Bias in Future Models?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10833-10843).

Thank you for bringing this very interesting and relevant paper to our attention! While the paper does not find any conclusive trends regarding the relation of synthetic data and gender bias amplification, it shows that synthetic data can have effects on various bias axes, such as age or skin color. We agree with the conclusions mentioned in the paper that these findings suggest dataset audits and possibly filtering are necessary to avoid introducing bias through synthetic data, which in turn justifies the need for large-scale analyses such as ours to better understand possible bias in synthetic data.

Therefore, we have added the following sentence to the suggested section:

“Synthetic images are not only used in everyday applications such as advertisements [61] and presentation slides [76] but are also increasingly used as training data for other foundation models [35,59,96,97]. Here, Chen et al. (2024) observed age and skin tone bias amplification at increased levels of synthetic images in the pretraining data and therefore recommend filtering web data to mitigate bias.”

---

In general, if the authors made an effort to switch from implicit to explicit references, this would definitely improve the flow of the manuscript.

We also think that explicit references are preferable, and we resorted to numbered references mainly because of space constraints. In our updated version, which we unfortunately cannot share, we have made references explicit as much as possible within the space constraints, in particular in all cases where the reference is the grammatical subject of a sentence. We hope that these changes will increase the reading flow of the paper, and we thank the reviewer once again for the suggestion.

We thank the reviewer for the helpful feedback, and we are happy that the reviewer mentions the scale and setup of our study as a strength of our work. We appreciate that the reviewer finds our comprehensive design offers significant advantages for investigating bias in TTI models, and that our findings are credible.

In our rebuttal we address the mentioned concerns about the roles of classifier-free guidance and prompt prefixes. For this, we ablate the effect of classifier-free guidance on female ratios, finding that not using CGF results in poor image quality, and mixed impact on bias in models that are more robust to different guidance scales. We also highlight our experiments regarding different prompt prefixes, where we find that variations do not systematically affect the measured bias.

We hope these propose the requested clarifications. Let us know if there are any remaining uncertainties.

In the following, we address each concern mentioned in the review individually.

---

The potential role of CFG in reinforcing or amplifying gender bias warrants further investigation, ideally through controlled ablation studies.

We thank the reviewer for this interesting suggestion. The role of CFG in the quality-diversity trade-off makes it a potential candidate for influencing bias. To investigate this, we conducted the requested ablation study. We sampled 50 “activity” prompts, stratified by their average female ratio, and re-generated images using two settings: (1) CFG turned off, and (2) a low CFG scale of 2.0.

As the reviewer anticipated, turning off CFG or lowering the guidance scale significantly degrades image quality for SD-3.5-Medium and SD-3-Medium, rendering most images unusable for analysis. We therefore proceed with the three models that produced coherent images (Flux, Flux-Schnell, SD-3.5-Large). The table below shows the mean absolute deviation of the female ratio $R_f$ from the $R_f$ values in our original study, where we use recommended CFG values.

From this, we derive the following key findings:

1. For models where CFG is a factor (Flux, SD-3.5-Large), changing the guidance scale has small or inconsistent effects on gender bias.
2. Lowering or disabling CFG severely impacts the basic image quality of several models (SD-3.5-Medium, SD-3-Medium).

Therefore, we conclude that deviating from the recommended CFG settings introduces a strong confounding variable. Any observed changes in bias could be artifacts of the model's failure to generate coherent subjects. Therefore, to ensure a fair and meaningful evaluation of bias on high-quality, in-distribution generations, we think that using the recommended guidance scale values is the most methodologically sound approach. We will add this ablation study to the supplementary material.

Furthermore, we check if the different treatment of CFG in Flux-Schnell has an impact on image diversity, and if diversity has an impact on bias. For this, we use DINOv2 image embeddings and calculate the average pairwise cosine similarity for all images generated for a given prompt group. We use this as an inverse proxy for diversity, where lower scores indicate higher diversity. The results are summarized below:

The diversity scores of FLUX-Schnell are in line with the other models (i.e., no outlier), so we conclude that the different treatment of CFG has no impact on image diversity in our case. Nonetheless, models generate images with varying levels of diversity, and their diversity ranking is consistent across prompt groups. For example, Flux models generate the most diverse images, while SD-3-Medium generates the least.

Combining these findings with our results on bias strength presented to Reviewer PQHE, suggests that higher image diversity may not directly translate to less bias. Although models can be ranked clearly by their diversity and skew (see the response to Reviewer PQHE), which are naturally related, this ranking does not correspond to a clear ranking by bias strength. This suggests that improving general capabilities like image diversity may not be a solution for mitigating bias. We believe this is an important finding, and we will add this analysis and discussion to the paper.

---

It is recommended that the authors explicitly investigate potential gender bias in prompt prefixes through controlled experiments.

We thank the reviewer for this critical question about disentangling the sources of bias. Our experimental design was constructed to isolate category-specific bias from other potential confounders. We can break down the sources as follows:

1. Inherent Model Bias: We agree that models are inherently biased toward generating male-gendered images, as noted in line 194, and also confirmed by other studies [A, B]. This likely explains the bias amplification trends: The ratio of female-gendered images for female-dominated activities is reduced, because models pull the gender distribution towards the male. Conversely, this may also cause the tendency to emphasize male skew.
2. Prompt Prefix Bias: To control for bias introduced by prompt wording, we intentionally used 5 distinct, neutral prefixes (“a person”, “someone”, etc.). Most importantly, we performed an ablation study on these prefixes (Appendix D.3, Fig. 7), which confirms that no single prefix introduces a significant gender skew. By averaging our results across these five validated prefixes, we neutralize the prompt prefix as a confounding variable.

We will clarify this in the main paper to assure that our methodology isolates and measures category-specific biases fairly.

[A] Ghosh and Caliskan: 'Person'== Light-skinned, Western Man, and Sexualization of Women of Color: Stereotypes in Stable Diffusion. In EMNLP Findings 2023
[B] Wu et al.: Stable diffusion exposed: Gender bias from prompt to image. In AIES 2024

---

Presenting model-specific results would facilitate a more precise understanding of the bias patterns across different TTI models.

This is a great suggestion. We will add figures showing model-specific results to the supplementary material for our updated version.

We thank the reviewer for the positive evaluation of our submission and for the great suggestions to improve it. We appreciate that the reviewer finds that we provide “valuable insight” and enable improved trustworthiness of our findings. In our rebuttal, we clarify our contributions, and we describe additional results regarding model comparisons, which we will include in our updated version. Furthermore, we clarify implications for debiasing resulting from our work, which we will also add to in our updated version. Finally, we justify using LAION-400M as a proxy for web-scale pretraining data.

We hope these clarifications address the concerns mentioned in the review. Kindly let us know any further questions or uncertainties. We are happy to discuss additional changes to our submission or any remaining doubts. In the following, we answer to all comments and concerns individually.

---

What novel findings can we extract from this analysis beyond reaffirming that bias exists?

The novelty of our work lies in three key areas that establish a new, more rigorous standard for bias analysis in T2I models. These contributions are:

1. Improved experimental methodology. Instead of classifying images holistically as male or female, we detect all people in images, label their perceived gender, and remove images with ambiguous gender or no people. We also exclude entire prompts that generate too few images to yield statistically significant conclusions. These are innovations of the experimental setup as commonly used for bias research of T2I models, and thus enable more valid analysis. We expect that future work will adopt our improved analysis pipeline.
2. Broader coverage of concepts. We systematically investigate gender bias at scale beyond the well-studied domain of occupations. By curating 3,217 prompts, we provide the resources to analyze bias in more subtle, everyday scenarios (e.g., household chores). This enables a more holistic understanding of the stereotypes T2I models perpetuate. Our work is also novel in its interdisciplinary approach, which links findings from social science with statistical results on gender distribution in generated images. We expect that this will inspire further interdisciplinary studies.
3. Using web-scale data as bias baselines. We establish gender-concept co-occurrence frequencies in LAION-400M as a baseline for gender bias. This is a significant contribution because it reduces reliance on demographic statistics. This method allows for a more direct analysis of how biases are amplified beyond real-world contexts, which may be linked to the models (as opposed to training data).

We will revise the paper to make our contributions clearer.

---

it would improve the completeness of the work to include recommendations or implications for future bias mitigation strategies based on the findings

We thank the reviewer for this important suggestion. While we have a preliminary discussion on debiasing in Appendix I, we agree that the implications of our findings for bias mitigation can be made more explicit and actionable.

Based on our analysis, we will add the following key recommendations to the paper:

1. Targeted Data Curation: Our fine-grained analysis identifies specific concepts and contexts (e.g., “laundry”, occupations involving physical work) where gender bias is most severe. This can inform mitigation efforts like targeted data collection or strategic rebalancing of existing training sets.
2. Hybrid Mitigation Strategies (Data + Model): Our finding that models can amplify bias present in web-scale data (LAION-400m) is a crucial insight. It implies that data-centric approaches alone may be insufficient. Future work should investigate hybrid strategies that address bias in both the training data and the training/inference process of the model.

We will add a dedicated subsection to the discussion (Sec. 5) in the main paper, and expand on it in Appendix I.

---

Are some models more biased than others? What factors (e.g., model size, training objective) contribute to the differences, if any?

This is an excellent question. To provide a detailed comparison of the models, we made a new analysis where we examine model-wise differences along two axes: bias direction (which gender a model favors relative to the average) and bias intensity (how skewed, i.e. close to 0 or 1, the gender ratios are).

1. Bias Direction: To see if any model consistently leans more male or female, we calculated the deviation of each model's female ratio $R_f$ from the average $R_f$ across all models for each prompt. As shown in the table below, the results are mixed.

In a few cases, models significantly deviate from the model mean, for example SD-3.5-Large generates more women than the average for object prompts, and SD-3.5-Medium for context prompts. Conversely, Flux-Schnell generates fewer women for context prompts, and SD-3-Medium for activity prompts. However, no model consistently shows a positive or negative deviation across all categories.

2. Bias Intensity (Skew): Next, we measure the intensity of bias by calculating the entropy of the gender distribution for each prompt group. Lower entropy indicates a more skewed distribution (i.e., generations are heavily skewed towards one gender), signifying more intense bias.

This analysis shows a clear and consistent trend. SD-3-Medium consistently produces the lowest entropy (most skewed) outputs, while SD-3.5-Large is the most balanced (highest entropy). The Flux models fall in between. This allows us to rank the models by the intensity of their gender bias: SD-3.5-Large (most balanced) > Flux / Flux-Schnell > SD-3.5-Medium > SD-3-Medium (most skewed).

This ranking suggests that larger, more recent models may produce more balanced gender representations. However, this conclusion should be taken with caution, since information on training data and objectives is not public. We thank the reviewer for suggesting this interesting analysis, which we will add to the main paper.

---

Given that many of the evaluated models are trained on larger, more diverse datasets than LAION-400M, how reliable are the comparisons made using LAION-400M?

Thank you for raising this critical point regarding the limitations of proprietary models. While we do not have access to the exact training data, our use of LAION-400M as a proxy is justified and, we argue, makes our findings on bias amplification more robust.

Our reasoning is twofold:

1. As noted by Udandarao et al. [A], concept frequencies and scaling trends are consistent across different web-scale datasets. This suggests LAION-400M is a reasonable, publicly available proxy for the type of data these models are trained on.
2. LAION-400M is likely a conservative baseline. Assuming the reviewer's premise is true, that the proprietary training sets are larger and more balanced than LAION-400M, it would mean our baseline is more biased than the models' actual training data. In this scenario, observing that models still amplify bias relative to our LAION-400M baseline makes our conclusion even stronger. The models are not only failing to mitigate the bias in comparison to our proxy dataset, they are amplifying it.

Therefore, we argue our conclusions are reliable because LAION-400M serves as a reasonable and, importantly, conservative baseline for measuring bias amplification.

[A] Udandarao et al.: No “zero-shot" without exponential data: Pretraining concept frequency determines multimodal model performance. In NeurIPS 2024

---

A discussion or acknowledgment of the additional axes of bias would have made the work more comprehensive and aligned with current trends in fairness research.

We agree that a comprehensive analysis has to consider intersectional biases beyond binary gender. We conducted experiments on race and age, which are detailed in Appendix I (line 1319).

There, our analysis finds a significant form of representational bias, i.e. models demonstrate a strong tendency to generate people who appear to be White and young-to-middle-aged when the prompt does not specify any attributes related to gender or age. This finding is also an important result about the apparent lack of (demographic) diversity in model outputs. This representational homogeneity, however, makes a deeper statistical analysis of intersectional bias (like the one we performed for gender) currently infeasible. There is simply not enough variation in the generated racial and age attributes across most prompts to draw statistically meaningful conclusions. The strong "default" of the models to a single demographic is a major limitation that we will discuss more explicitly in the main paper, as it is a critical form of bias and therefore should a priority for future work.

We thank the reviewer for the clear, constructive, and helpful feedback. We appreciate that the reviewer recognizes the broadness of our analysis as a strength. We also agree that our findings regarding bias amplification with respect to LAION-400m are an important contribution of our study.

In our rebuttal, we address the concerns regarding novelty, and we provide justification for the need of large datasets to study gender bias. We also describe new results regarding the accuracy of our VQA model on synthetic data. Furthermore, we confirm that we will make our data, including all images and gender labels, available after publication.

Based on these clarifications, we hope the reviewer will support us in making our research known to the community, and we are looking forward to a constructive discussion.

Below, we reply to the reviewer's concerns individually.

---

Is there any novel technical contribution I missed, that could be highlighted? Will all images be released? How will this work enable future works in bias analysis?

While our work does not introduce a new technical method, its novelty lies in three key areas that establish a new, more rigorous standard for bias analysis in T2I models. These contributions are:

1. Improved experimental methodology. Instead of classifying images holistically as male or female, we detect all people in images, label their perceived gender, and remove images with ambiguous gender or no people. We also exclude entire prompts that generate too few images to yield statistically significant conclusions. These are innovations of the experimental setup as commonly used for bias research of T2I models, and thus enable more valid analysis. We expect that future work will adopt our improved analysis pipeline.
2. Broader coverage of concepts. We systematically investigate gender bias at scale beyond the well-studied domain of occupations. By curating 3,217 prompts, we provide the resources to analyze bias in more subtle, everyday scenarios (e.g., household chores). This enables a more holistic understanding of the stereotypes T2I models perpetuate. Our work is also novel in its interdisciplinary approach, which links findings from social science with statistical results on gender distribution in generated images.
3. Using web-scale data as bias baselines. We establish gender-concept co-occurrence frequencies in LAION-400M as a baseline for gender bias. This is a significant contribution because it reduces reliance on demographic statistics. This method allows for a more direct analysis of how biases are amplified beyond real-world contexts, which may be linked to the models (as opposed to training data).

Finally, to directly answer the reviewer's questions:

• Data Release: We confirm that we will release all data on huggingface, i.e. the 2.3 million filtered images, the full set of 3.2 million generated images, the 3,217 prompts, all detected bounding boxes, and all perceived gender labels. We believe this dataset will be a great resource for the community.
• Enabling Future Work: Our contributions directly enable future work by providing (a) a more rigorous and reliable audit protocol, (b) a comprehensive set of prompts and a massive, labeled dataset for studying bias in under-explored contexts, and (c) a new baseline methodology for analyzing bias amplification.

We will revise the paper to make our contributions clearer.

---

How much do analysis results change when image set size changes?

Our large number of images per prompt is required for statistical precision. As the reviewer suggests, we conduct a subsampling experiment to quantify the stability of $R_f$ with varying number of images. For each prompt, we create 1,000 bootstrap samples for sample sizes ($n \in [20, 25, …, 200]$) and calculate the mean absolute deviation of the sample $R_f$​ from the $R_f$ calculated on our full data. The resulting mean absolute deviation values in the following table demonstrate that smaller sample sizes lead to considerable measurement error:

Only with at least 100 images per prompt does the average deviation reliably drop below 3 percentage points. For smaller samples, the noise could easily obscure the effects we aim to measure. For instance, a 6 percentage point average deviation for a sample size of 20 is too high for precise analysis across thousands of prompts.

These findings strongly support the scale of our study. While smaller samples might be sufficient to merely show that some bias exists, they are inadequate for the goal of our work, which is to precisely quantify and compare gender biases across different models and a broad range of concepts. We will add this analysis and justification to the supplementary material.

---

can you evaluate how well the VQA model does in detecting genders in synthetic images, even in cases where there may be artifacts?

We thank the reviewer for this suggestion. To address this, we conducted two new experiments as requested:

Quantitative Validation on Synthetic Data: We evaluate InternVL on the SocialCounterfactuals dataset [A], which contains over 170,000 synthetic images from an older Stable Diffusion model. On this dataset, InternVL achieves 99.7% accuracy and a Cohen's kappa of 0.994 against the ground truth labels, which means near-perfect performance on synthetic images with known attributes.

Human Agreement Study: We also conducted a study on our own generated images. We randomly sampled 120 bounding boxes (40 labeled male, 40 female, 40 unclear by InternVL) and had them labeled by two human annotators unaffiliated with this submission. We then compared the MLLM labels to the human labels.

The model-to-human agreement (Annotator 1: $\kappa$=0.78 and Annotator 2: $\kappa$=0.71) is substantial and nearly identical to the human-to-human inter-annotator agreement ($\kappa$=0.775). Here, it is critical to note that InternVL performs on par with a human annotator for this task, as its agreement with a human is bounded by the agreement between two humans. This validates our use of InternVL for labeling perceived gender in our generated images. We will add a detailed description of these experiments to the supplementary material.

[A] Howard et al.: Socialcounterfactuals: Probing and mitigating intersectional social biases in vision-language models with counterfactual examples. In CVPR 2024

---

it would be valuable to observe how different models represent certain clusters & what kinds of prompts show the most variation

We agree with the reviewer that qualitative examples would strengthen the paper. We will add new figures providing a detailed qualitative analysis of the "laundry" cluster, as suggested. We will show representative images for each of the five models, including examples of female-labeled, male-labeled, unclear, and multi-person generations to provide a direct visual comparison. This shows the gender difference for SD-3.5-Large, that people are detected well, and that gender labels appear accurate.

To inspect prompt variance, we calculate the standard deviation across models for all prompts and show the 10 prompts with highest standard deviation in the following table:

Interestingly, 8 of the 10 prompts with highest variance are related to sports, which shows that across models, associations between gender and sports are unstable. Prompts in the laundry cluster have an average std. dev. of 0.22, which ranks them in the 56th percentile (from lowest to highest).

---

I agree that LAION-400m may be representative, but is there any citation to justify this?

Indeed, there is evidence that LAION-400m is a representative sample of web-scale data. First, it was sourced from CommonCrawl, which mirrors web content at large scale. Second, a study by Udandarao et al. [B] showed similar concept frequency scaling trends in LAION-400m and smaller datasets, so these trends will likely extrapolate to larger datasets. Therefore, we consider LAION-400m to be a good proxy for multimodal datasets used to train foundation models, even if they are in reality larger. We will include this reference in the paper.

[B] Udandarao et al.: No “zero-shot" without exponential data: Pretraining concept frequency determines multimodal model performance. In NeurIPS 2024

---

Lines 5-7 (abstract): the wording used here seems to suggest 200 images per prompt per model

Thank you for pointing out that this statement is imprecise. In our updated version, this sentence reads: “We create a dataset of 3,217 gender-neutral prompts and generate 200 images per prompt (40 images over 5 prompt variations) from five leading T2I models”. However, we would like to clarify that we generate 200 images per prompt per model: 5 models x 3217 prompts x 5 prompt variations x 40 images per variation = 5 models x 3217 prompts x 200 images.

---

some prior works do consider more images per prompt per model, e.g. TIBET

We invite the reviewer to check if our overview of related work in Appendix J (especially in Tab. 12) already addresses this concern. There, we list the total number of images and the number of gender-neutral prompts used in the respective work. We will add TIBET to this table in our updated version.