AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generation

We thank Reviewer kTpz for acknowledging the clarity of our writing, the justification of our adaptive mapping network, and the experimental results. The concerns raised by the reviewer are address below:

Calculating anchor loss requires predefined information on non-dominant bias attributes for each occupation

We would like to clarify that our anchor loss does not require pre-defined information about non-dominant bias attributes. The anchor prompt is derived from the ground-truth attribute class of the training image, which is readily accessible. Our method operates without requiring prior knowledge of the original bias distribution of the target concepts, ensuring its applicability without human pre-determination.

Inherent bias within the diffusion model

We appreciate the reviewer’s observation. As with most previous bias mitigation approaches, our work primarily focuses on addressing biases introduced to the diffusion model by text embeddings, as these are more identifiable and actionable. The inherent biases within the diffusion model itself are challenging to isolate and remain an open research question.

Regarding potential attacks such as concept erasing, we acknowledge that this could impact the effectiveness of our method. However, robustness against adversarial attacks falls outside the scope of our current research, which aims to mitigate bias in standard, unaltered setups. This is an important direction for future exploration.

About FID metric

We thank the reviewer for suggesting alternative metrics for evaluating image quality. Following the suggestion, we evaluated the image quality of SD1.5, Fair Mapping (FM), and our method using the ImageReward [1] metric. We computed the average reward score across 2,400 images from SD1.5 (24 test occupations, each with 100 images) and 7,200 images for both FM and our method (24 test occupations, each with 100 images for gender, race, and age bias mitigation). The results are as follows:

ImageReward comprehensively considers human preference, image fidelity, and text-image alignment when assessing image quality. These results demonstrate that our method achieves fairer results while maintaining acceptable image quality, especially when compared to FM, which exhibits significant degradation in image quality.

[1] ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation. NeurIPS, 2023.

We thank Reviewer JdSh for recognizing the novelty and importance of our method. We also appreciate your acknowledgment of our creative use of adaptive inclusive tokens and the method's ability to generalize to unseen concepts and handle multiple attributes. The concerns are addressed below:

Practicality and efficiency

We thank the reviewer for their concern about the practicality and efficiency of our method. Compared to previous approaches that require extensive fine-tuning of diffusion models (e.g., ~48 hours on 8 NVIDIA A100 GPUs for a single attribute [1]) or large-scale external network training [2], our method is significantly more efficient. It requires only one hour on a single GPU using a small set of easily obtainable SD-generated images. This demonstrates that our adaptive mapping network and inclusive token training are lightweight in the context of bias mitigation.

Furthermore, once trained, the adaptive inclusive token generalizes to unseen concepts across different domination groups, eliminating the need for retraining. This enables reusability and makes our method practical and efficient for real-world applications.

[1] Fine-tuning Text-to-Image Diffusion Models for Fairness. ICLR, 2024.
[2] ITI-GEN: Inclusive Text-to-Image Generation. ICCV, 2023.

Qualitative results

We thank the reviewer for recognizing our quantitative improvements over the baseline. However, we are unclear about the specific qualitative examples being referred to as “many prompts still generating images with a predominantly single gender.” In all qualitative examples addressing gender bias mitigation (Figures 3, 7, and 10), our method consistently demonstrates clear improvements in gender diversity compared to the baseline.

For figures targeting other biases, such as race or age, it is reasonable that gender bias may persist, as those experiments were not designed to address gender bias specifically. We did not claim that our method mitigates all types of bias using single-attribute inclusive token; rather, we focus on the target attribute for each inclusive token. We hope this clarifies the observed discrepancy.

Question about effectiveness

We thank the reviewer for recognizing the simplicity of our approach. As demonstrated by both quantitative and qualitative results, our method effectively mitigates the attribute bias of interest. Furthermore, Table 3 in the main paper highlights the flexibility of our method, showcasing how stacking different inclusive tokens can address multiple biases simultaneously. This demonstrates that our approach, while straightforward, is both effective and versatile.

Clarification on $D_{KL}$ metric

We appreciate the reviewer’s observation regarding the $D_{KL}$ values of the original SD1.5, SD2.1, and SDXL models. It is indeed noteworthy that SD1.5 achieves the most inclusive results, with $D_{KL}$ rankings for gender bias as follows: 0.3584 (SD1.5), 0.4166 (SD2.1), and 0.4919 (SDXL). However, we are unclear about which specific qualitative results the reviewer finds inconsistent with the $D_{KL}$ metric.

It is important to note that $D_{KL}$ solely measures the fairness of the model outputs and does not account for image quality. A possible explanation for the observed bias in more advanced models could be the larger and potentially more biased datasets used for training. Additionally, we would like to highlight that with our adaptive inclusive token, all three models demonstrate significant improvements in $D_{KL}$, further validating the effectiveness of our method.

A single token vs. multiple tokens

We had similar initial intuitions that multiple tokens, with higher complexity, might outperform a single token due to their potentially greater expressiveness. However, upon analyzing the results, we observed that multiple tokens often learned redundant concepts from the reference set, as our method does not explicitly instruct them on what to capture. Additionally, the reference training sets we use are relatively small, making multiple tokens prone to overfitting due to their higher expressiveness. In this case, a single token proves sufficient and even outperforms multiple tokens by avoiding both redundancy and overfitting.

We sincerely thank Reviewer JdSh for their thoughtful follow-up and for acknowledging that we addressed concerns regarding efficiency.

Regarding the evaluation accuracy concerns raised by Reviewer QTbU, we would like to emphasize that the CLIP zero-shot classifier is widely used in the bias mitigation literature [1,2,3,4,5] for classifying sensitive attributes thanks to its robust performance and broad applicability. While it is true that CLIP is trained on real-world datasets and may reflect certain biases, our experiments demonstrate that it provides sufficiently accurate attribute classifications for evaluating fairness, as shown in our responses to Reviewer QTbU. Furthermore, its widespread adoption ensures comparability with prior work in this area. While we acknowledge that no single metric is perfect, we believe our experimental results, including both quantitative and qualitative evaluations, provide a comprehensive view of the effectiveness of our method.
[1] ITI-GEN: Inclusive Text-to-Image Generation. ICCV, 2023.
[2] How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? EMNLP, 2022
[3] Fair Text-to-Image Diffusion via Fair Mapping. arXiv, 2023.
[4] Balancing Act: Distribution-Guided Debiasing in Diffusion Models. CVPR, 2024.
[5] FairRAG: Fair Human Generation via Fair Retrieval Augmentation. CVPR, 2024.

On the question of bias in unconditional generation versus specifying clear conditions, we thank the reviewer for raising this important concern, as it allows us to further clarify and validate the motivation for our work. The key motivation of our method lies in addressing inherent biases in text-to-image generation when prompts are neutral and no explicit attribute is specified. This is a critical issue for fairness because many real-world use cases involve users providing high-level prompts without detailed specifications. Relying on users to explicitly define attributes (e.g., specifying "female doctor" or "male nurse") shifts the responsibility of fairness onto the user and does not address the underlying bias in the model. The definition of inclusiveness for a generative model focuses on how inclusive the outputs are when given no explicit attribute conditions. Generating female doctors when explicitly instructed with "female doctor" does not reflect the fairness of the model but rather its prompt-following accuracy. Inclusiveness is better measured by the model's ability to generate balanced results when the prompt is neutral.

Regarding the yoga instructor example in Figure 7, we would like to clarify that there is no gender specified in the prompt. The upper SDXL results are generated with the prompt "A photo of a yoga instructor," while the bottom SDXL With Ours results are generated with the prompt "A photo of a <gender-inclusive> yoga instructor." These results clearly demonstrate the effectiveness of our trained adaptive inclusive token, <gender-inclusive>, in bringing more male figures into the generation of yoga instructors, addressing the observed bias in the original model.

Furthermore, to address the reviewer's suggestion regarding attribute compositionality, we have already validated our method in Table 3. By leveraging different inclusive tokens trained for distinct sensitive attributes, our method successfully mitigates multiple biases simultaneously, demonstrating the flexibility and robustness of our approach. This capability further underscores the motivation of our work to create more inclusive generative models capable of addressing a variety of biases in diverse scenarios.

In summary, our work seeks to ensure that generative models produce fair and balanced outputs in scenarios where neutrality is expected. This fills an important gap in existing methods and remains distinct from explicitly conditioned generation tasks, where prompt-following accuracy is evaluated instead of fairness. We sincerely hope that our detailed clarifications and additional validations address the reviewer’s concerns and highlight the strengths of our method in tackling bias in generative models. By emphasizing the importance of inclusiveness in neutral prompts and demonstrating the effectiveness of our approach through attribute compositionality, we aim to showcase the significance and robustness of our work. We kindly ask the reviewer to consider these points when reassessing their score.

We thank Reviewer SjSS for reviewing our paper. The concerns raised are addressed below:

Introduction of factual error

We thank the reviewer for raising this concern. As discussed in Section 5 of the main paper, we acknowledge the potential introduction of factual inaccuracies when targeting even distributions, particularly for non-neutral concepts. Our method is specifically designed to address neutral concepts where all classes are factually plausible, as stated on Line 320.

In scenarios where fairness and factual accuracy conflict, our approach prioritizes fairness for neutral cases while avoiding explicit interventions in contexts requiring adherence to factual correctness. This ensures that our method remains applicable and effective within its intended scope.

Motivation and theoretical discussion

The motivation for the adaptive mapping network and anchor loss is discussed in detail in Section 3.1, “Is Textual Inversion Effective for De-biasing?” Specifically, we identify the limitations of textual inversion in addressing fairness, such as the imbalanced debiasing performance across different domination groups and the concept drifting issue. These issues motivate the need for a more flexible and concept-preserving mechanism to achieve balanced outputs.

The adaptive mapping network is designed to address these shortcomings by introducing a learnable transformation of text embeddings, enabling context-aware adjustments. The anchor loss further restricts the effect of the inclusive token. Together, these components provide a theoretically grounded and practically motivated framework for mitigating biases in T2I models.

Specific structure and training process of adaptive mapping network

The specific structure and training process of the adaptive mapping network are detailed in Section 4.1, “Implementation Details”.

We recognize that the diagrams may lack clarity and will ensure they are revised in the final version to better illustrate the framework and convey our intentions effectively. Additionally, we will further emphasize key implementation details in the text to enhance understanding.

Quality of generated models

Regarding the "quality of generated models," we evaluate our generation results using standard metrics such as FID (for image quality) and CLIP score (for text-image alignment), as detailed in Section 4.1 Evaluation Metrics. These metrics provide a quantitative assessment of the quality and consistency of the generated outputs.

As for the "impact of prompts, models, and data volume on the final model," our experiments use well-defined prompts and consistent model settings to ensure fair comparisons. The effect of these factors is controlled to focus on evaluating fairness and quality improvements. If the reviewer could clarify the specific concerns, we would be happy to provide additional explanations or conduct further analysis if needed.

Experiment setup

Our setup includes 2.4K images per attribute for $D_{KL}$ calculation, which is notably larger than the 1K images used in ITI-GEN, ensuring a robust evaluation. For complex scene experiments, we focus on demonstrating that the learned adaptive inclusive token is not overfitted to specific sentence structures or image layouts (e.g., close-up human figures). We believe the three examples provided sufficiently prove this point.

Regarding the mention of "limited datasets," we clarify that our experiments are not tied to any specific dataset but rather use a diverse range of prompts to generate images for evaluating the method's effectiveness. If the reviewer could elaborate further on the concerns, we would be happy to address them in more detail.

Discussion on practical use

Our method is particularly valuable in real-world T2I applications where users seek to achieve fair and balanced outputs regarding certain target attribute (e.g., gender) without explicitly specifying particular attribute classes (e.g., "female"). This ensures that the generated images maintain diversity while addressing fairness concerns. For example, in scenarios such as content generation, inclusive advertising, or educational tools, our approach enables the generation of unbiased outputs without requiring extensive user intervention.

Experiment setting of Figure 1 (a)

In Figure 1(a), we evaluate bias in current T2I models using two sets of stereotypically gender-dominated occupations: male-dominated ("doctor," "chief," "farmer," "architect," "software developer") and female-dominated ("ballet dancer," "yoga instructor," "cosmetologist," "fashion designer," "flight attendant"). For each occupation, we generate 100 images using Stable Diffusion 1.5 (SD1.5) with the prompt "A photo of a {occupation}." The first-row results demonstrate significant bias in SD1.5: male-dominated occupations predominantly produce male figures, while female-dominated occupations generate mostly female figures. For TI (a fixed inclusive token method), biases are reduced but inconsistently, with more significant reductions for female-dominated occupations, leading to amplified imbalance across sets. In contrast, our method employs an adaptive inclusive token, enabling customized adjustments that shift the distribution toward equality rather than disproportionately favoring one gender. This results in more balanced and fair outcomes across both occupation sets.

Setting of Table 2

The adaptive mapping network and the anchor loss are independent components. In the "without adaptive mapping network" setting, the Text Transformer (Figure 2, right) directly takes $V_{T_i}$ as input, bypassing the adaptive mapping network. In this configuration, the placeholder $<i>$ embeddings are the only learnable parameters, similar to the textual inversion approach. The anchor loss in Eq. (2) is still computed as described, as it depends on the generated embeddings and not on the adaptive mapping network itself. This setup isolates the impact of the adaptive mapping network, allowing us to evaluate its contribution to fairness.

We thank Reviewer QTbU for recognizing the importance of addressing biases in T2I models and the social benefits of our work. We also appreciate your acknowledgment of the simplicity, model-agnostic nature, and extensive experimental validation of our method. The concerns are addressed as follows:

Non-technical causes of biases in T2I model

We appreciate the reviewer's point that biases in T2I models often originate from dataset limitations. Prior works [1, 2] confirm that biases are predominantly inherited and amplified from training data. However, unbiased datasets are impractical to curate in real-world scenarios. Given the increasing prevalence and societal impact of T2I models, it is critical to develop methods that mitigate these biases from a model-centric perspective to ensure fairer and more inclusive outputs. Regarding the mention of “unclear prompts” as a source of bias, we respectfully disagree. Bias in T2I models primarily involves the assumption of particular social attributes based on implicit associations (e.g., associating "doctor" with male figures in response to the prompt "a doctor"). Bias mitigation methods directly address these issues by ensuring outputs remain consistent with the intended neutrality of prompts.

[1] The Bias Amplification Paradox in Text-to-Image Generation. NAACL, 2024.
[2] Survey of Bias in Text-to-Image Generation: Definition, Evaluation, and Mitigation. ArXiv, 2024.

Potential inaccuracy caused by biased CLIP model

We acknowledge the reviewer's concern regarding the potential bias in the CLIP zero-shot classifier due to its training on real-world datasets. To assess this, we conducted an experiment by generating gender-specific figures for stereotypically male- and female-dominated occupations (e.g., "a photo of a female doctor"). Using the neutral prompt "a photo of a doctor," we calculated the CLIP scores for male and female figures, as shown in the table below:

The results indicate that while CLIP scores exhibit a slight bias towards more frequent attribute classes, the differences are marginal. Although this limitation is recognized, our use of CLIP follows established practices in the literature [1, 2, 3] to classify sensitive attributes. This alignment ensures consistency and comparability with prior work.

[1] ITI-GEN: Inclusive Text-to-Image Generation. ICCV, 2023.
[2] Debiasing Text-to-Image Diffusion Models. ArXiv, 2024.
[3] FairRAG: Fair Human Generation via Fair Retrieval Augmentation. CVPR, 2024.

No significant improvement in Table 1 and qualitative results

In Table 1, we report KL divergence as the fairness metric. We note that KL divergence inherently has an upper limit (0.6931 for binary cases) and exhibits non-linear degradation (slowing as it approaches zero). This behavior may obscure incremental improvements. Regarding FID and CLIP scores, our primary goal is fairness improvement, not outperforming others in these metrics. The combined results demonstrate that our method achieves fairer outputs without compromising image quality or text-image alignment. While our method achieves similar quantitative results to FM, we emphasize FM’s significant degradation in text-image alignment, as illustrated in the third row of Fig. 3(b) and Fig. 4. Such misalignment is critical in T2I tasks, which is also reflected by FM's lower CLIP scores in Table 1.

Qualitatively, our method produces results that are diverse in target attributes while avoiding issues observed in other methods, such as multiple subjects in EI or invisible occupation concepts in FM. These aspects demonstrate that our method presents significant improvements over prior approaches in generating fair and coherent T2I results.

Enhance the performance of other methods

Our method is indeed model-agnostic, provided the text encoder can handle the newly learned embeddings. The adaptive mapping network is trained to transform original occupation-related text embeddings into pseudo embeddings, which are then combined with the original embeddings to achieve fairer outputs. As long as the text embeddings remain unaltered by the other methods, our approach can be seamlessly integrated to enhance their performance without compromising compatibility.

We thank Reviewer QTbU for their thoughtful follow-up and for raising these concerns, as they allow us to further clarify key aspects of our work.

First, regarding the accuracy of the CLIP zero-shot classifier, we acknowledge the potential bias stemming from its training data, as mentioned in our initial response. However, this bias is minor and does not significantly affect its effectiveness for attribute classification in our experiments. The use of CLIP zero-shot classifiers for sensitive attribute classification is a common practice in bias mitigation literature [1,2,3,4,5] due to its robust performance and broad applicability. Additionally, the consistent and meaningful results across our experiments demonstrate that the metrics derived from CLIP remain reliable for evaluating fairness in generative models.

Second, we believe there is a misunderstanding regarding fairness in the bias mitigation field. Bias mitigation is not simply about enabling the model to follow explicit prompts better—for example, generating a female doctor when explicitly instructed with “female doctor.” Such cases merely demonstrate the model’s ability to follow a conditioned prompt and do not reflect its inherent fairness. True fairness lies in how the model responds to neutral prompts where sensitive attributes are left unspecified, such as “a doctor” or “a yoga instructor.” In such scenarios, models often reflect societal biases present in their training data, leading to skewed outputs (e.g., associating "doctor" with male figures and "yoga instructor" with female figures). Our work, as well as the entire bias mitigation literature, seeks to address this fundamental issue of inherent bias, which is critical to enabling fair and inclusive generative outputs.

Additionally, as pointed out in the ITI-GEN paper, "directly expressing the desired attributes in the prompt often leads to sub-optimal results due to linguistic ambiguity or model misrepresentation." This demonstrates that even when sensitive attributes are explicitly included, the model may fail to produce fair or meaningful outputs due to its inherent limitations in interpreting prompts. This further validates that bias mitigation is a non-trivial challenge requiring solutions beyond explicit prompt specification. Our approach addresses this gap by introducing adaptive inclusive tokens that enable fairer outputs in neutral cases, while also mitigating potential linguistic ambiguities.

Finally, regarding the proposed solutions' social benefits, we respectfully disagree that they are limited. By enabling more inclusive outputs for neutral prompts, our method reduces reliance on explicit attribute specification and empowers broader and fairer usage of T2I models in diverse real-world applications.

We hope this clarifies the points of misunderstanding and demonstrates the importance of our work. We kindly ask the reviewer to reconsider their assessment in light of these explanations, as we believe our method contributes meaningfully to addressing fairness challenges in generative models and provides valuable benefits for inclusive AI applications.

[1] ITI-GEN: Inclusive Text-to-Image Generation. ICCV, 2023.
[2] How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions? EMNLP, 2022
[3] Fair Text-to-Image Diffusion via Fair Mapping. arXiv, 2023.
[4] Balancing Act: Distribution-Guided Debiasing in Diffusion Models. CVPR, 2024.
[5] FairRAG: Fair Human Generation via Fair Retrieval Augmentation. CVPR, 2024.

Thank you for addressing my concerns. However, the main issue regarding accuracy remains unclear. I also agree with other reviews regarding the motivation problem. The authors claimed "bias" observed in current large T2I models may stem from social structures—for example, differing gender ratios in certain occupations. Additionally, providing a clear and specific prompt, such as explicitly including gender, could effectively mitigate the claimed bias in large models. Based on this, the proposed solutions to address this "bias" may offer limited social benefits. In summary, I would like to maintain my current ratings.