FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models

Q1. SDID originally computes concept vectors in the h-space. However, I believe the authors compared their method to a version that performs this operation in the prompt embedding space. Did you also evaluate how the approach compares to the original SDID?

A1. You are correct in noting this distinction. SDID originally operates in the diffusion model's latent h-space (the semantic latent space within the denoising network), while our implementation adapts it to work in the prompt embedding space for fair comparison with other post-hoc methods including FairImagen. Since FairImagen and other baselines (TBIE, ITI-GEN) operate on CLIP prompt embeddings rather than internal diffusion latent representations, we standardized all comparisons to the prompt embedding space to ensure methodological consistency. We acknowledge that the original h-space SDID implementation might yield different performance characteristics. Evaluating this comparison would be valuable future work to provide a more comprehensive assessment of debiasing approaches across different operational spaces.

Q2. How can you address that the approach struggles to scale effectively to more complex scenarios, as shown in Figure 3d where many CEO images fail to accurately represent the CEO concept and exhibit visual inconsistencies?

A2. The visual inconsistencies in Figure 3d arise from the inherent trade-off between fairness and semantic fidelity when debiasing across multiple demographic attributes simultaneously. The stronger empirical noise required for joint gender+race debiasing can introduce variation in non-demographic visual elements (background, style). This is a known challenge in fairness-aware generation. Specifically, as we increase demographic diversity, maintaining consistent occupation-specific visual cues becomes difficult. To address this, users can tune the $\lambda$ parameter and noise scale based on their specific requirements. For applications prioritizing visual consistency, we recommend using lower noise values or debiasing single attributes. We are exploring adaptive noise strategies that preserve occupation-relevant features while diversifying demographics, which would improve scalability to complex scenarios.

Q3. How can you address that the approach is very sensitive to hyperparameters such as noise injection scale, hidden dimension size etc. which makes the approach difficult to scale effectively?

A3. We acknowledge the hyperparameter sensitivity and provide extensive ablation studies to guide parameter selection. In Section 4.5.2 (Figure 1), we show that hidden dimension size offers a predictable trade-off: lower dimensions (200-500) improve fairness but reduce quality, while higher dimensions (1500-2000) preserve quality but may reintroduce bias. Similarly, Section 4.5.3 (Figure 2) demonstrates that empirical noise scale follows a smooth progression from 0 to 1. To address scalability concerns, we recommend: (1) starting with default values ($d=1000$, $\epsilon=0.5$) that work well across diverse prompts, (2) using our development set methodology to tune parameters for specific applications, and (3) leveraging the monotonic relationships we observed - parameters don't require exhaustive search but can be adjusted directionally based on desired fairness-quality balance. Future work could explore adaptive parameter selection based on prompt characteristics.

Q4. How can you address that the approach debiases only the input embeddings at inference but does not tackle biases embedded within the pretrained diffusion model itself?

A4. If we only performed fairPCA without adding noise, this would only debias the embeddings, and not resolve biases present within the diffusion model. Unsurprisingly, this doesn't work well, and many biases continue to exist. The addition of noise provides a strong signal that overrides weak biases within the pretrained model in much the same way that prompting "female doctor" causes a diffusion model to generate an image of a female doctor, despite the intrinsic bias of the model that causes it to typically generate 'doctor' as male.

The inability to significantly alter the diffusion model is a fundamental limitation of post-hoc approaches. While the pretrained diffusion model contains learned biases from its training data, our method operates on the interface between text and image generation, which provides significant but not complete control. We chose this approach for practical reasons: (1) it enables immediate deployment without expensive retraining, (2) it works with proprietary models where internal weights are inaccessible, and (3) it allows users to adjust bias mitigation dynamically based on their needs. Our experiments show that despite not modifying the model internals, manipulating prompt embeddings can substantially reduce bias in generated outputs. However, we acknowledge that combining our approach with model fine-tuning could yield even stronger debiasing results. Future work could explore hybrid approaches that leverage both embedding-space and model-space interventions for comprehensive bias mitigation.

Q5. How can you prevent neutral prompts like "doctor" from still generating mostly male images? What component in the approach explicitly prevents this behavior when the model may confuse implicit bias for explicit gender information?

A5. Although "doctor" may be neutral to humans, we found that the model tends to generate white male doctors, revealing implicit biases in supposedly neutral occupations. FairImagen addresses this through two key mechanisms: (1) FairPCA projection removes demographic-correlated variance from all embeddings, including neutral prompts, by projecting them into a subspace orthogonal to learned demographic directions. This ensures that embeddings for "doctor" lose their implicit association with specific gender/race attributes. (2) The empirical noise injection then adds controlled randomness sampled from the actual distribution of demographic attributes, encouraging diverse outputs even when the original embedding had strong implicit bias. The $\lambda$ parameter allows fine-tuning this debiasing strength - for heavily biased neutral terms like "doctor," users can increase $\lambda$ to more aggressively remove the implicit demographic signal while preserving the occupational semantics.

Q6. How can you address the concern that the technical novelty is limited because the core idea mainly involves integrating FairPCA into the prompt embedding space of Stable Diffusion?

A6. For a good discussion as to why papers such as ours should be seen as novel, please see the article "Novelty in Science" by Michael Black.

While FairPCA is an established technique, our contribution lies in adapting and extending it specifically for the text-to-image generation domain. Our technical contributions include: (1) Applying FairPCA to prompt embeddings in diffusion models, which hasn't been explored before. (2) Introducing empirical noise injection to prevent over-neutralization of outputs. (3) Demonstrating how to handle multiple demographic attributes simultaneously. (4) Providing extensive empirical evaluation showing the effectiveness of this approach across different bias scenarios. The value of our work is in showing that simple, principled methods can achieve strong bias mitigation without model retraining, making fair image generation accessible to practitioners who cannot afford to retrain large diffusion models.

Q1. How does FairImagen perform when compared against the existing SOTA FairQueue?

A1. Thank you for highlighting FairQueue - an excellent post-hoc approach that addresses key limitations in prompt learning methods. We have now included FairQueue in our comparison, and it demonstrates strong performance over baseline and ITI models as expected, particularly in quality preservation (MUSIQ: 0.621) while achieving meaningful fairness improvements.

Q2. How does FairImagen perform when considering more fine-grain protected attributes e.g., Facial expression, or more nuance protected attributes e.g., Age?

A2. We appreciate the reviewer's valuable suggestion. Adding more complex demographics like age significantly strengthens our paper. We have conducted additional experiments on age debiasing with the following results:

It should be noted that due to the inherent difficulty in distinguishing precise ages (e.g., differentiating between 35 and 38 years old), we categorized age into three distinct groups: young, middle-aged, and elderly. The results demonstrate that our proposed FairImagen methods achieve the highest fairness scores (0.227-0.242) among all approaches, representing a substantial improvement over the baseline (0.136). Notably, this improvement in fairness is achieved while maintaining competitive image quality metrics, with MUSIQ scores remaining close to baseline performance. This confirms that our approach effectively generalizes to age debiasing, successfully addressing demographic bias across multiple attributes.

Q3. What is the main difference between FairImagen and other Post-hoc editing method like ITI-Gen and FairQueue?

A3. The key differences lie in our methodological approach and technical implementation: (1) ITI-GEN learns discriminative token embeddings from reference images by aligning visual and textual embedding differences, then injects these learned tokens after the original prompt. However, this approach can introduce distortions and semantic inconsistencies. (2) FairQueue identifies that prompt learning methods like ITI-GEN create abnormalities in early denoising steps and proposes Prompt Queuing and Attention Amplification to modify cross-attention maps during generation. (3) FairImagen takes a fundamentally different approach by using FairPCA to project text embeddings into a fairness-aware subspace that explicitly removes group-specific variance before generation begins, offering precise control through the λ parameter and handling multiple demographics simultaneously. We will add detailed technical comparisons and positioning relative to these concurrent works in our camera-ready version.

Q1. What would be the main challenges or steps needed to adapt FairImagen to other embedding architectures like Stable Diffusion 3 (CLIP + T5)? How can you address that experiments are all restricted to CLIP-based image generation models and it is unclear how generalizable FairImagen is to non-CLIP architectures?

A1. We appreciate the reviewer's constructive suggestions. To address concerns about generalizability beyond CLIP-based architectures, we have extended our experiments to include alternative text encoders, specifically T5 and OpenCLIP:

The results demonstrate that FairImagen successfully generalizes across different text encoder architectures. All three variants achieve substantial improvements in fairness compared to the baseline (0.0758), with FairImagen-T5 achieving 5.5× improvement and FairImagen-OC achieving 5.4× improvement in fairness scores. Meanwhile, all variants maintain strong image quality metrics, with accuracy scores above 0.805 and MUSIQ scores around 0.57.

These results validate that our method's core principle—leveraging attention mechanisms to identify and mitigate bias—effectively transfers across different text encoding architectures. This confirms that FairImagen can be readily adapted to emerging text-to-image models regardless of their underlying text encoder, including hybrid architectures like Stable Diffusion 3's CLIP+T5 approach.

Q2. How can you show results on broader demographic attributes beyond binary gender and race/ethnicity categories?

A2. We appreciate the reviewer's valuable suggestion. Adding more complex demographics like age significantly strengthens our paper. We have conducted additional experiments on age debiasing with the following results:

It should be noted that due to the inherent difficulty in distinguishing precise ages (e.g., differentiating between 35 and 38 years old), we categorized age into three distinct groups: young, middle-aged, and elderly. The results demonstrate that our proposed FairImagen methods achieve the highest fairness scores (0.227-0.242) among all approaches, representing a substantial improvement over the baseline (0.136). Notably, this improvement in fairness is achieved while maintaining competitive image quality metrics, with MUSIQ scores remaining close to baseline performance. This confirms that our approach effectively generalizes to age debiasing, successfully addressing demographic bias across multiple attributes.

Q3. Noise direction and magnitude.

A3. The noise direction is not arbitrary. Instead, it is learned from actual group-specific embedding distributions (Section 3.3). We sample from the empirical distribution of how strongly embeddings align with demographic attributes, making it data-driven rather than purely empirical. For the magnitude parameter $\epsilon$, we provide extensive ablation studies (Figures 2 and Appendix F) showing its effect on the fairness-quality trade-off. We currently tune $\epsilon$ on a development set. However, we envision that future work could explore adaptive methods based on prompt characteristics or user-specified fairness constraints.

Q1. How did you choose the methods to compare in sec 4.4? Why not other methods like [18]?

A1. We specifically focus on post-hoc methods that operate at inference time without model retraining as indicated in our title "FairImagen: Post-Processing for Bias Mitigation". Reference [18] (Kim et al.) is a fine-tuning-based approach that requires retraining parts of the diffusion model, which falls outside of our post-processing scope. We selected comparison methods (TBIE, SDID, ITI-GEN, etc.) that directly compete in the same post-hoc embedding manipulation space to ensure fair evaluation within our targeted paradigm.

Q2. Sec 4.5.1: I am not sure if I follow the claims in this section in the table 5. Which of the methods in table 5 are post-hoc? every method except FairPrompt?

A2. All methods compared in Tables 2-4 (collectively referred to as Table 5) are post-hoc methods, including Base, FairPrompt, ForcePrompt, SAL, CDA, TBIE, SDID, SDID-AVG, ITI-GEN, and FairImagen. These methods operate at inference time without requiring model retraining. The key distinction is in their implementation approach: some modify prompt embeddings directly (like FairImagen, TBIE, ITI-GEN), whilst others like FairPrompt and ForcePrompt modify the input prompts themselves before embedding generation. All are considered post-hoc as they avoid the computational overhead of fine-tuning the diffusion model.

Q3. Table 5: I see that many numbers are very close to each other., Can you provide other statistics, like std? or run some statistical test to see if the differences are statistically significant? especially for CLIP and MUSIQ

A3. We thank the reviewer for highlighting the need to assess the stability of our results. We repeated every experiment with 10 independent random seeds (1 – 10) and report the resulting standard deviation for each metric below:

The results reveal an important finding: the difference between our proposed FairImagen and the Base model falls within 1-2 standard deviation magnitude across the quality metrics. This indicates that the impact of our debiasing approach on image quality is only marginally higher than the natural variation introduced by different random seeds. We consider this trade-off acceptable, as it demonstrates that our method achieves meaningful fairness improvements while introducing quality changes that are comparable to the inherent stochasticity of the generation process itself. This validates the effectiveness of our approach in balancing fairness and quality preservation.

Q4. 4.5.4: How can we make sure results in Fig 3 are not cherry picked? can you provide ratio of major.minor attributes with all 4 scenarios?

A4. The results in Figure 3 are randomly picked from our generated samples. We randomly selected representative examples from each method's outputs to ensure fair comparison. Regarding the major/minor attribute ratio, since there are more than three demographic groups in our scenarios, simple ratio metrics are not appropriate evaluation measures. Instead, we evaluate fairness quantitatively using the fairness scores presented in Table 5. Figure 3 serves as illustration, while Table 5 provides the comprehensive quantitative evaluation across all methods and scenarios.

Q5. Can you provide high-res (1Kx1K) comparison between FairImagen, FairPrompt and the third runner in MUSIQ? You have been using SD3 and I see this as a big advantage but still see small images when comparing quality. We want to know what exactly we lose in the trade-off.

A5. Thank you for this good suggestion. We will add high-resolution (1K×1K) comparisons in the next revision. However, due to NeurIPS submission limitations this year, we cannot update the files or provide additional links at this time.

Q6. What happens to gender neutral professions such as teacher/doctor?

A6. This is an interesting question. Even for seemingly gender-neutral professions like "doctor," we found that Stable Diffusion models tend to predominantly generate white male images. This demonstrates that bias exists even in supposedly neutral occupations, further highlighting the necessity of our work. FairImagen helps mitigate these implicit biases to produce more balanced demographic representations across all professions, including those that should indeed be gender-neutral.

Q7. How can you justify the measures in Table 1 for "strong bias mitigation" or "easy deployment"? Why don't we see the level of quality degradation for generations as a row? How can you address the concern that authors designed this table to have most ✓ for post-hoc editing?

A7. "Strong bias mitigation" refers to the methods ability to effectively reduce demographic biases in generated images. "Easy deployment" means that the method can be applied without the need for retraining models. Your suggestion about adding quality degradation level as a row is excellent, and we will include this in the revision. We did not intentionally design this table to favor post-hoc methods. These criteria reflect practical concerns encountered in the literature, in our work and, in aspects we care about. Other researchers might prioritize different criteria based on their specific use cases.

Q8. How can you provide generalization to other diffusion encoders since you mentioned choosing post-hoc method because of "simplicity and compatibility with a wide range of off-the-shelf diffusion models"?

A8. We appreciate the reviewer's insightful question. To demonstrate that FairImagen is indeed an off-the-shelf post-hoc method, we have evaluated our approach with multiple text encoders including T5 and OpenCLIP in addition to the standard CLIP encoder. The results are as follows:

These results demonstrate that our method successfully adapts to different encoder architectures without requiring architecture-specific modifications. The consistent performance across encoders validates our claim of being an off-the-shelf solution—FairImagen can be directly applied to various pre-trained diffusion models regardless of their text encoding choice. This compatibility is achieved because our method operates on the attention mechanisms that are common across these architectures, rather than relying on encoder-specific features. This flexibility makes FairImagen practical for real-world deployment across diverse text-to-image generation systems.