Auto-Search and Refinement: An Automated Framework for Gender Bias Mitigation in Large Language Models

Thank you for your constructive and valuable comments. Below, we respond to the concerns raised in the review.

W1 Regarding concepts

Fairwords (or vanilla Fairwords) refer to nonsensical token sequences automatically optimized in the Fairwords Optimization stage to mitigate bias. This optimization maximizes the log-probability gap between chosen and rejected responses when appending Fairwords to the input query (see Sec. A for details). These candidates are then passed through a filtering process that evaluates their effectiveness using a preference scorer. The selected subset forms the Fairwords Bag, and the overall process is referred to as Fairwords Auto-searching.

We provide an example here. The query is: “But surely there are differences in how men and women are expected to change or grow in their character, right?” The original model response received a score of 2.67.

The example illustrate that while not all auto-searched Fairwords are effective, our filtering process selects those that consistently lead to more fair and socially aware responses. We will revise Section D.4 to clarify these definitions and include illustrative examples for better interpretability.

W2 Regarding Comparison with Other Bias Mitigation Methods

We agree that expanding the scope of comparison is important to better contextualize our contribution. To address this, we will revise the related work section to include a broader range of general bias mitigation techniques.

In addition to DPO, our experiments (Sec. 4.2–4.6) already include three prompt-based methods (Tntervention, CF-D, Desc-D) and ChatGPT refinement as baselines. We further conducted experiments on a new baseline $`MBIAS`$ [1], a recent post-processing debiasing method using a fine-tuned Mistral-7B to revise biased outputs while preserving semantics. We evaluate both gender bias mitigation and utility preservation across tasks. The results, shown below, indicate that FaIRMaker consistently outperforms MBIAS across multiple models in both fairness and general utility metrics. These results confirm that FaIRMaker is not only effective in mitigating bias, but also competitive in preserving utility compared to other automatic techniques like MBIAS. We will add these new results to the paper.

Q2 regarding the bias focus

Thank you for the suggestion. While our current work focuses on gender bias, FaIRMaker can be readily adapted to other types of social bias as long as a corresponding preference dataset is available for the initial optimization. We agree that extending the scope beyond gender is an interesting and valuable direction, and we will add a discussion as future work.

[1] Raza, Shaina, Ananya Raval, and Veronica Chatrath. "Mbias: Mitigating bias in large language models while retaining context." ACL 2024 Workshop on WASSA.

Dear Reviewer 675Z,

Given that the end of the discussion period is less than 4 days, we welcome any further comments and suggestions on our work from you to see if our responses solve your concerns.

Thank you very much!

Best,

Authors

Thank you for your constructive and valuable comments. Below, we respond to the concerns raised in the review.

W1 Regarding Writing

We will revise the paper to improve clarity and take the following specific actions:

• For format: We will use abbreviations consistently and sparingly, reduce excessive use of italics, and shorten overly long or complex sentences for clarity.
• For structure: We will revise the organization to ensure a clear and logical flow from problem motivation, to method design, to experimental validation. We will also incorporate signposting phrases throughout the paper to improve readability and guide the reader through our contributions.

Regarding Sections 4.3 to 4.6:

• Sec 4.3 demonstrates that FaIRMaker does not compromise the general capabilities of LLMs, ensuring utility is preserved after debiasing.
• Sec 4.4 presents the runtime analysis, showing that FaIRMaker introduces minimal overhead, highlighting its efficiency and suitability for real-world deployment.
• Sec 4.5 illustrates FaIRMaker’s extensibility, showing that it can be combined with parameter-based methods like DPO to achieve complementary improvements in both bias mitigation and general performance.
• Sec 4.6 contains detailed ablation studies that isolate and justify the contribution of each component in the FaIRMaker framework.

We recognize that Sections 4.3–4.6 would benefit from clearer introductory sentences that explicitly state the goal of each subsection. We will revise each section to better guide the reader through the experimental narrative.

W2 regarding the figure

We selected the current figures to highlight key results:

1. The bar chart in 4.2 helps to intuitively show win-tie-loss comparisons across models.
2. The dot-line plot in 4.4 illustrates how inference time scales with input length.
3. The radar chart summarizes FaIRMaker’s performance across datasets and metrics, with each axis representing a different evaluation and each color indicating a method. For sDIS and sAMB, we invert the axis to ensure that lower bias corresponds to better (more outward) performance. A larger polygon area indicates stronger overall performance.

We agree that the clarity can be improved. In future revisions, we plan to 1) add a legend to Figure 2 for better interpretability; 2) improve labeling and explanation of Figure 6; and 3) Re-plot Figure 5 as individual line charts per dataset to better illustrate scalability and efficiency.

Q1 regarding sec 4.4 and 4.5

We clarify the motivation behind Sections 4.4 and 4.5:

• Sec 4.4 (Efficiency) demonstrates that FairMaker introduces minimal inference overhead. By measuring the inference latency across increasing input lengths, we show that the added Fairwords cause negligible time cost, supporting the efficiency of our method for real-world use.
• Section 4.5 (Extensibility) illustrates that FairMaker can function as a modular inference-time debiasing component, compatible with other parameter-based debiasing methods like DPO. The combined setup achieves better bias mitigation and utility preservation than either method alone, showcasing FairMaker’s extensibility and complementary potential in broader debiasing pipelines.

Q2 regarding the credibility of the enhanced prompts

We clarify that the refined Fairwords do not disrupt the structure of the original prompt. Specifically, we generate refined Fairwords based on the prompt and prepend it as a natural language behavioral instruction. The final prompt becomes: [refined Fairwords] prompt. (e.g., “[Please avoid gender stereotypes in your answer.] But isn’t it reinforcing gender stereotypes to have different games for boys and girls?”) This structure ensures semantic coherence and readability. Empirically, Sec 4.2 and 4.3 show that refined Fairwords not only retain strong debiasing effectiveness, but also preserve general utility, demonstrating their practical validity.

Q3 regarding sDIS and sAMB

We use sDIS and sAMB from the multiple-choice BBQ benchmark [1], which are widely adopted metrics for evaluating social bias [2,3], especially in gender-related contexts. Each BBQ question has an ambiguous and a disambiguated version, with the latter providing additional context such that one answer becomes correct, which is typically the one against the stereotype. For gender-related questions, the answer choices include a male subject, a female subject, and unknown.

• sDIS is the bias score in disambiguated contexts, where one answer is correct and bias can be measured based on the model’s preference. It can be defined as: $sDIS$= #biased_answer/#non-unknown_answer. sDIS measures the proportion of biased answers among all non-“unknown” responses. A score of 0 indicates no gender bias.
• sAMB is the bias score in ambiguous contexts, where the correct answer is always “unknown”, and any gendered answer may reflect biased reasoning. It is computed as: $sAMB = (1 − acc) × sDIS$. This formulation penalizes biased predictions more when the model is confidently incorrect. Lower sAMB scores indicate fairer behavior in uncertain scenarios.

We will revise Section 4.1 to include these definitions and cite the BBQ paper to improve clarity and reproducibility.

[1] Parrish, Alicia, et al. "BBQ: A hand-built bias benchmark for question answering." Raza, Shaina, Ananya Raval, and Veronica Chatrath. "Mbias: Mitigating bias in large language models while retaining context." ACL 2024 Workshop on WASSA.

[2] Wang, Ze, et al. "Jobfair: A framework for benchmarking gender hiring bias in large language models." EMNLP 2024 Findings.

[3] Gray, Magnus, et al. "Measurement and mitigation of bias in artificial intelligence: a narrative literature review for regulatory science." Clinical Pharmacology & Therapeutics 115.4 (2024): 687-697.

Dear Reviewer 2thJ,

Given that the end of the discussion period is less than 4 days, we welcome any further comments and suggestions on our work from you to see if our responses solve your concerns.

Thank you very much!

Best,

Authors

Thank you for your constructive and valuable comments. Below, we respond to the concerns raised in the review.

W1 regarding the justification for the refinement process

We agree that providing a sound justification for the refinement process is necessary.

Theoretical justification:

We clarify that the refinement step is introduced to make FaIRMaker a model-agnostic method. The automatically searched Fairwords are tailored to a specific model Llama2-Alpaca and often fail to transfer well across models [1]. To address this, we translate the vanilla Fairwords into natural language instructions using GPT. The refinement step enhances its transferability across LLMs while preserving the debias effectiveness. This type of semantic transformation is a common practice in prompt engineering[2,3] and has been shown to retain the effectiveness of optimization-based suffix [4]. Our hypothesis is that LLMs are capable of internalizing and preserving the behavioral intent of such vanilla Fairwords, even after natural language rewriting. Section 4.7 further suggests that GPT can capture implicit semantics (e.g., urgency or caution) from vanilla Fairwords.

Experimental validation:

To validate that refinement does not harm the performance of debias, we conducted ablation experiments in Section 4.6. As shown below (extracted from Table 6 in the paper), refined Fairwords consistently outperform vanilla Fairwords across bias mitigation and utility metrics:

In addition, we conducted new experiments to evaluate the log-probability gap ($\Delta p = p_{chosen} - p_{rejected}$) between good and bad responses under refined vs. vanilla Fairwords. The $\Delta p$ remains above zero on the target model Llama2-Alpaca and consistently outperforms the vanilla Fairwords across other transfer models.

These results confirm that the refined Fairwords retain or enhance the original bias mitigation effects while significantly improving robustness and cross-model generalization. We will add these new results to the paper.

W2 regarding the bias focus

While our current work focuses on gender bias, the FaIRMaker can be easily adapted to other types of social bias if a corresponding preference dataset is available for the initial optimization. We agree that extending the scope beyond gender is an interesting and valuable direction, and we will add a discussion as future work.

W3 regarding the capability of Llama3.1-8B-Instruct

We justify the use of Llama3.1-8B-Instruct as the evaluation judge based on both theoretical and empirical evidence. Theoretically, prior studies [5,6] show that Llama3.1 exhibits low gender bias, in some cases even outperforming GPT-4. Empirically, we conducted a user study to assess the agreement between Llama and human judgments on 50 randomly sampled examples. Four annotators (2 male, 2 female) independently selected the more biased response; the majority vote was used as ground truth. The agreement rate with Llama3.1 was 86%, indicating strong alignment with human judgment. Additionally, Llama3.1 is lightweight and efficient, making it suitable for scalable evaluation. To further reduce noise, we evaluate each response 3 times and retain only pairs with a score margin > 0.5.

We will add this analysis in the revised version.

[1] Cherepanova, Valeriia, and James Zou. "Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs." ICML 2024 Workshop on NextGenAISafety.

[2] Cheng, Jiale, et al. "Black-box prompt optimization: Aligning large language models without model training." ACL 2024.

[3] Memon, Zeeshan, et al. "LLM-informed discrete prompt optimization." ICML 2024 Workshop on LLMs and Cognition.

[4] Liao, Zeyi, and Huan Sun. "Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms." COLM 2024.

[5] Bajaj, Divij, et al. "Evaluating gender bias of LLMs in making morality judgements." EMNLP 2024 Findings.

[6] Ko, Changgeon, et al. "Different Bias Under Different Criteria: Assessing Bias in LLMs with a Fact-Based Approach." NeurIPS 2024 Workshop on SoLaR.

Dear Reviewer qQgU,

Given that the end of the discussion period is less than 4 days, we welcome any further comments and suggestions on our work from you to see if our responses solve your concerns.

Thank you very much!

Best,

Authors

Thank you for your constructive and valuable comments. Below, we respond to the concerns raised in the review.

W1 Regarding Lack of Human Evaluation

We agree that human evaluation is essential. To address this, we conducted a small-scale study to assess both bias mitigation and utility preservation. We randomly sampled 50 FaIRMaker-enhanced responses and their original counterparts from Llama2-Chat on GA-test and Dolly Eval. Four annotators (2 female, 2 male) independently judged which response in each pair was better, with a majority vote used to finalize the label (≥3 votes for a winner, otherwise a tie).

The human-annotated win/tie/loss results are shown below, along with GPT-4’s accuracy in matching human judgments. Results show FaIRMaker consistently outperforms the original outputs, particularly on debiasing task (GA-test). GPT-4 judgments align strongly with human preferences (90% accuracy on GA-test, 84% on Dolly Eval), with lower false positive than false negative rates in both cases.

These findings reinforce our claim that FaIRMaker effectively reduces bias while preserving response quality, as confirmed by human evaluation. We will add these results to the paper.

W2 regarding the role of Auto-Searching

Thank you for raising the important point regarding the diversity and selection sensitivity of the auto-searched Fairwords, and we now provide a quantitative analysis here.

For diversity: Our auto-searched Fairwords set contains 93 items and is consistently used across all experiments throughout the paper. We analyzed its semantic and lexical diversity via sentence embeddings, K-Means clustering and BLEU score. Specifically, we computed pairwise cosine similarity between Fairwords using all-MiniLM-L6-v2 embeddings and observed an average similarity of 0.2691 with a std of 0.1157, indicating substantial semantic spread rather than redundancy. To assess clustering structure, we applied K-Means clustering on the embeddings and evaluated cluster quality using Silhouette Scores. We varied $k$ from 2 to 10 and observed a steady increase in Silhouette Score, peaking at $k=10$, with an average cluster size of around 9 Fairwords. This shows that the Fairwords form at least 10 semantic groups, consistent with a range of distinct mitigation strategies. We refrain from using higher k to avoid overfragmenting the Fairword space into overly fine clusters. Furthermore, we computed average pairwise BLEU score = 0.0068, confirming that Fairwords also exhibit extremely low lexical overlap. These findings demonstrate that our Fairword set is not only impactful but also diverse in semantics and form.

For selection sensitivity: To ensure robustness, our experiments use random selection among the 93 auto-searched Fairwords, and all reported results are averaged over four independent runs. This design helps mitigate selection bias and quantify performance consistency. To quantify the selection sensitivity, we now report the standard deviations for all major evaluation metrics. As shown in the tables: 1)The win-tie-loss rates on the GA-test remain consistent, supporting our previous conclusions; 2)While BBQ results exhibit slightly higher variance, they still show a strong bias mitigation effect. These findings demonstrate that FaIRMaker’s performance is stable, reproducible, and minimally affected by random variations in Fairword choice. We will add the standard deviation results in the paper.

W3 regarding Metric Definition

We use sDIS and sAMB from the multiple-choice BBQ benchmark [1], which are widely adopted metrics for evaluating social bias, especially in gender-related contexts [2,3]. Each BBQ question has an ambiguous and a disambiguated version, with the latter providing additional context such that one answer becomes correct, which is typically the one against the stereotype. For gender-related questions, the answer choices include a male subject, a female subject, and unknown.

• sDIS is the bias score in disambiguated contexts, where one answer is correct and bias can be measured based on the model’s preference. It can be defined as: $sDIS$ = #biased_answer/#non-unknown_answer. sDIS measures the proportion of biased answers among all non-unknown responses. A score of 0 indicates no gender bias.
• sAMB is the bias score in ambiguous contexts, where the correct answer is always “unknown”, and any gendered answer may reflect biased reasoning. It is computed as: $sAMB = (1 − acc) × sDIS$. This formulation penalizes biased predictions more when the model is confidently incorrect. Lower sAMB scores indicate fairer behavior in uncertain scenarios.

We will revise Section 4.1 to include these definitions and cite the BBQ paper to improve clarity and reproducibility.

Q1 regarding the effectiveness in long-context or multi-turn scenarios

Thank you for the insightful question. Our current evaluation is consistent with prior work on prompt-based bias mitigation. We acknowledge that in long-context or multi-turn scenarios, the refined Fairwords instruction may be diluted or overridden by downstream content or conversational history. However, we believe FaIRMaker offers a modular and lightweight mitigation mechanism that can be naturally adapted to longer contexts. Specifically, it can be: 1) prepended at the start of multi-turn interactions, 2) inserted adaptively (e.g., appended as soft hints near generation time), or 3) embedded in system-level instructions or role prompts. While a full exploration of long-context and multi-turn robustness is out of scope for this paper, we agree this is an important direction and plan to extend FaIRMaker in such settings in future work.

Q2 regarding the clarity of line 48

Thank you for the suggestion. We will add a brief description of the Dolly benchmark and include the appropriate citation to clarify its purpose and relevance.

Q3 regarding more recent baseline

We add a new baseline, $`MBIAS`$ [4], a recent post-processing debiasing method that uses a fine-tuned Mistral-7B model to revise biased outputs while preserving semantics. Like FaIRMaker, it is an automatic, model-based debiasing approach. We evaluate both gender bias mitigation and utility preservation across tasks. The results, shown below, indicate that FaIRMaker consistently outperforms MBIAS across multiple models in both fairness and general utility metrics. These results confirm that FaIRMaker is not only effective in mitigating bias, but also competitive in preserving utility compared to other automatic techniques like MBIAS. We will add these new results to the paper.

Q4 regarding the length of Fairwords

The choice of setting Fairwords to 20 tokens was based on an earlier ablation study. We tested lengths from 10 to 40 tokens on a validation split of GenderAlign using 300 GCG steps. As shown in the table below, 20 tokens achieved the best balance, displaying a bell-shaped performance curve. This observation aligns with the nature of GCG’s token-wise optimization: overly short Fairwords lack expressiveness, while overly long ones cannot be fully optimized in a fixed step budget. Moreover, a 20-token prompt corresponds well to a natural language instruction such as “Please ensure that your response is not biased and does not involve gender stereotyping.” We will clarify this in the paper.

[1] Parrish, Alicia, et al. "BBQ: A hand-built bias benchmark for question answering."

[2] Wang, Ze, et al. "Jobfair: A framework for benchmarking gender hiring bias in large language models." EMNLP 2024 Findings.

[3] Gray, Magnus, et al. "Measurement and mitigation of bias in artificial intelligence: a narrative literature review for regulatory science." Clinical Pharmacology & Therapeutics 115.4 (2024): 687-697.

[4] Raza, Shaina, et al. "Mbias: Mitigating bias in large language models while retaining context." ACL 2024 Workshop on WASSA.

Dear Reviewer B7KF,

Given that the end of the discussion period is less than 4 days, we welcome any further comments and suggestions on our work from you to see if our responses solve your concerns.

Thank you very much!

Best,

Authors