Response to W1

We sincerely thank the reviewer for their insightful comments and valuable feedback.

We agree that the theoretical guarantee of performance preservation (Section 4.3) relies on strong assumptions—namely, that the bias detector achieves a high TPR/FPR ratio and the LoRA correction benefits minority groups with minimal adverse effects on the majority. FairNet is designed to satisfy these assumptions in practice.

1. Justifiability of Assumptions

Section 4.3 ("Why FairNet Satisfies the Condition", p.6) outlines how these conditions are encouraged by our design:

• LoRA benefits minority groups. The contrastive loss explicitly aligns minority and majority representations to enhance fairness.
• Limited impact on majority. Soft-label guidance ensures only minor perturbations to majority samples, typically preserving or slightly improving their performance.

FairNet is designed to satisfy theoretical conditions under realistic assumptions. Table B further shows that even with a suboptimal TPR/FPR ratio, FairNet performs no worse than the ERM baseline, demonstrating its robustness.

2. Intersectional Bias Evaluation

We further validated FairNet on the HateXplain dataset across race and gender attributes (Supplementary D.1, p.21). Results show consistent fairness gains (e.g., BERT-base EOD(Race): 20.1 → 9.8; EOD(Gender): 12.9 → 8.2) with stable accuracy, demonstrating robustness under intersectional settings.

3. Planned Revision

We will revise Section 4 to clarify that the theory assumes a binary group setting. While the assumptions may not always strictly hold, our experiments show that FairNet remains effective in more complex bias scenarios, and we plan to extend the theory accordingly in the future.

Response to W2:

We sincerely thank the reviewer for raising the important question concerning FairNet’s sensitivity to the quality of its internal bias detector. This is indeed a critical consideration.

As detailed in Section D.2 of the supplementary material ("Impact of Training Data Size for FairNet-Partial", p. 22), we conducted a dedicated quantitative analysis. Table 5 shows:

• Under minimal supervision (0.1% labeled): The detector performs poorly (TPR/FPR = 10.9), yet FairNet-Partial achieves notable fairness gains: WGA improves from 77.9% to 82.1%, EOD drops from 10.0% to 7.1%, with accuracy maintained at 95.7%.
• With more labels: Detector quality improves (TPR/FPR up to 27.2), and fairness metrics improve correspondingly—demonstrating a smooth, monotonic scaling with supervision quality.

1. FairNet-Unlabeled with weak detectors
We include an extension showing that even in fully unsupervised settings, FairNet is robust:

Table A: Performance of FairNet-Unlabeled (extension to Table 5)

2. Sensitivity to detector noise
We simulate degraded detector quality via label noise, as shown in Table B. Even at 100% noise (TPR/FPR = 1.14), FairNet performs comparably to ERM:

Table B. Impact of varying label noise on FairNet’s performance

This highlights a key safety property: even when the detector fails, FairNet’s performance does not degrade below the baseline.

Response to W3:

We sincerely thank the reviewer for highlighting the need for broader empirical evaluation. We agree that such validation is important and summarize below the extensions we have conducted to address these concerns:

1. Evaluation on Multiple and Intersectional Attributes

Section D.1 (p. 21) reports results on the HateXplain dataset across both race and gender attributes, showing that FairNet can reduce bias across multiple sensitive dimensions simultaneously. We will make this clearer in the revised version.

2. Robustness to Detector Failure

Section D.2 (p. 22) and Table B present a sensitivity analysis covering a wide range of detector qualities (TPR/FPR), including extreme cases where the detector is noisy or fails completely. Results show that FairNet degrades gracefully and never underperforms the ERM baseline.

3. Threshold Analysis

Section D.3 (p. 23) and Table C present an ablation study on the activation threshold $\tau$. Lower thresholds trigger LoRA more frequently, enhancing fairness at a modest cost to accuracy; higher thresholds better preserve accuracy with reduced fairness improvement. We adopt a default setting of $\tau = 0.5$, which offers a balanced trade-off. The method remains robust across a range of $\tau$ values, allowing practitioners to adjust it to suit specific application needs.

Table C. Impact of activation threshold on FairNet’s performance

4. Scalability and Deployment

FairNet introduces minimal overhead in terms of inference time and memory (Table D). Its components are lightweight and designed to scale efficiently with additional attributes. We will clarify this in the revised paper to reinforce its practicality.

Response to W4:

We sincerely thank the reviewer for pointing out the lack of discussion on computational complexity and scalability.

FairNet is designed to be lightweight and modular:

Lightweight Components

The bias detectors are implemented as small MLPs with negligible cost. LoRA adds fewer than 1% additional parameters, making it highly efficient.

Linear Scalability

FairNet scales linearly with the number of sensitive attributes. Each attribute introduces one small detector and one LoRA module. These components are independent and can optionally be shared for multi-label scenarios. As a result, the total overhead remains modest, even with multiple attributes or high-dimensional inputs.

Planned Revision

To clarify this, we will include a new subsection in the supplementary material titled "Computational Complexity Analysis", with the following quantitative results:

Table D. Computational overhead analysis of FairNet
Parameters are reported in millions (M), and FLOPs in giga-FLOPs (GFLOPs).

FairNet increases parameter count by only 0.67% (ViT) and 0.28% (BERT), and GFLOPs by 8.2% and 3.7%, respectively—demonstrating that it remains efficient and scalable in practice.

Response to Q1:

We sincerely thank the reviewer for this thoughtful and important question. In brief, FairNet demonstrates strong robustness to the accuracy of the bias detector. Even under extreme scenarios—such as when the detector is weak (e.g., only 0.1% of the training data is labeled in a semi-supervised setting) or nearly fails completely (e.g., 80% label noise injection)—FairNet still achieves substantial fairness improvements without degrading performance below the baseline. This indicates a desirable safety property of the method.

We have provided detailed quantitative evidence in our response to Weakness 2, which includes results for both FairNet-Partial and FairNet-Unlabeled, as well as a comprehensive sensitivity analysis with noise-injected detectors (see Table B). We kindly refer the reviewer to that section for full details.

Response to Q2:

We sincerely thank the reviewer for this insightful question. Across all experiments, we use a fixed default activation threshold $\tau = 0.5$ without dataset-specific tuning, which supports the generality of our method.

The system is also robust to the choice of this threshold. As shown in our response to Weakness 3 (Point 3), the ablation study in Table C demonstrates that $\tau$ provides a clear and predictable trade-off between task accuracy and fairness. Practitioners can flexibly adjust this value to align with application-specific goals. For a detailed discussion, we respectfully direct the reviewer to that section.

Response to W1:

Thank you for your insightful comments regarding our framing of fairness and the fairness–accuracy tradeoff.

On Bypassing the Fairness–Accuracy Tradeoff:

We agree that improving WGA while holding majority group accuracy constant will mathematically lead to an improvement in overall accuracy. Our claim of bypassing the 'performance–fairness tradeoff' is made in the context of prior work, where fairness interventions often result in a net decrease in overall accuracy-typically because gains for minority groups are outweighed by significant performance degradation for majority groups. For example, in Table 1, GroupDRO achieves a high WGA on CelebA (87.4%) but sacrifices overall accuracy (94.0%), which is lower than the baseline ERM (95.8%).

In contrast, FairNet’s key contribution lies in its ability to substantially improve worst-group performance while preserving, and in some cases slightly enhancing the overall accuracy. This stands in contrast to the common pattern where fairness improvements come at the cost of top-line model performance.

On the EOD Metric and Representation Alignment:

We appreciate your observation that a reduction in average EOD may result from overall accuracy gains rather than targeted fairness improvements. To address this, we provide a more detailed view of group-wise error distributions in our multi-attribute debiasing experiments in the Supplementary (Table 4, page 21). We report disaggregated True Positive Rates (TPR) and False Positive Rates (FPR) across demographic groups.

For instance, in the case of DistilBERT-base, the baseline ERM model achieves a Female TPR of 67.5%. After applying FairNet targeted at the Female demographic (FairNet-Fe), the Female TPR increases substantially to 78.6%, while the Male TPR and FPRs for both groups remain largely unchanged. This indicates that FairNet is not merely improving overall accuracy; rather, it is directly correcting representational disparities by improving sensitivity for the disadvantaged group. Such targeted improvements are consistent with the fairness criterion of Equalized Odds, where the goal is to equalize TPR and FPR across groups.

Our theoretical argument in lines 215-225 is that aligning latent representations provides a direct mechanism for achieving this form of conditional fairness. The experimental results discussed above offer concrete empirical support for that claim.

Thank you again for pointing out this ambiguity. We will revise the relevant sections in the manuscript to clarify our definitions.

Response to W2:

Thank you for your thoughtful analysis of our theoretical contribution in Section 4.3. Your interpretation is correct: Equation (8) formalizes an intuitive condition-overall performance will improve if the accuracy gains on correctly identified minority samples outweigh the performance degradation caused by misclassified majority samples. The main purpose of Equation (8) is to establish a condition under which fairness interventions do not degrade overall performance. Identifying such a no-degradation condition is a key conceptual contribution of our work.

We would like to clarify that our novelty lies not in the inequality itself, but in the architectural design that explicitly aims to satisfy this condition in practice. As discussed in lines 343-351, conventional debiasing methods are typically static and global. They lack an internal mechanism for selectively applying corrections. Thus, they cannot take advantage of the performance-preservation condition, which fundamentally depends on the selective power of the detector.

The contribution of Section 4.3 is to provide a formal guarantee that our proposed architecture, by decoupling bias detection from correction, can provably improve WGA without degrading overall accuracy. This theoretical result supports our broader claim that dynamic, instance-level approaches are more effective than static methods in balancing fairness and accuracy.

Response to W3:

We thank the reviewer for pointing out the lack of methodological clarity in our original manuscript. We will incorporate the following key details into the main text to improve transparency and reproducibility.

(a) Selection of sample pairs for contrastive learning: As detailed in Section C.3.2 of the supplementary material (Lines 991–1021), we do not construct positive/negative pairs by selecting individual instances during training. Instead, we precompute target target representations using the baseline ERM model, and define contrastive targets as follows:

• Positive target: For an anchor sample $x_a$ (from the minority group with label $y_a$), the positive target is defined as the average embedding of all majority group samples that share the same label $y_a$.
• Negative target: Similarly, the negative target is computed as the average embedding of samples with labels $y_n \ne y_a$.

The contrastive loss is then constructed to encourage the LoRA-adjusted representation of the anchor sample $x_a$ to be closer to the positive target and farther from the negative target.

(b) Pseudo-label generation in FairNet-Unlabeled: In the unsupervised setting, we apply the Local Outlier Factor (LOF) algorithm on the intermediate representations $h^{(l)}(x)$ from the baseline model. Clustering is performed based on LOF scores, and the cluster with the smallest number of samples is treated as the minority class, while the rest are treated as majority class. These pseudo-labels are then used to train both the bias detector and the contrastive learning module with LoRA. This approach is based on the assumption that instances from underrepresented or minority groups tend to behave as outliers in the representation space.

Response to W4:

This is a very important and valid ethical concern. We thank the reviewer for encouraging us to use more precise language and to engage more directly with the ethical dimensions of our work.

We agree that the phrase 'identifying biased instances' may be misleading. A more accurate description, which we will adopt in the revised manuscript, is that the detector identifies instances predicted to belong to a pre-defined, performance-disadvantaged minority group. The 'bias' we aim to mitigate is a systemic property of the model’s performance on that group, rather than a property of any individual instance. The intervention is thus applied to instances from groups for which the model is known to have systematically lower accuracy.

This approach, which involves group-aware interventions, does raise important ethical questions around disparate treatment. However, it aligns with a well-established paradigm in the algorithmic fairness literature, shared by many in-processing (e.g., GroupDRO) and post-processing methods that leverage group information to promote equitable outcomes. The objective is to address observed disparities in performance metrics, which may necessitate treating different groups differently.

We believe this is a crucial discussion to include. Accordingly, we will add a dedicated paragraph to the limitations section or introduce a new ethics statement to explicitly address this issue.

Response to Q1:

This is an excellent question regarding the implementation details of our experimental setup. While our notation is general and permits a more complex configuration, in the experiments presented in this paper, we adopted a simplified yet effective design that balances performance and efficiency.

Specifically, we employ a single bias detector ($D_\phi$), which operates on intermediate representations extracted from the encoder of the base model. The output of this single, strong detector is then used to conditionally activate all LoRA modules inserted at various intermediate layers. In other words, all LoRA modules are controlled by a unified prediction regarding minority group membership. This design avoids potential inconsistencies or conflicts that might arise from using multiple detectors across different layers, and it is highly effective in practice.

We appreciate your close reading. Our intent was merely to indicate the insertion point of the detector, not that a separate detector is placed at each layer. We will clarify this point in the revised manuscript to avoid ambiguity.

Furthermore, we agree that ablation studies on the number and placement of detectors represent a compelling direction for future work. One could explore strategies such as majority voting across layer-specific detectors or introducing detectors tailored to individual layers. However, such designs would increase model complexity and inference cost, which pose significant challenges. We plan to explore these directions in future research.

Response to Q2:

Thank you for this thoughtful suggestion. "we remove the contrastive loss from Eq.(3) and only train on $L_\textnormal{task}$" is indeed a more accurate description of our ablation setup. We will revise it in the manuscript for clarity. As you correctly inferred, the loss is not applied separately to each layer.

Your suggestion to evaluate this ablation in the fully-supervised CelebA setting is particularly valuable. As shown in the table below, FairNet-Full allows us to isolate the effect of the contrastive loss more cleanly, without the confounding influence of the detector. Compared to using cross-entropy loss alone, the contrastive loss yields a notable improvement in WGA, supporting the hypothesis that alignment with the majority group's representation is beneficial. This result strengthens the empirical justification for our approach.

Response to W1:

Thank you for this insightful comment, which gives us the opportunity to clarify FairNet's core contributions and distinguish it from related literature. While we agree that FairNet can be broadly categorized within the 'detect-then-correct' paradigm, its novelty lies in how it adapts this concept to the unique challenges of algorithmic fairness, its specific technical components, and its theoretical guarantees.

We would like to highlight three key areas of novelty that differentiate FairNet from prior work in selective prediction and test-time adaptation:

1. Novel Problem Formulation and Goal:

   Selective prediction aims to improve model reliability by abstaining from predictions on low-confidence samples. FairNet's goal is fundamentally different: it aims to always make a prediction while ensuring that the prediction is fair. Specifically, it works to improve the accuracy for the worst-performing subgroup (WGA) without degrading overall performance.

   Test-time adaptation (TTA) primarily addresses domain shift between training and test sets. In contrast, FairNet is designed to correct biases that exist within the training distribution itself, targeting unfairness between subgroups rather than adapting to a new domain.

2. Unique Correction Mechanism and Training Objective:

   Unlike existing methods, FairNet's correction mechanism is a conditional LoRA module trained with a novel contrastive loss function. This loss is a core contribution and is specifically designed to minimize intra-class representation gaps between different sensitive groups. It directly targets the representational disparities that cause unfairness-a mechanism not present in standard TTA or selective prediction frameworks. Notably, the LoRA parameters are learned during the training phase, rather than updated on-the-fly at test time based on batch statistics as in many TTA methods.

3. Theoretical Guarantees for Fairness:

   FairNet is supported by a theoretical analysis that provides a formal guarantee for improving fairness without performance loss. We establish a clear condition (Equation 8) linking the bias detector's quality (TPR/FPR ratio) to a provable increase in worst-group performance. This formal underpinning for achieving fairness without the typical accuracy trade-off is a significant contribution that distinguishes FairNet from more heuristic 'detect-then-correct' approaches.

In summary, while the high-level concept has precedents, FairNet's novelty comes from its specific application to instance-level fairness, its unique contrastive training objective tailored for bias correction, and its formal theoretical guarantees. These contributions address a distinct and critical challenge in machine learning not covered by existing selective prediction or TTA literature.

Response to Q1:

Thank you for this excellent question. It allows us to describe our experimental setup and recommendations more clearly.

While our framework supports both approaches, we used and would recommend a single, shared detector trained via multi-label learning to handle multiple sensitive attributes, primarily for its parameter efficiency. We encourage this method when multiple biases need to be addressed simultaneously.

However, FairNet is flexible. For use cases that require more granular control, one can train separate detectors for each attribute. The advantage of this approach is that biases can be addressed progressively, ensuring that correcting for a new attribute does not negatively impact fairness guarantees already achieved for others. This method offers greater control at the cost of a modest increase in parameter overhead.

Response to Q2:

Thank you for these important questions regarding our unsupervised approach. We address each of them below.

1. Unsupervised Method Used

   For the FairNet-Unlabeled experiments, we utilized the Local Outlier Factor (LOF) method. As described in the Supplementary Material (Section C.3.1), LOF operates on the model's intermediate representations to identify samples that are outliers. This is based on the hypothesis that instances from underperforming minority groups often manifest as representational outliers in a standard ERM-trained model.

2. Threshold Selection

   The activation threshold ($\tau$) is a key hyperparameter in our method, designed to balance the trade-off between accuracy and fairness. A lower threshold encourages more frequent bias correction, enhancing fairness, while a higher threshold tends to preserve overall accuracy.

   In our experiments, we set $\tau$ = 0.5 uniformly across all datasets to demonstrate the general applicability and robustness of our approach. Although $\tau$ can be tuned on a validation set (e.g., via grid search) to better suit specific deployment objectives, we found the default value of 0.5 to yield a balanced performance in practice.

   To further illustrate the impact of $\tau$, we conducted an ablation study on the CelebA dataset (in a supervised setting for clearer interpretation). As shown in Table A, varying $\tau$ results in different trade-offs between fairness and accuracy, highlighting its practical utility as a tunable parameter.

   Table A: Impact of activation threshold on FairNet’s performance (CelebA)

   Threshold	TPR (%)	FPR (%)	TPR/FPR	ACC (%)	WGA (%)	EOD (%)
   0.0	100.0	100.0	1.00	94.1	87.1	4.8
   0.2	98.8	18.7	5.28	95.4	86.9	5.1
   0.4	96.7	6.34	15.3	95.6	86.5	5.4
   0.5	94.1	3.45	27.2	95.9	86.2	5.8
   0.6	72.3	2.60	27.8	95.9	85.7	6.1
   0.8	62.9	1.48	42.5	96.0	82.1	7.4
   1.0	0.0	0.0	--	95.8	77.9	10.6

   TPR and FPR refer to the detector’s performance on the minority group. Best values in each column are bolded.

3. Resulting TPR/FPR on Held-Out Data

   For the LOF-based unsupervised detector, we evaluated its performance on the held-out test set for CelebA. The resulting metrics are summarized in the following Table B.

   Table B: Performance of FairNet-Unlabeled (extension to Table 5)

   	TPR (%)	FPR (%)	TPR/FPR	ACC (%)	WGA (%)	EOD (%)
   FairNet-Unlabeled	74.2	7.71	9.62	95.8	81.9	7.5
   ERM	-	-	-	95.8	77.9	10.6

   TPR and FPR refer to the detector’s performance on the minority group.

   These results demonstrate that FairNet-Unlabeled can effectively improve fairness outcomes even without labeled sensitive attributes, highlighting the promise of unsupervised bias detection in real-world scenarios.

Response to W1:

We appreciate your observation, and you are absolutely right that our method introduces additional components during inference—specifically, a bias detector and a LoRA-based correction module—which lead to some increase in computational cost.

Table A: Computational overhead analysis of FairNet. Parameter counts are reported in millions (M), and floating-point operations (FLOPs) are in Giga-FLOPs (GFLOPs).

To address this concern, we performed a detailed analysis to quantify this overhead. As shown in the table A, the GFLOPs for minority and majority groups differ slightly because samples flagged as minority by the detector activate a conditional LoRA module, which adds a small computational cost due to the altered execution path.

However, since the proportion of minority samples is typically low, the overall impact on inference efficiency remains minimal. For example, in the case of the BERT-base model, FairNet increases the parameter count by only 0.28%, and the total GFLOPs by just 3.7% (from 10.88 to 11.29 GFLOPs). A similar level of overhead is observed for ViT.

These results suggest that the computational overhead introduced by FairNet remains minimal, supporting its practicality for real-world deployment, including latency-sensitive applications.

Response to Q1

Thank you for raising this important concern about test-time efficiency when using multiple detectors. We agree that computational overhead is a critical consideration for real-world deployment, and we appreciate the opportunity to clarify how FairNet addresses this.

First, the design of our bias detectors is explicitly optimized for efficiency. As noted in Section 3.2, each detector $D_{\phi}^{(l)}$ is a lightweight module—typically a small MLP preceded by attention pooling on intermediate representations $h^{(l)}(x)$. This ensures that even with multiple detectors (e.g., for different sensitive attributes), the additional computational cost remains minimal. The attention pooling operation, which aggregates hidden states into a fixed-size vector, is designed to be computationally inexpensive, and the MLP layers use small hidden dimensions to avoid bottlenecks.

Second, the conditional activation of LoRA modules further mitigates efficiency concerns. Even with multiple detectors, LoRA adjustments are only applied when triggered by a high risk score, meaning that the majority of samples (i.e., unbiased instances) incur no additional LoRA computation beyond the base model. This selective correction ensures that the overall inference speed remains dominated by the base model’s runtime, with minimal overhead from detectors.

Planned Revision. To clarify this aspect, we will include a new subsection in the supplementary material titled "Computational Complexity Analysis". This section will provide quantitative results. As shown in the table A presented in 'Response to W1' (Table A: Computational overhead analysis of FairNet), FairNet adds only 0.67% and 0.28% to the total parameter count for ViT and BERT respectively. The total GFLOPs increase by just 8.2% (ViT) and 3.7% (BERT), confirming that FairNet maintains high efficiency and is well-suited for scalable deployment.

Response to W1:

Thank you for this insightful comment. We agree that validating theoretical guarantees with systematic error bars is essential for connecting theory to practical performance.

To address this, we have updated Supplementary Table 5 to include standard deviations across five runs with different random seeds. The results confirm that key metrics—TPR, FPR, and the TPR/FPR ratio—remain stable within a narrow range, even under limited labeled data. This supports the practical reliability of the theoretical “moderate TPR/FPR” condition. （Due to space/word constraints, only a selection of results is presented here.）

Additionally, we tested FairNet's robustness under label noise to simulate degraded detector performance. As shown below, FairNet maintains performance above the ERM baseline even when the detector fails entirely, confirming its robustness and safety guarantees.

Response to W2:

Thank you for the thoughtful comment. We acknowledge that our experiments use mid-scale datasets and appreciate the opportunity to clarify this choice.

CelebA, MultiNLI, and HateXplain are widely-used benchmarks in fairness research, selected for their well-characterized biases. Their moderate size allowed us to perform controlled ablation studies (Table 2) and isolate the effects of individual components, which was essential for validating FairNet’s design.

Importantly, FairNet is built for scalability. Both the bias detectors and LoRA modules are lightweight and parameter-efficient, ensuring low computational overhead. We agree that testing FairNet on large-scale datasets is a valuable direction for future work.

Response to W3:

Thank you for highlighting this important issue. While intersectional bias is noted as future work in our conclusion, we do include a foundational experiment addressing multiple sensitive attributes simultaneously.

As shown in Supplementary Section D.1 and Table 4 (p. 21), we evaluate FairNet on the HateXplain dataset considering both race ("African American") and gender ("Female"). FairNet reduced the EOD for race from 20.1 to 9.8 and for gender from 12.9 to 8.2, without harming overall accuracy. This demonstrates FairNet’s potential in handling multi-attribute fairness and lays groundwork for future exploration of intersectional bias.

Response to W4:

Thank you for raising this important concern about test-time efficiency.

FairNet is designed to remain efficient, even when handling multiple biases. Users can either deploy separate lightweight detectors or use a single multi-label detector, with additional biases handled by adding LoRA modules—without a linear increase in computation.

Moreover, the detectors operate on intermediate representations and consist of only an attention pooling layer and a small MLP, keeping their overhead minimal.

In practice, applying FairNet to BERT-base increases the total parameter count by only 0.28% and test-time GFLOPs by just 3.7%. These small increments ensure that test-time performance remains largely unaffected, even in multi-attribute settings.

Response to W5:

Thank you for the helpful suggestion. We initially focused the ablation study on CelebA due to its clearly defined attributes, which made it suitable for isolating the effects of each component.

To further demonstrate the generalizability of FairNet, we have now conducted a full ablation study on the MultiNLI dataset. As shown below, the results are consistent with those on CelebA, reinforcing the effectiveness and modularity of our framework.

These findings confirm that the individual components of FairNet contribute meaningfully across different datasets.

Response to Q1:

Thank you for the insightful question. Yes, the detector threshold (τ) is dataset-dependent, but it is designed to be easily tunable. As shown below, varying τ controls the trade-off between fairness and accuracy: lower thresholds lead to more frequent bias correction (improving fairness), while higher thresholds prioritize accuracy.

In our experiments, we uniformly set τ = 0.5 for all datasets to achieve a balanced trade-off, and found this setting to be robust in practice. The threshold can be tuned via grid search on a validation set with minimal overhead, allowing practitioners to align FairNet with different deployment goals.

Response to Q2:

Thank you for this excellent question. You are right—FairNet introduces some test-time overhead, and we have quantified this in detail.

For BERT-base, FairNet adds only 0.28% more parameters and increases test-time GFLOPs by just 3.7%. ViT shows similarly small overhead. This confirms that FairNet remains highly efficient and suitable for real-world, latency-sensitive deployments.

Response to Q3:

Thank you for the thoughtful question. Yes, FairNet can handle multiple sensitive attributes, and we have implemented this capability in our experiments.

On the HateXplain dataset, we applied FairNet to address both race ("African American") and gender ("Female") biases simultaneously. As detailed in Supplementary Section D.1 and Table 4 (p. 21), FairNet reduced the EOD for race from 20.1 to 9.8 and for gender from 12.9 to 8.2, with overall accuracy maintained (79.8% baseline vs. 79.7% with FairNet).

These results demonstrate that FairNet is indeed capable of mitigating fairness gaps across multiple attributes at once.

Response to Q4:

Thank you for this important question. FairNet-Unlabeled relies on pseudo-labels from LOF, and we find its performance to be stable and effective in practice.

Our results (Table 1, p. 8) show that on CelebA, FairNet-Unlabeled improves WGA from 77.9% to 82.3%, reduces EOD from 10.6% to 7.3%, while maintaining the same high accuracy (95.8%) as the ERM baseline—all without using any sensitive attribute labels.

To evaluate stability, we compared FairNet-Unlabeled to a supervised detector with noisy labels. Its performance is comparable to detectors trained with 60–80% label noise. Moreover, even under 100% noise (fully random pseudo-labels), FairNet does not fall below ERM baseline performance.

These findings confirm that FairNet-Unlabeled is not only effective, but also robust—even for outlier samples or noisy signals.

Response to Q5:

Thank you for this insightful question. While our study focuses on fairness under a fixed distribution, we agree that robustness to distribution shift is critical.

Although we did not explicitly test FairNet under distribution shift, its design may help in such scenarios. In particular:

• The contrastive loss promotes class-level representations that are invariant to sensitive attributes, reducing reliance on spurious correlations.
• The conditional LoRA module enables flexible adjustments without overfitting to specific data patterns.

These properties could contribute to better generalization under distribution shift, though we have not yet validated this empirically. We appreciate the suggestion and consider this an important direction for future work.