The Disparate Benefits of Deep Ensembles

We thank the reviewer for their critical assessment of our work and the insightful questions. We first address the main question, followed by responses to the specific concerns outlined in the review.

Main Question

We appreciate the request for clearer evidence linking predictive diversity to the disparate benefits effect. As you correctly noted, with real-world datasets we cannot directly control predictive diversity. This limits us in Fig 3 to comparing tasks where the disparate benefits effect is observed (top row) versus not observed (bottom row) as per the results in Tab 1.

However, believe the experiment on synthetic data reported in Apx F.1. aligns with your suggestion. There, we systematically varied the level of predictive diversity using a parameter $\alpha$, while keeping all other factors constant. As shown in Fig 16 and Tab 19, the disparate benefits effect increases with higher diversity and vanishes entirely when $\alpha = 0$. Importantly, this setup uses equal base rates ($p(Y=y, A=a) = 0.25$ for all combinations), ruling out base rate differences as the driver of the observed disparities. Based on your feedback, we will move this experiment into the main paper in the camera-ready version to strengthen our core argument.

Comments

Base rates

We provide the base rates in the legends of Fig 3 (see Fig 13-15 for all tasks), as well as Fig 4c for the synthetic experiment. Notably, there are some tasks (e.g. FF Y=age, A=gender) with very balanced base rates exhibiting significant levels of disparate benefits (see Tab 1). Also, we designed the synthetic task with equal base rates to eliminate this possible confounding factor.

Fairness/accuracy tradeoff

We appreciate the concern about a potential fairness/accuracy tradeoff. However, we do not believe the disparate benefits effect merely reflects such a tradeoff. In some tasks (Fig 7 and 8), both accuracy and fairness improve through ensembling. Moreover, for tasks where fairness declines, we find that applying Hardt Post-processing (HPP) to the Deep Ensemble (DE) restores fairness to the level of individual models, without sacrificing the ensemble’s accuracy gains (Fig 6, red dot directly above the dotted line). This suggests the effect is not intrinsically an accuracy/fairness tradeoff.

Single model

Thank you very much for your observation. We inspected the average predictive diversity per target and group for all 5 model variants and obtained similar results to those reported in Fig 3. Based on your feedback, we will add a new set of Figures in the revised version of the paper, summarizing these results.

Hardt post-processing

We agree to the point that DEs are not different from any other probabilistic classifier regarding HPP. In essence, HPP takes any predictive distribution and group attribute as input independent of how this distribution was obtained. However, we believe that our finding regarding the improved calibration of the ensemble and thus its heightened sensitivity to the threshold is an interesting and novel finding.

Fig 6

We thank the reviewer for pointing out the lack of clarity regarding the two red dots in Fig 6, which correspond to different levels of fairness tolerance to be achieved by the post-processing of the DE.

• The first level of fairness tolerance is indicated by the dotted line and corresponds to the average fairness of individual ensemble members (gray dot next to the dotted line). The red dot closest to the dotted line corresponds to applying HPP to the DE to achieve the same level of fairness as the individual members (dotted line). Note how this is achieved while increasing accuracy. Thus, it illustrates that the DE remains Pareto-dominant by increasing both fairness and accuracy.
• The second level of fairness tolerance is marked by the dashed line and it corresponds to 0.05, which is a commonly used value in the literature. In this case, the DE is also Pareto-dominant when compared to the individual models after post-processing both to the same tolerance level.

We acknowledge that Fig 6 contains a lot of information and will revise its caption and description for clarity.

We believe these revisions and clarifications directly address the reviewer’s central concerns—particularly regarding predictive diversity, base rate confounding, and the distinct fairness behavior of ensembles. We hope this supports a more favorable reassessment of our work.

We thank the reviewer for their very positive assessment of our work. We are pleased that they consider our experiments to be well designed, described and sufficient to support our claim that ensembling can have a negative impact on fairness.

Questions

Q1

We acknowledge that the sample size (5 ensembles per task) is relatively small, primarily due to the computational cost of training large, fully independent ensembles (e.g., 5 seeds × 10 models = 50 models, totaling ~3000 GPU hours for our experiments). This way we can assure independence of individual samples. To give more insight into the normality condition, we performed the Shapiro-Wilk test. While we acknowledge that normality is difficult to verify with small samples, the results generally support the use of the t-test. Noteworthy, there is one case rejecting normality at p < 0.05: SPD on CX (age). We will make the respective entry in Tab 1 non-bold and provide the following additional table in the final version of the paper.

Shapiro-Wilk test p-value

We also considered increasing the sample size by combining members across seeds, but this would introduce dependencies between ensemble members, which we believe would compromise the statistical validity more than the limited sample size does. That said, we are open to alternative suggestions for robust significance testing if the reviewer has a preferred method in mind.

Q2

Yes, we selected according to Tab 1 as the disparate benefits effect is not directly visible in Fig 3. The purpose of Fig 3 is to illustrate how the disparate benefits effect emerges when there are differences in the predictive diversity between groups (top row vs bottom row in Fig 3). These differences are highlighted by the black arrows in the figure, showing greater disparities in predictive diversity precisely in the tasks where we have found disparate benefits as per Tab 1.

For clarity, as explained in the caption of Fig 3, the top and bottom rows show examples of datasets where disparate benefits are and are not present, respectively. In the top row, we observe significant differences in predictive diversity, while in the bottom row, there are no or residual differences. This contrast demonstrates how predictive diversity differences correlate with the presence or absence of the disparate benefits effect. We will revise the caption and the accompanying text to clearly explain the distinction between the effect itself (shown in Tab 1) and its explanation (shown in Fig 3).

Remarks

Causation between predictive diversity and disparate benefits

We fully agree that causality is very hard to establish. The real world experiments are observational in nature. Therefore, we conducted the interventional synthetic experiment. Note that we provide an extension of the experiment in Apx F.1., where we change the feature diversity per group (the $\alpha$ parameter). There we see strong correlations between the differences in predictive diversity and the disparate benefits effect (see Tab 19 and Fig 16).

Definition of "disparate benefits effect"

Thank you very much for this feedback. We agree with your assessment that the definition of the disparate benefits effect in the abstract is the best one and will adapt the description in the introduction accordingly, providing a more formal definition.

Effect size

The disparate benefits effect depends on the unfairness of the base models. For example, the absolute increase shown in Table 1 of SPD in 2.2% for FF (age/gender) corresponds to a 10% relative increase in unfairness. Furthermore, the change in unfairness is closely linked to the increase in accuracy due to ensembling, which is in the range of 1-3% on our considered tasks. Therefore, the absolute effect size of the disparate benefits can not be expected to be substantially larger than the effect of ensembling itself.

Scatter plot

Thank you for this suggestion, we will provide such plots in the final version of the paper.

We thank the reviewer again for this thoughtful and constructive feedback. We hope to have properly addressed the questions and concerns.

We sincerely thank the reviewer for engaging deeply with our work and for providing their thoughtful and positive assessment. We’re pleased that you found the experimental design sound, the analyses sufficiently elaborate, and the results well-supported. Your detailed comments helped us reflect on both the limitations and opportunities for improvement in the framing and positioning of our work. We would like to respond in more detail to two specific comments:

The paper obscures quite some information with regards to fairness by only considering binary sensitive attributes (numerical and categorical attributes are mapped to be binary). This limits the scope of the analysis significantly and this choice could be communicated at an earlier stage of the paper.

We fully agree that explicitly stating the scope of the fairness setting—particularly the use of binary sensitive attributes and labels—is crucial for readers to assess our findings. We chose to focus on the canonical setting of binary labels and attributes to make our findings easily accessible and interpretable. However, exploring non-binary labels and group attributes (also possibly intersectional) is indeed an important direction for future work.

While we currently state our setting in the opening paragraphs of Sections 3 (formal setup) and 4 (dataset description), we acknowledge that this may come too late for some readers. To make this clearer upfront, we propose updating the final paragraph on the first page to state: “... each with multiple binary target variables and protected group attributes.” to ensure that readers are immediately aware of our setup before engaging with the technical content.

Relation To Broader Scientific Literature

Thank you for highlighting several important and relevant works on fairness in ensemble learning. While our Related Work section focused primarily on studies most closely related to the disparate benefit phenomenon—such as Ko et al. (2023)—we fully agree that placing our contribution within the broader literature on ensembles and fairness will strengthen the paper.

In the revised version, we will expand the Related Work section to include the studies you suggested (Gohar et al., 2023; Kenfack et al., 2021; Mishra & Kumar, 2023; Tayebi & Garibay, 2023) as well as additional relevant works (e.g., Grgić-Hlača et al., Bhaskaruni et al.). For example:

• Tayebi & Garibay apply pre-processing interventions to ensemble members of a Deep Ensemble (on tabular data), offering a complementary perspective to our work.
• Gohar et al. investigate multiple research questions around the fairness impact of different ensembling techniques for classical models (e.g. SVMs, logistic regression, naive bayes), extending the discussion beyond deep learning.

We believe these additions will further clarify our paper’s relationship to the broader fairness literature and underscore its specific contributions.

Once again, we sincerely thank the reviewer for their valuable feedback. We appreciate your support of our work and are committed to addressing these points to improve the clarity and contextualization of our contribution in the final version. Please don’t hesitate to suggest any additional revisions that might further strengthen the paper.

---

Grgić-Hlača, N., Zafar, M. B., Gummadi, K. P., & Weller, A. (2017). On fairness, diversity and randomness in algorithmic decision making. arXiv:1706.10208.

Bhaskaruni, D., Hu, H., & Lan, C. (2019, November). Improving prediction fairness via model ensemble. In 2019 IEEE 31st International conference on tools with artificial intelligence (ICTAI) (pp. 1810-1814).

We thank the reviewer for their thorough assessment of our work. We are pleased that it was found to be well written and to offer valuable insights. In response to the questions raised:

Limited generalizability of findings due to focus on vision datasets

We agree with your observation and acknowledge this limitation in the final paragraph of the paper: “The main limitations of our study are that we focus on vision tasks, and hence on ensembles of convolutional DNNs. [...]”. That said, we aim to partially mitigate this limitation by including a diverse range of vision tasks spanning facial analysis (including under distribution shift) and medical imaging. These are application domains where Deep Neural Networks and Deep Ensembles are commonly deployed, making them highly relevant for studying their fairness implications. Moreover, we note that Deep Ensembles are rarely used in the tabular settings often featured in fairness research, where tree-based models remain dominant (Shwartz-Ziv, Ravid, and Amitai Armon. "Tabular data: Deep learning is not all you need." Information Fusion 81 (2022): 84-90). This context motivated our focus on vision data, where Deep Ensembles are widely considered.

The paper does not introduce its own fairness algorithm

We agree that our paper does not propose a new fairness algorithm—this was not our objective. The main contribution of our paper is to show the existence of the disparate benefits effect of Deep Ensembles (Section 5) and to investigate its underlying causes (Section 6). While we do not claim to introduce a novel fairness method, we believe that presenting this new challenge without discussing potential remedies would leave the analysis incomplete. Therefore, in Section 7, we explore ways to mitigate the negative consequences of disparate benefits. We focus on post-processing methods that can be applied on the already trained individual ensemble members for practicality. Specifically: (1) we show that weighting members non-uniformly does not improve the performance-fairness trade-off whereas (2) Hardt Post-processing (HPP) is very effective in improving fairness while maintaining the utility of Deep Ensembles; and (3) we provide insights into why this is the case (improved calibration / threshold sensitivity).

[Hardt Post-processing] is not specifically designed for ensembling. This also weakens the connection between the intuition in Section 6 and the mitigation strategy.

Indeed, HPP is not specifically designed to operate on ensembles. However, its model-agnostic nature allows it to operate on any predictive distribution, making it broadly applicable. Since Deep Ensembles produce predictions by averaging the output probabilities of individual models (Eq. 1), HPP can be applied in exactly the same way as with individual ensemble members.

Regarding “the intuition in Section 6”, we note that predictive diversity is the reason that ensembles predict differently than individual models - if there were zero predictive diversity, all models would agree and the ensemble would predict exactly the same as every ensemble member. The averaging of diverse, and sometimes conflicting, predictive distributions leads to improved calibration (as shown in Figure 5). We show that HPP can effectively leverage this property: Deep Ensembles are more sensitive to thresholding decisions than individual models, and HPP, which relies on optimizing group-specific thresholds under fairness constraints, is particularly well-suited to take advantage of this threshold sensitivity. Thus, the improved calibration and diversity in ensembles directly support the effectiveness of HPP as a mitigation strategy.

Comparison with various types of fairness algorithms

We agree that this is an important direction for future work. We explicitly acknowledge this in the final sentence of the paper: “Furthermore, we intend to investigate the disparate benefits effect for Deep Ensembles where pre- or in-processing fairness methods have been applied to individual ensemble members.” Given the wide variety of fairness interventions available, we believe this direction warrants a dedicated study. Expanding the scope further within the 8-page limit would have come at the expense of depth in our current analysis.

We thank the reviewer again for their valuable feedback and thoughtful remarks. We hope that our rebuttal provides a more complete perspective on the contributions and scope of our study, and kindly ask the reviewer to consider a reassessment of their evaluation in light of these points.