Size-adaptive Hypothesis Testing for Fairness

Originality and Significance for NeurIPS

We thank the Reviewer for clearly acknowledging the clarity and the technical quality of our work. On the other hand, Reviewer ujf4 believes that Significance and Originality of our work are low and do not reach the bar for NeurIPS. We respectfully disagree with this assessment and in the following we discuss why our contribution does not amount only to repackaging standard statistics. Classical tools such as the multivariate CLT and the Delta method are indeed decades old; however, their application, calibration, and validation for high-resolution fairness auditing is not.

To the best of our knowledge, no existing work provides a standardized, statistically calibrated test for fairness violations that adapts to subgroup size and gives both asymptotic and finite-sample guarantees. Current methods (δ-SP, γ-SP, etc.) rely on fixed thresholds with no error control. Our proposal fills this methodological gap and is designed to become a standard testing protocol for fairness auditing, analogous to how χ² or t‑tests are used in classical statistics. We would like to point out that both Theorem 1 and 2 follow from a smart use of combined CLT, Delta Method and Slutsky theorem, and the simple formulation follow from a refined choice of combining probabilities of disjoint events from which we observe that many practically used fairness metrics are derivable and which allows for an easy computation of the covariance matrix.

Moreover, the Reviewer writes:

‘’Since the basic setting of demographic parity is mastered, one obvious way to improve is to study a more ambitious problem, such as conditional or causal variants of parity, which would likely provide more significant theoretical and methodological challenges.’’

Statistical parity is not “mastered” in real deployments; regulators and practitioners still rely on threshold choices and point estimates. We chose it as a canonical case to establish the statistical foundations. However, our framework is metric- and model- agnostic and readily extends to other fairness metrics. Conditional or causal notions are indeed exciting directions, and our resolution-limit analysis provides the very diagnostic tool needed before estimating causal effects in sparse conditions. Specifically, Theorem 2 already generalizes to the mentioned conditional variants.

NeurIPS values contributions that close the gap between statistical theory and machine learning practice in societally critical settings. By unifying analytic variance, Bayesian calibration, and a decision-theoretic resolution limit, then demonstrating their impact on real datasets, we deliver a rigorously grounded, practically deployable method that the fairness-in-ML community still lacks. In these regards, the scores given by the Reviewer to Originality and Significance, seem not to be sustained.

Finally, the Reviewer gave a general rating of 2: Reject [For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations], which does not appear to be adequately sustained by the arguments in the review. Our work offers a rigorously grounded, ready-to-use tool that addresses an urgent need in responsible AI practice, and thus meets the bar for significance and relevance at NeurIPS.

Regarding uncertainty in the constructed classifier $f$

We thank the reviewer for raising this important point. In our framework, the classifier $f$ is indeed derived from data, but it is treated as a fixed, given object at the moment of fairness auditing. This assumption reflects a common operational scenario: fairness audits are typically conducted after model training, on a deployed or pre-trained classifier whose predictions are to be evaluated for fairness.

Our methodology focuses on quantifying the statistical uncertainty in fairness metrics, arising from the finite sample size of the evaluation data, especially in small or rare subgroups. The uncertainty we account for is thus epistemic, stemming from sampling variability rather than model uncertainty.

That said, we agree that in settings where the classifier outputs probabilistic predictions, it would be possible and potentially useful to incorporate model uncertainty into the fairness analysis. For instance, one could assess fairness over distributions of predicted probabilities, or incorporate uncertainty from stochastic or Bayesian classifiers into the fairness tests. This is a promising direction, and one that complements our work: our current methodology can serve as a principled post hoc test of fairness, while future extensions could integrate upstream uncertainty in the predictions of $f$ itself.

In summary, our approach does not require the classifier to express uncertainty, but is compatible with extensions that incorporate such uncertainty when available. We will integrate this discussion in the manuscript and we plan to explore it in future work.

Bootstrap intervals: why they fall short in sparse intersectional settings

Thank you for raising this important point. We experimented extensively with non-paramentric bootstrap intervals prior to the submission and decided not to include them because, in our settings, bootstrap often fails to offer any advantages over either the frequentist or the Bayesian approaches. However, we agree that it would be better to include a discussion about this point in the paper anyway.

We highlight now some characteristics of the bootstrap approach. First, in moderate to large subgroups, bootstrap intervals converge both theoretically and empirically to the same Wald limits delivered by our analytic CLT. At that scale, bootstrap therefore adds only computational overhead without any improvement. In the small $n_s$ intersectional slices where analytic Wald intervals break down, bootstrap inherits the same weaknesses. Resampling with replacement in class unbalanced scenarios frequently generates samples in which one of the four contingency cells is zero (e.g., no predicted positives for group $S$), producing degenerate or undefined estimates of $SP$. Moreover, the resulting empirical distribution is highly discrete and strongly skewed, which yields uncentered confidence intervals. For these reasons, we opted for the analytic CLT, whose validity can be proved, and for the Dirichlet–multinomial credible interval, which remains calibrated even when cell counts are in the single digits. Nonetheless, the camera‑ready version will include a dedicated appendix comparing our method with the bootstrap approach, documenting the pathologies above and clarifying the regimes in which bootstrap is a viable alternative.

We hope these clarifications address the Reviewer’s concerns. We would be grateful to engage the Reviewer in a fruitful discussion and to provide further details if needed.

Thank you for the thoughtful and constructive feedback. Below we address each weakness in detail and describe the revisions we will incorporate to strengthen the paper.

W1: To the best of our knowledge, in the fairness literature, there is no universally accepted analogue of the χ², t-, or log-rank tests that offers to practitioners a standardized, statistically calibrated test for detecting fairness violations that adjusts to subgroup size while providing both asymptotic and finite-sample guarantees. What exists instead are score heuristics such as δ-SP or γ-SP whose numerical thresholds (±0.05, ±0.10, etc.) are chosen ad-hoc and carry no type-I/type-II guarantees. Regulators, standards bodies, and applied ML teams have repeatedly called for principled, statistically grounded decision rules that can withstand legal and scientific scrutiny.

Our method addresses this gap and is intended to serve as a standard protocol for fairness auditing, in the same way that χ² or t‑tests function in classical statistics. We note that Theorems 1 and 2 result from an effective combination of the Central Limit Theorem, the Delta Method, and Slutsky’s theorem, while the concise formulation arises from a careful way of combining probabilities of disjoint events. This not only shows that many commonly used fairness metrics can be derived within our framework but also enables straightforward computation of the covariance matrix.

W2: We thank the Reviewer for pointing out the need for more clarity on how small subgroup sizes affect our first test (the Wald test).

The concern raised is that small subgroup sizes might inflate the variance estimate, leading to conservative behavior. However, under the asymptotic regime in Theorem 1, this is not the case. The Wald variance estimate for subgroup $S$ can be rewritten as:

$\hat{\sigma}^2 = \frac{p̂_S (1 - p̂_S)}{n_S}$

+ \frac{p̂_\bar{S} (1 - p̂_\bar{S})}{n_{\bar{S}}}.

Since $n_S \approx n \cdot P(S)$, the variance of the estimator decreases at rate $1/n$, even for rare groups. Both asymptotically and for small samples, small subgroups do not lead to inflated variance.

That said, the more subtle issue is that in finite samples, the plug‑in variance $\hat{\sigma}^2$ may actually underestimate uncertainty when $n_S$ is small. In particular, when the estimated subgroup rate $\hat{p}_S$ is very close to 0 or 1, the term $\hat{p}_S (1 - \hat{p}_S)$ in the variance formula becomes near zero. This drives the Wald variance estimate to be artificially small, even though the underlying sampling uncertainty is high for such extreme proportions with small $n_S$.

This can make the Wald test anti‑conservative, falsely rejecting the null due to high variance in small groups not captured by the asymptotic approximation.

This is precisely why we introduce the second test: a Bayesian Dirichlet–multinomial method that provides calibrated inference in the small-sample regime. We will clarify this transition in the paper: the problem is not variance inflation, but undercoverage due to inappropriate reliance on asymptotics when subgroup counts are small.

W3: We have explored the impact of the prior choice and the incorporation of prior information (derived from the training set) and the revision will include a full analysis contrasting:

• Uniform (flat) Dirichlet priors, which impose no substantive prior information;
• Empirical‑Bayes priors derived from training‑set frequencies. For each subgroup we set

where $c_{i,j}$ is the number of training samples with label $i\in\{0,1\}$ in group $j\in\{s,\bar s\}$.

This construction scales the prior strength by $\lambda$ while preserving the empirical class proportions, and caps the influence when any cell count is extremely small.

The following table shows the preliminary results with the confidence intervals (CI) for some selected subgroups from the Adult dataset:

This preliminary sensitivity analysis confirms prior influence for extremely small subgroups, and that all methods rapidly converge to the asymptotic regime as $n_S$ grows. For the smaller $n_S$ case, the Wald intervals are spuriously narrow, clearly underrepresenting the true sampling uncertainty. In contrast, the uniform-prior Bayesian credible intervals are much larger, appropriately reflecting the scarcity of data. As we introduce empirical-Bayes priors, these intervals shrink toward the Wald width but remain more conservative than asymptotic bounds, demonstrating that modest data-informed priors can stabilize inference without overconfidence. With hundreds or thousands of samples, all methods produce nearly identical intervals, illustrating that prior effects vanish once more data is available.

W4: Thank you for raising this important point. We experimented extensively with non-parametric bootstrap intervals prior to the submission, and decided not to include them because, in our settings, bootstrap often fails to offer any advantages over either the frequentist or the Bayesian approaches. However, we agree that it would be better to include a discussion about this point in the paper anyway. We highlight now some characteristics of the bootstrap approach. First, in moderate to large subgroups, bootstrap intervals converge both theoretically and empirically to the same Wald limits delivered by our analytic CLT. At that scale, bootstrap therefore adds only computational overhead without any improvement. In the small $n_s$ intersectional slices where analytic Wald intervals break down, bootstrap inherits the same weaknesses. Resampling with replacement in class unbalanced scenarios frequently generates samples in which one of the four contingency cells is zero (e.g., no predicted positives for group $S$), producing degenerate or undefined estimates of $SP$. Moreover, the resulting empirical distribution is highly discrete and strongly skewed, which yields uncentered confidence intervals.

In the last row of the table reported in W3, we display the Bootstrapping intervals obtained. One can observe how similar they are to the asymptotic confidence interval, even in the small sample regime, highlighting their inadequate estimation (uncentered with small variance) in such cases. For these reasons, we opted for the analytic CLT, whose validity can be proved, and for the Dirichlet–multinomial credible interval, which remains calibrated even when cell counts are in the single digits. Nonetheless, the camera‑ready version will include a dedicated appendix comparing our method with the bootstrap approach, documenting the pathologies above, and clarifying the regimes in which bootstrap is a viable alternative.

Concerning the lack of experiments with additional metrics, we have already highlighted it as a limitation. We explained how the framework can be used to compute confidence intervals for some further metrics e.g. Equal Opportunity, so no extra theory is required; the work is purely computational. Running a full set of new experiments for additional metrics (with several datasets and subgroup configurations) is not realistic within the short rebuttal window, but can be easily done and we plan to do it for a possible camera‑ready version, placing the resulting figures and tables in the appendix. That addition will demonstrate the size‑adaptive behaviour under an additional metric while keeping the main claims unchanged.

We hope these clarifications address the Reviewer’s concerns. Our revisions will add the requested variance discussion, prior-sensitivity experiments, bootstrap comparison, and example extensions to other metrics. We believe these additions highlight the paper’s practical value and methodological novelty. We welcome any follow-up questions and would be happy to provide further details or clarifications.

Dear Reviewer,

We thank you again for your questions, which will allow us to improve the paper.
With respect to the variance expression, the Reviewer is correct, $p̂_S$ indicates $P(f(x)=1|x\in S(\mathcal{X}))$, to be consistent with the notation of the paper,

$p̂_S$

and $p̂_{\bar S}$

should be replaced respectively with

$p̂_{1, S}/(p̂_{1, S}+p̂_{0, S})$

and $p̂_{1, \bar S}/(p̂_{1, \bar S}+p̂_{0, \bar S})$.

We appreciate you catching this inconsistency and are glad our previous response addressed your concerns.

W1/Q1 We thank the Reviewer for the feedback. We already identify the lack of conditional‑parity experiments as a limitation. We already explained how the framework can be used to compute confidence intervals for Equal Opportunity and the extention to Equalized Odds is straightforward, so no extra theory is required; the work is purely computational. Running a full set of new experiments for Equal Opportunity and/or Equalized Odds (with several datasets and subgroup configurations) is not realistic within the short rebuttal window, but can be easily done in time for the camera‑ready version, so that the resulting figures and tables can be placed in an appendix. That addition will demonstrate the size‑adaptive behaviour under a conditional metric while keeping the main claims unchanged.

W2/Q2 The specific cut‑off is a pragmatic choice. Thirty observations is the rule‑of‑thumb widely used to trust CLT‑based Wald tests because coverage and power stabilise around that cell size in typical multinomial settings. Users may raise or lower $\tilde n$ if their data warrant it; the method itself remains unchanged. While an ablation study on the sensitivity of this minimum support parameter is not doable in this short timeframe, we can absolutely launch it and collect/report the results in time for the camera ready version of the paper.

W3/Q3 To address the reviewer’s request, we performed a simulation study using Gaussian distributions.
The true Statistical Parity (SP) value (computed from the CDF of the normal) is covered in about 95 % of cases by the Bayesian posterior credible interval. Coverage is insensitive to sample size, but the interval narrows as more data are used.

We next describe the setup in detail. We consider two groups, $S$ and $S'$, drawn from different normals:

We define a simple classifier:

$f(x)= 1$ if $x > 0.5;$ $f(x)= 0$ otherwise.

For each repetition we sample $n$ points for $S$ and $m$ for $S'$, compute the credible interval for SP, and check if it contains the true value. We repeat this 1000 times. The true statistical parity value can be computed analytically from the cumulative density function of the Gaussian distributions and it is SP(S)=0.383.

The credible interval hits its nominal 95% coverage for any $n,m$. As $n,m$ grow, the interval tightens around the true SP. With $n=m\in\{5,10,30,100,1000\}$, the last‑run intervals are:

Bayesian credible intervals for different sample sizes; the last column is the empirical probability (over 1000 runs) that the interval contains 0.

Larger samples quickly force the interval away from 0, enabling confident detection of fairness violations.

We would be grateful to engage the Reviewer in a fruitful discussion and to provide further details if needed.

We thank the Reviewer for their positive scores, their in-depth understanding of our work, and their appropriate comments, which will be useful to improve the presentation of our work. We next answer the reviewer's questions.

Questions:

Q1: [...] many notions of fairness [...]

Unfortunately, the cited tutorial lists definitions only in prose and offers no mathematical formulation (see slide 2 "Limitations of this tutorial"), and we couldn't find supporting documents for the tutorial. To answer the Reviewer's question about which definitions can be modeled by our famework, we refer to the paper 'Fairness definition explained' [Verma and Rubin, 2018]. Our framework can model the first 13 definitions of table 1, while the last 7 are excluded. Essentially, our framework can model all those based on the confusion matrix or related conditional probabilities.

Q2: [...] many fairness measures have "pitfalls" with respect to intersectionality [...]

Precisely. Because most metrics were not designed for sparse, intersectional subgroups, fixed‑threshold tests often suffer high false‑positive and false‑negative rates. Our contribution is to quantify when and how severe these failures become, and to provide size‑adaptive tests that remain reliable in such settings. Furthermore, prior fairness metrics based on thresholds do not have Type-I/II error guarantees.

Q3: [...] or whether we can rigorously guarantee fairness?

Thank you for the clarification. We agree that “can” is more appropriate, given that our paper focuses on testing for fairness. We will adopt this wording in the revision.

Q4: [...] would allow for a fixed threshold.

A linear adjustment to γ‑SP has several limitations. There is no principled way to choose its slope or intercept, it comes with no statistical guarantees, and, as Figure 4 shows, even the best‑tuned linear correction would still misclassify points that our method separates correctly.

Q5: [...] Theorem 2 showing asymptotic normality [...]

The reviewer is correct: Theorem 2 is obtained by applying the multivariate CLT, the delta method and Slutsky’s theorem. One interesting technical choice was to express the fairness metrics as functions of probabilities of disjoint events. This keeps the covariance matrix analytic and easily computable; without that disjointness the algebra would be considerably heavier. The novelty is therefore not the asymptotic argument itself but (i) showing that the resulting Wald test slots seamlessly into our size‑adaptive framework and (ii) pairing it with a calibrated Bayesian test that covers the small‑sample regime where the asymptotics are unreliable. We will add a short remark to the camera ready version of the manuscript to make this clearer.

Q6: Is it really "rigorous" to set thresholds [...]

In finite-sample inference, all frequentist tests based on sampling distributions are approximate unless the distribution is exactly known. For large groups, our Wald test is derived from a formal CLT with a plug‑in variance estimator. This is not an ad-hoc correction: under standard regularity conditions, the rejection region is calibrated to control type I error at level α as n→∞, which is the definition of an asymptotically valid test. By the multivariate CLT and delta method, we obtain a limiting normal distribution with an explicit variance estimator. This ensures that, as n grows, the rejection region converges to one with exact type I error α. In other words, the α‑level control is not heuristic but a formal consequence of the CLT under standard regularity conditions. Moreover, this is the same notion of “rigor” used in essentially all classical hypothesis tests (e.g., t‑tests, χ² tests, Wald tests in GLMs). Calling it approximate is correct in a finite-sample sense, but asymptotic tests are considered rigorous because their error properties are analytically characterized and converge to the nominal level. In contrast, fixed-threshold fairness audits offer no such convergence guarantee at all - their type I/type II error is uncontrolled even asymptotically.

Q7: [...] the impact of prior information/bias?

We have explored the impact of the prior choice and the incorporation of prior information (derived from the training set) and the revision will include a full analysis contrasting:

• Uniform (flat) Dirichlet priors, which impose no substantive prior information;
• Empirical‑Bayes priors derived from training‑set frequencies. For each subgroup we set

where $c_{i,j}$ is the number of training samples with label $i\in\{0,1\}$ in group $j\in\{s,\bar s\}$.

This construction scales the prior strength by $\lambda$ while preserving the empirical class proportions, and caps the influence when any cell count is extremely small.

The following table shows the preliminary results with the confidence intervals (CI) for some selected subgroups from the Adult dataset:

This preliminary sensitivity analysis confirms prior influence for extremely small subgroups, and that all methods rapidly converge to the asymptotic regime as $n_S$ grows. For the smaller $n_S$ case, the Wald intervals are spuriously narrow, clearly underrepresenting the true sampling uncertainty. In contrast, the uniform-prior Bayesian credible intervals are much larger, appropriately reflecting the scarcity of data. As we introduce empirical-Bayes priors, these intervals shrink toward the Wald width but remain more conservative than asymptotic bounds, demonstrating that modest data-informed priors can stabilize inference without overconfidence. With hundreds or thousands of samples, all methods produce nearly identical intervals, illustrating that prior effects vanish once more data is available.

Q8: I feel like the word "rigorous" [...]

The testing framework is rigorous; the specific cut‑off is a pragmatic choice. Thirty observations is the rule‑of‑thumb widely used to trust CLT‑based Wald tests because coverage and power stabilise around that cell size in typical multinomial settings. Users may raise or lower \tilde n if their data warrant it; the method itself remains unchanged. For the Bayesian method, we set K = 10.000. Ten‑thousand Dirichlet-multinomial samples keep Monte‑Carlo noise below 0.2 pp at the 5 % level while remaining computationally trivial. Any larger K only reduces that already‑negligible error; smaller K risks noticeable numerical variance. Also in this case, practitioners can calibrate K to their precision/speed trade‑off.

Q9: [...] why select alpha = 0.05? [...]

Our test is built exactly in the Neyman–Pearson framework: for every subgroup $S$, we define $H0:SP(S)=0$ and a rejection region calibrated to control type I error at level $\alpha$. The choice of $\alpha=0.05$ in the experiments is not a claim of optimality; it is a standard convention to pick one operating point on the type I/type II error curve for comparison with prior fairness audits. More importantly, the method is agnostic to the specific $\alpha$. For any $\alpha \in (0,1)$, the procedure yields a valid $α$‑level test (asymptotically for the Wald version, exactly for the Bayesian small-sample version). By sweeping α, one can trace out the full ROC‑like tradeoff between false fairness violations (type I) and missed unfairness (type II). This is the general Neyman–Pearson sense: we provide a test statistic with a calibrated null distribution, and practitioners or regulators can set α based on the cost of false positives vs. false negatives.

In short, $\alpha=0.05$ is a reporting choice, not a methodological constraint; the framework itself already encodes the Neyman–Pearson tradeoff and supports any α the user deems appropriate. We can make this explicit in the main text.

Furthermore, to strengthen the connection with the Neyman–Pearson lemma it should be possible to define a likelihood ratio test, which is the uniformly most powerful test in the set of level alpha tests. We will consider this extension in a possible camera ready version.

Q10: There are some other works on addressing intersectionality in ML + fairness [...]

We are grateful to the Reviewer for pointing out these related papers: we will add them to our literature review.

We welcome any follow‑up questions and would be happy to provide additional details or clarifications.