Assessing Uncertainty in Similarity Scoring: Performance & Fairness in Face Recognition

We would like to thank you for your feedback. Here are our answers to your concerns.

1. This paper dedicates excessive length and involves many complex formulas to introduce some basic knowledge of face recognition evaluation, which seems unnecessary.

In our opinion, the formulas introduced are necessary since they shed light onto the complexity of the structure of an empirical similarity ROC curve (and that of fairness metrics) in Face Recognition (FR), given an evaluation dataset, which determines the methodology to assess its variability/uncertainty.

For instance, Eq. (2-3) explain the U-statistic nature of the statistics involved in its computation and underlines the differences with the (simpler) related work of Bertail et al. (2008), which deals with basic i.i.d. averages in contrast to the present FR framework (see Section 3.1). In addition, Equations (8), (9) and (10) are at the heart of our method (the recentered bootstrap) and give an insight into the reasons why the recentering technique is necessary, underestimating the FRR metric otherwise (see Section 3.2).

All the notations presented in the paper are involved in our main results (Proposition 1 and Theorem 1) which prove the asymptotic validity of the recentered bootstrap.

Note that we chose to put the majority of formulas within the supplementary material in order to make the reading easier. For instance, the supplementary material contains the list of popular FR fairness metrics, as well as their empirical counterparts, their attribute-driven versions and their bootstrap versions.

2.1. While fairness in face recognition is a pressing issue, the contributions of this paper appear limited.

First, we present (and give intuitions about) a new bootstrap method, the recentered bootstrap, which can be applied to similarity scoring tasks. Without this recentering, the naive bootstrap of Bertail et al. (2008) significantly fails, as we show within the paper (e.g. Section 3.2). This new bootstrap method allows to compute proven valid confidence intervals for the ROC curve and popular FR fairness metrics (see Theorem 1). This is the first work to tackle this problem, which has significant impact in real-life applications.

Secondly, we highlight that relying on empirical ROC curves (and fairness metrics) solely to compare models is inaccurate: practitioners of Face Recognition (FR) know that the performance measure varies when the evaluation dataset is changed, even if it is drawn in the same population (partially observable unavoidably). Fig. 1 shows that ArcFace is better than CosFace on one dataset and worse than CosFace on the other dataset. Both datasets share the same identities but not the same images. If only one of these datasets is available, one would jump to an erroneous conclusion. The confidence band method we present in the paper avoids such conclusions as both models are indistinguishable in terms of performance.
As many recent FR papers [1,2,3,4] improve the state-of-the-art performance (ROC) by very slight margins on one/two specific evaluation datasets, it becomes necessary to question the uncertainty regarding the true performance of these models. One new architecture/loss function might obtain a slightly better empirical ROC curve than other papers, while this empirical ROC might be contained within our ROC confidence bands for those other papers. Incorporating the uncertainty band in future model comparisons would lead to trustworthy conclusions.

Thirdly, we chose to illustrate a model selection based on fairness in Figure 4. In this case, we employ a model deployment/production point of view. Face Recognition already has a considerable societal impact, while being biased against some subgroups of the population. In this sense, FR applications may have strict fairness constraints. For instance, one hypothetical legislation could be that the FRR metric for women should be lower than 4 times the FRR for men. In Figure 4 (zone B, e.g. FAR=6e-4), this constraint would forbid the ArcFace model (the FRR max-min fairness has confidence bands which can go higher than 4), i.e. the fairest model. Indeed, as for Fig. 1, one evaluation dataset might have this fairness metric equal to 4. Between AdaCos and ArcFace, one would then choose AdaCos, which is the model having the worst empirical fairness measure.

2.2. As the authors themselves note, typically, global ROC curves and their comparisons with specific attribute-driven ROC curves would suffice for the uncertainty analysis in most face recognition scenarios.

We think that comparing global ROC curves with specific attribute-driven ROC curves is a poor method of estimating the uncertainty of any algorithm as it depends on the choice/availability of those attribute labels within the evaluation dataset.

But more importantly, we have highlighted that relying on empirical ROC curves (and fairness metrics) solely to compare models is inaccurate: practitioners of FR know very well that the performance measure varies when the evaluation dataset is changed, even if it is drawn in the same population (partially observable unavoidably). Fig. 1 shows that ArcFace is better than CosFace on one dataset and worse than CosFace on the other dataset. Both datasets share the same identities but not the same images. If only one of these datasets is available, one would jump to an erroneous conclusion. The confidence band method we present in the paper avoids such conclusions as both models are indistinguishable in terms of performance.

In a similar manner than for Fig. 1, the ranking of the best attribute-driven ROC curves can be permuted when changing the evaluation dataset. Thus, no trustworthy conclusion can be made, when only looking at empirical attribute-driven ROC curves. The only proper way to compare attribute-driven ROC curves is to compute them, along their confidence bands.

Fourthly, a significant contribution we make is the comparison of FR fairness metrics depending on their uncertainty. The question of measuring fairness is complex and the NIST, responsible for fairness audit of academic/industrial FR algorithms, is still hesitant on which fairness metric one should use to compare models [5]. Besides quantifying the differentials in FAR/FRR (among subgroups), such fairness metrics should be robust and exhibit a low uncertainty. Indeed, with high-impact applications, a fairness audit should be as trustworthy as possible and fairness measures should not vary a lot depending on the evaluation dataset. In addition, the subgroups considered become smaller and smaller when considering intersections of attributes (e.g. black women of age 40-50) and evaluation datasets might not have lots of data for such specific subgroups. This results in uncertain measures for those subgroups, altering the robustness of fairness measures. It becomes necessary to design fairness measures which are as robust as possible. We compare popular FR fairness measures, used by the NIST, depending on their uncertainty in Fig. 5 and conclude that the max-geomean metric is the more adequate. This result is supported by similar findings in the supplementary material B.2 (when varying the FR models, the sensitive attribute and the evaluation dataset). Note that this uncertainty quantification will be useful when evaluating the soundness of future fairness metrics.

Lastly, we provide in the supplementary material B.1 the coverage of the bands obtained with the recentered bootstrap. This coverage metric is the gold standard to quantify the accuracy of any method which builds confidence bands. The ideal property of a confidence interval is that specifying its confidence level $1-\alpha_{CI}$ (also called the nominal coverage) should lead to an interval having truly a probability equal to $1-\alpha_{CI}$ to contain the true (unknown) quantity $\mathrm{ROC}(\alpha)$. The coverage measures what this probability truly is and it should be equal (or very close) to the specified nominal coverage (see B.1). From Fig. 6 and Table 1, we observe that, even with finite datasets, it is the case for the recentered bootstrap, which suggests that the latter provides an appropriate tool for assessing the uncertainty of the performance measure.

We added two baselines for the revised version of the paper, to compare with the rencentered bootstrap.

The first one is the naive bootstrap of Bertail et al. (2008). The non pairwise setup of the latter work significantly underestimates the ROC curve, leading to strongly inaccurate confidence intervals, as seen in Fig. 9 and Table 2.

As pointed out in the related works section of the paper, no method to quantify the uncertainty in measuring performance/fairness in FR is documented in the literature to the best of our knowledge. The one described and analyzed in this paper is the first one. As explained in Section 3.1, using the (Gaussian) asymptotic law of the empirical similarity ROC curve is not an option given the very complex form of its covariance structure (involving probability density functions). That being said, we designed a more meaningful baseline than the naive bootstrap of Bertail et al. (2008). The sole reasonable alternative is to use the bootstrap technique we propose to evaluate the variance of the empirical ROC at a certain FAR level and build a Gaussian confidence interval based on it (see supplementary material B.1). As now shown in the revised version (see Fig. 9 and Table 2), the coverage of this Gaussian approximation is much more accurate than the naive bootstrap, but less satisfactory than our recentered bootstrap.

The approach we propose is easy to use and proven valid, theoretically and empirically (very satisfactory coverage). It avoids the bias that would be caused by a naive implementation of the bootstrap technique and has the potential to be widely used by practitioners, insofar as absolutely no alternative is available today.

[1] Huang, Yuge, et al. "Curricularface: Adaptive curriculum learning loss for deep face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

[2] Meng, Qiang, et al. "Magface: A universal representation for face recognition and quality assessment." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.

[3] Boutros, Fadi, et al. "Elasticface: Elastic margin loss for deep face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[4] Kim, Minchul, Anil K. Jain, and Xiaoming Liu. "Adaface: Quality adaptive margin for face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[5] Grother, Patrick. "Face Recognition Vendor Test (FRVT) Part 8: Summarizing Demographic Differentials." (2022).

We would like to thank you for your careful reading of the manuscript and your feedback. Here are our answers to your concerns.

2.1. The experiments demonstrate the scenarios and ways in which the method proposed in this paper can be applied, but it is not sufficient to illustrate its effectiveness

First, we highlight that relying on empirical ROC curves (and fairness metrics) solely to compare models is inaccurate: practitioners of Face Recognition (FR) know that the performance measure varies when the evaluation dataset is changed, even if it is drawn in the same population (partially observable unavoidably). Fig. 1 shows that ArcFace is better than CosFace on one dataset and worse than CosFace on the other dataset. Both datasets share the same identities but not the same images. If only one of these datasets is available, one would jump to an erroneous conclusion. The confidence band method we present in the paper avoids such conclusions as both models are indistinguishable in terms of performance.
As many recent FR papers [1,2,3,4] improve the state-of-the-art performance (ROC) by very slight margins on one/two specific evaluation datasets, it becomes necessary to question the uncertainty regarding the true performance of these models. One new architecture/loss function might obtain a slightly better empirical ROC curve than other papers, while this empirical ROC might be contained within our ROC confidence bands for those other papers. Incorporating the uncertainty band in future model comparisons would lead to trustworthy conclusions.

Secondly, we chose to illustrate a model selection based on fairness in Figure 4. In this case, we employ a model deployment/production point of view. Face Recognition already has a considerable societal impact, while being biased against some subgroups of the population. In this sense, FR applications may have strict fairness constraints. For instance, one hypothetical legislation could be that the FRR metric for women should be lower than 4 times the FRR for men. In Figure 4 (zone B, e.g. FAR=6e-4), this constraint would forbid the ArcFace model (the FRR max-min fairness has confidence bands which can go higher than 4), i.e. the fairest model. Indeed, as for Fig. 1, one evaluation dataset might have this fairness metric equal to 4. Between AdaCos and ArcFace, one would then choose AdaCos, which is the model having the worst empirical fairness measure.

Thirdly, a significant contribution we make is the comparison of FR fairness metrics depending on their uncertainty. The question of measuring fairness is complex and the NIST, responsible for fairness audit of academic/industrial FR algorithms, is still hesitant on which fairness metric one should use to compare models [5]. Besides quantifying the differentials in FAR/FRR (among subgroups), such fairness metrics should be robust and exhibit a low uncertainty. Indeed, with high-impact applications, a fairness audit should be as trustworthy as possible and fairness measures should not vary a lot depending on the evaluation dataset. In addition, the subgroups considered become smaller and smaller when considering intersections of attributes (e.g. black women of age 40-50) and evaluation datasets might not have lots of data for such specific subgroups. This results in uncertain measures for those subgroups, altering the robustness of fairness measures. It becomes necessary to design fairness measures which are as robust as possible. We compare popular FR fairness measures, used by the NIST, depending on their uncertainty in Fig. 5 and conclude that the max-geomean metric is the more adequate. This result is supported by similar findings in the supplementary material B.2 (when varying the FR models, the sensitive attribute and the evaluation dataset). Note that this uncertainty quantification will be useful when evaluating the soundness of future fairness metrics.

1. This paper is not the first to apply the bootstrap method to ROC curves and there isn’t enough difference between it and Bertail’s work proposed in 2008. The concept of uncertainty proposed in this paper is relatively new in the field of ROC curve estimation, but it is too simplistic in the field of uncertainty estimation, which needs deeper elaboration.

Indeed, this paper is not the first to use the bootstrap to assess the uncertainty of a ROC curve, but it is the first to propose a proven valid bootstrap method applying to similarity ROC curves, and to highlight bias issues related to the distribution of empirical ROC’s. The work of Bertail et al. (2008), specialized to i.i.d. data, does not apply to the pairwise setup, to the similarity scoring framework (e.g. Face Recognition) in particular.

In Face Recognition, the evaluation step of any algorithm consists in forming all pairs of data and computing a score for each pair. This pairwise nature leads the empirical evaluation metrics (FAR, FRR) to be of the form of U-statistics, instead of being simple i.i.d. averages as in the work of Bertail et al. (2008). A direct consequence is that the scores averaged in the FAR/FRR metrics exhibit a complex dependence structure (see Section 3.1). This is not the case in the framework of Bertail et al. (2008) where the scores are independent.

Besides, applying Bertail et al. (2008) to the pairwise setup of Face Recognition (the naive bootstrap) yields strongly inaccurate confidence bands, as highlighted in Figures 2-3. Because of the pairwise nature of the evaluation metrics, such a bootstrap procedure tends to underestimate a lot the FRR metric, resulting in confidence bands for the ROC curve which do not even contain the empirical ROC curve most of the time (see Section 3.2 and Fig. 3). The recentered bootstrap we present within the paper counteracts this strong underestimation.

In addition, we show in the supplementary material B.1 the poor coverage of the naive bootstrap of Bertail et al. (2008), compared to the coverage of the recentered bootstrap (see Fig. 9 and Table 2). Thus, the naive bootstrap is not appropriate to measure the uncertainty of similarity ROC curves (it is appropriate for the non pairwise setup) while our recentered bootstrap has a coverage very close to the specified nominal coverage.

Lastly, our work proves the validity of the recentered bootstrap for the ROC curve, but also for popular FR fairness metrics, which is not the case for the bootstrap of Bertail et. al (2008).

The difference between our work and the method of Bertail et al. (2008) is hopefully clarified in the revised version of the paper (Related Works, Section 3.1, Section 3.2 and supplementary material B.1).

Regarding the relevance of the approach to uncertainty in the performance of FR systems we consider, we underline that, as it is inherent in the finiteness of the sizes of evaluation datasets only, no concept could be more exhaustive that the distributions themselves of empirical similarity ROC curves, the gold standard in evaluation, and these distributions are precisely what the proposed methodology estimates with guarantees.
We provide lower/upper bounds of a confidence interval, around the empirical ROC curve, at any confidence level. Our Theorem 1 proves that the obtained interval asymptotically contains the true ROC curve with a probability equal to the specified confidence level, which is exactly what a confidence interval should do. By construction, the width of the confidence interval is directly related to the variance of the empirical ROC curve which one computes from data (the randomness coming from the data at disposal, i.e. the aleatoric uncertainty). The same asymptotic property is also proved for fairness metrics. Lastly, our method holds even in the non asymptotic regime, as detailed in B.1 with the coverage being very close to the nominal coverage (specified confidence level).

2.3. Besides, there is no comparison of the proposed method with other methodologies.

This is a good remark. In the following, we explain how to compare confidence band methods and then we detail the chosen baselines.

As said in Part 2 of our answer, the coverage metric is the gold standard to quantify the accuracy of any method which builds confidence bands. The ideal property of a confidence interval is that specifying its confidence level $1-\alpha_{CI}$ (also called the nominal coverage) should lead to an interval having truly a probability equal to $1-\alpha_{CI}$ to contain the true (unknown) quantity $\mathrm{ROC}(\alpha)$. The coverage measures what this probability truly is and it should be equal (or very close) to the specified nominal coverage (see B.1). This coverage metric is the only appropriate way of comparing our method to others. Among several methods for building confidence bands, the most accurate is the one having an estimated coverage which is the closest to the specified nominal coverage.

We added two baselines for the revised version of the paper.

The first one is the naive bootstrap of Bertail et al. (2008). The non pairwise setup of the latter work significantly underestimates the ROC curve, leading to strongly inaccurate confidence intervals, as seen in Fig. 9 and Table 2.

As pointed out in the related works section of the paper, no method to quantify the uncertainty in measuring performance/fairness in FR is documented in the literature to the best of our knowledge. The one described and analyzed in this paper is the first one. As explained in Section 3.1, using the (Gaussian) asymptotic law of the empirical similarity ROC curve is not an option given the very complex form of its covariance structure (involving probability density functions).
That being said, we designed a more meaningful baseline than the naive bootstrap of Bertail et al. (2008). The sole reasonable alternative is to use the bootstrap technique we propose to evaluate the variance of the empirical ROC at a certain FAR level and build a Gaussian confidence interval based on it (see supplementary material B.1). As now shown in the revised version (see Fig. 9 and Table 2), the coverage of this Gaussian approximation is much more accurate than the naive bootstrap, but less satisfactory than our recentered bootstrap.

Lastly, we provide in the supplementary material B.1 the coverage of the bands obtained with the recentered bootstrap. This coverage metric is the gold standard to quantify the accuracy of any method which builds confidence bands. The ideal property of a confidence interval is that specifying its confidence level $1-\alpha_{CI}$ (also called the nominal coverage) should lead to an interval having truly a probability equal to $1-\alpha_{CI}$ to contain the true (unknown) quantity $\mathrm{ROC}(\alpha)$. The coverage measures what this probability truly is and it should be equal (or very close) to the specified nominal coverage (see B.1). From Fig. 6 and Table 1, we observe that, even with finite datasets, it is the case for the recentered bootstrap, which suggests that the latter provides an appropriate tool for assessing the uncertainty of the performance measure.

[1] Huang, Yuge, et al. "Curricularface: Adaptive curriculum learning loss for deep face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

[2] Meng, Qiang, et al. "Magface: A universal representation for face recognition and quality assessment." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.

[3] Boutros, Fadi, et al. "Elasticface: Elastic margin loss for deep face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[4] Kim, Minchul, Anil K. Jain, and Xiaoming Liu. "Adaface: Quality adaptive margin for face recognition." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[5] Grother, Patrick. "Face Recognition Vendor Test (FRVT) Part 8: Summarizing Demographic Differentials." (2022).

2.2. There is a lack of experimental evidence that the proposed uncertainty is a better evaluation metric to compare the performance and fairness of Face Recognition models.

Our proposed uncertainty metric (Eq. 11) is not a substitute for ROC or fairness metrics. It is introduced for Fig. 5, as detailed above. More generally, this metric allows one to (i) compare the uncertainty at several FAR levels for a fixed model, (ii) compare the uncertainty of different models at any FAR level, and (iii) compare the uncertainty of different fairness metrics at any FAR level (Fig. 5). We only presented (iii) within the paper. This allows for a selection between existing/future fairness metrics depending on their robustness with respect to the randomness of the evaluation data at disposal. In the same spirit, our confidence bands give additional information about the ROC curve (or fairness metrics) without being a substitute for those metrics.

We point out that any alternative uncertainty measure for the performance will be a by-product of the distribution of the empirical ROC curve. Regarding the uncertainty of (scalar) statistics measuring fairness, we agree that there is currently no full consensus about fairness metrics in FR but, as all of them will be necessarily summaries of the empirical ROC curves, the methodology promoted here can be applied to quantify the uncertainty for any fairness criterion.

We would like to thank you for your careful reading of the manuscript and your positive feedback. Here are our answers to your concerns.

1. There is a lack of justification of why Adacos has lower uncertainty than Arcface in fairness metric. To fairly compare the performance of Adacos and Arcface, training those methods on Fair Face Recognition dataset [A] might help to justify the performance better.

Indeed, AdaCos exhibits a surprisingly low uncertainty of its FRR max-min fairness, compared to ArcFace. We did not find any valid reason for this. The only valid justification would be to make the link between their loss function and the uncertainty (or at least a proxy) of their ROC for each gender. This task is complex and would be a work/paper on its own. For instance, the AdaCos/ArcFace loss function does not use any gender label for training. So, one would be forced to measure an uncertainty proxy during training for each gender (e.g. the variance of the logits for each gender, i.e. of the similarities between the embedding and the prototype of its identity for each gender for instance). Even doing so, there is a difference between the uncertainty one could measure during training and the uncertainty we tackle within the paper, which is about the randomness of evaluation data for each identity. To make a simple analogy, in the same way as ArcFace may have a better empirical fairness than AdaCos at some FAR levels, one model could have better uncertainty than others. In both cases, those models are not a priori trained to have the best fairness/uncertainty. Note that, as seen in B.3, the observed lower uncertainty of FRR fairness for AdaCos (compared to ArcFace) holds when changing the fairness metric.

In our opinion, training AdaCos/ArcFace on the Fair Face Recognition dataset would not change the conclusions. As the training set we consider (the MS-Celeb-1M-v1c-r dataset), the Fair Face Recognition training set is strongly imbalanced in terms of gender, skintone or age. Any model trained on those training sets, with a normal procedure, would be biased.

2. Does the training dataset affect the uncertainty of the face recognition methods?

An interesting question would be to find what happens to the fairness/uncertainty of a model when trained on a dataset with varying proportions of attributes (e.g. gender). Here again, this would be a work on its own, as the paper [1] details the influence of the gender balance in training data on the resulting gender-driven ROC’s. This is the equivalent of measuring the impact of the gender balance in training data on the empirical fairness metrics. A similar work could extend the study with our work, adding the uncertainty. We would like to underline that the purpose of the paper is not about the influence of the training set on the uncertainty, but rather about means of measuring the uncertainty about the randomness of evaluation data for any model, allowing for more trustworthy comparisons.

[1] Albiero, Vítor, Kai Zhang, and Kevin W. Bowyer. "How does gender balance in training data affect face recognition accuracy?." 2020 ieee international joint conference on biometrics (ijcb). IEEE, 2020.

3. Instead of the background and preliminaries, it is more important to include more comparison of fairness metrics with various FR methods in the supplementary material in the main manuscript. As the paper focuses on the application of fair recognition technology, it is important to add more justification on the corresponding problem.

We think that the preliminaries are necessary to introduce our multi-sample statistical setup, as well as the performance/fairness metrics. Those are the ingredients which make the U-statistic nature of the empirical FAR/FRR clear, i.e. the main reason why the naive bootstrap fails and how to counteract the underestimation of the ROC. The paper has applications in fairness, but also in performance (ROC) as seen in Fig.1. In the revised version of the paper, we added a comparison of the fairness metrics for ArcFace in B.4. Note that we prove the consistency of our confidence intervals for the ROC curve and any derivative/by-product of the attribute-driven ROC’s (which can be fairness metrics or completely different purpose metrics). This means that a simple algorithm exists to compute the uncertainty of lots of metrics deriving from ROC curves.