Characterizing the variability in face recognition accuracy relative to race

K Vangara, MC King, V Albiero… - Proceedings of the …, 2019 - openaccess.thecvf.com
Proceedings of the IEEE/CVF Conference on Computer Vision and …, 2019openaccess.thecvf.com
Many recent news headlines have labeled face recognition technology as" biased" or"
racist". We report on a methodical investigation into differences in face recognition accuracy
between African-American and Caucasian image cohorts of the MORPH dataset. We find
that, for all four matchers considered, the impostor and the genuine distributions are
statistically significantly different between cohorts. For a fixed decision threshold, the African-
American image cohort has a higher false match rate and a lower false non-match rate. ROC …
Abstract
Many recent news headlines have labeled face recognition technology as" biased" or" racist". We report on a methodical investigation into differences in face recognition accuracy between African-American and Caucasian image cohorts of the MORPH dataset. We find that, for all four matchers considered, the impostor and the genuine distributions are statistically significantly different between cohorts. For a fixed decision threshold, the African-American image cohort has a higher false match rate and a lower false non-match rate. ROC curves compare verification rates at the same false match rate, but the different cohorts achieve the same false match rate at different thresholds. This means that ROC comparisons are not relevant to operational scenarios that use a fixed decision threshold. We show that, for the ResNet matcher, the two cohorts have approximately equal separation of impostor and genuine distributions. Using ICAO compliance as a standard of image quality, we find that the initial image cohorts have unequal rates of good quality images. The ICAO-compliant subsets of the original image cohorts show improved accuracy, with the main effect being to reducing the low-similarity tail of the genuine distributions.
openaccess.thecvf.com
Showing the best result for this search. See all results