Federal study finds race, gender bias in facial recognition technology

A study by a U.S. agency has found that facial recognition technology often performs unevenly based on a person’s race, gender or age.

But the nuanced report published Thursday is unlikely to allay the concerns of critics who worry about bias in face-scanning applications that are increasingly being adopted by law enforcement, airports and a variety of businesses.

Facebook wants your face:Should you say yes to facial recognition?

The problem with facial recognition:Study says it's too white and male, calls for more women, minorities

Facial recognition:Do you really control how your face is being used?

The National Institute of Standards and Technology has been studying facial recognition for nearly two decades, but this is the first time it has investigated demographic differences in how face-scanning algorithms are able to identify people.

The study was prompted in part by growing concern among lawmakers and privacy advocates that biased results in commercial face recognition software could entrench racial discrimination in the criminal justice system and elsewhere.

The report cautions against “incomplete” previous research alleging biased facial recognition that has alarmed the public, but also confirms similar trends showing higher error rates for women, the youngest and oldest people, and for certain racial groups depending on which image database or software is being used.

Facial recognition ban?:Why some cities, states and lawmakers want to curb facial recognition technology

Pew survey:Americans trust police more than tech giants to use facial recognition

Face recognition scanners at US airports:Here's what to know

“There is a wide range of performance and there’s certainly work to be done,” said Craig Watson, manager of NIST’s research group that studies biometric technology. “The main message is don’t try to generalize the results across all the technology. Know your use case, the algorithm that’s being used.”

NIST, which is a part of the Commerce Department, tested the algorithms of 99 mostly commercial software providers that voluntarily submitted their technology for review. It ran those algorithms on millions of FBI mugshots, visa application photos and other government-held portrait images such as those taken at border crossings.

Microsoft was among the major tech companies that participated in the research, along with dozens of lesser-known video surveillance providers and numerous China-based companies such as SenseTime, Hikvision and Tencent. Amazon, which markets face-scanning software to U.S. police agencies, did not participate.

Watson said that’s because Amazon’s cloud-based software doesn’t work with NIST’s testing procedures, though the agency is in talks with the company about how to test its algorithms in the future.

The agency’s report credits two widely-cited studies of facial recognition bias by Massachusetts Institute of Technology researchers for serving as a “cautionary tale” about uneven results across race and gender boundaries, though it also suggests they sowed public confusion in the way they sought to measure performance.