Are hiring algorithms fair? They're too opaque to tell, study finds

November 20, 2019

ITHACA, N.Y. - Time is money and, unfortunately for companies, hiring new employees takes significant time - more than a month on average, research shows.

Hiring decisions are also rife with human bias, leading some organizations to hand off at least part of their employee searches to outside tech companies who screen applicants with machine learning algorithms. If humans have such a hard time finding the best fit for their companies, the thinking goes, maybe a machine can do it better and more efficiently.

But new research from a team of Computing and Information Science scholars at Cornell University raises questions about those algorithms and the tech companies who develop and use them: How unbiased is the automated screening process? How are the algorithms built? And by whom, toward what end, and with what data?

They found companies tend to favor obscurity over transparency in this emerging field, where lack of consensus on fundamental points - formal definitions of "bias" and "fairness," for starters - have enabled tech companies to define and address algorithmic bias on their own terms.

"I think we're starting to see a growing recognition among creators of algorithmic decision-making tools that they need to be particularly cognizant of how their tools impact people," said Manish Raghavan, a doctoral student in computer science and first author of "Mitigating Bias in Algorithmic Employment Screening: Evaluating Claims and Practices," to be presented in January at the Association for Computing Machinery Conference on Fairness, Accountability and Transparency.

"Many of the vendors we encountered in our work acknowledge this (impact) and they're taking steps to address bias and discrimination," Raghavan said. "However, there's a notable lack of consensus or direction on exactly how this should be done."

The researchers scoured available public information to begin to understand these tools and what measures, if any, companies have in place to evaluate and mitigate algorithmic bias. Shielded by intellectual property laws, tech companies don't have to disclose any information about their algorithmic models for pre-employment screenings - though some companies did choose to offer insight.

The researchers honed in on 19 vendors who specialize in algorithmic pre-employment screenings, which, they found, include questions, video interview analysis and games. They combed company websites, webinars and any available documents for insights into vendor claims and practices.

Very few vendors offer concrete information about how they validate their assessments or disclose specifics on how they mitigate algorithmic bias, researchers found.

"Plenty of vendors make no mention of efforts to combat bias, which is particularly worrying since either they're not thinking about it at all, or they're not being transparent about their practices," Raghavan said.

Even if they use such terms as "bias" and "fairness," these can be vague. A vendor can claim its assessment algorithm is "fair" without revealing how the company defines fairness.

It's like "free-range" eggs, Raghavan said: There is a set of conditions under which eggs can be labeled free range, but our intuitive notion of free range may not line up with those conditions.

"In the same way, calling an algorithm 'fair' appeals to our intuitive understanding of the term while only accomplishing a much narrower result than we might hope for," he said.

The team hopes the paper will encourage transparency and conversation around what it means to act ethically in this domain of pre-employment assessments through machine learning.

Given the challenges, could it be that algorithms are just not up to the job of screening applicants? Not so fast, Raghavan said.

"We know from years of empirical evidence that humans suffer from a variety of biases when it comes to evaluating employment candidates," he said. "The real question is not whether algorithms can be made perfect; instead, the relevant comparison is whether they can improve over alternative methods, or in this case, the human status quo.

"Despite their many flaws," he said, "algorithms do have the potential to contribute to a more equitable society, and further work is needed to ensure that we can understand and mitigate the biases they bring."
The paper's co-authors are Solon Barocas, assistant professor of information science, currently at Microsoft Research; Jon Kleinberg, Tisch University Professor of Computer Science and interim dean of CIS; and Karen Levy, assistant professor of information science.

The research is supported, in part, by a grant from the MacArthur Foundation to Cornell's Artificial Intelligence, Policy, and Practice Initiative; the National Science Foundation; and Microsoft Research.

Cornell University

Related Algorithms Articles from Brightsurf:

A multidisciplinary policy design to protect consumers from AI collusion
Legal scholars, computer scientists and economists must work together to prevent unlawful price-surging behaviors from artificial intelligence (AI) algorithms used by rivals in a competitive market, argue Emilio Calvano and colleagues in this Policy Forum.

Students develop tool to predict the carbon footprint of algorithms
Within the scientific community, it is estimated that artificial intelligence -- otherwise meant to serve as a means to effectively combat climate change -- will become one of the most egregious CO2 culprits should current trends continue.

Machine learning takes on synthetic biology: algorithms can bioengineer cells for you
Scientists at Lawrence Berkeley National Laboratory have developed a new tool that adapts machine learning algorithms to the needs of synthetic biology to guide development systematically.

Algorithms uncover cancers' hidden genetic losses and gains
Limitations in DNA sequencing technology make it difficult to detect some major mutations often linked to cancer, such as the loss or duplication of parts of chromosomes.

Managing data flow boosts cyber-physical system performance
Researchers have developed a suite of algorithms to improve the performance of cyber-physical systems - from autonomous vehicles to smart power grids - by balancing each component's need for data with how fast that data can be sent and received.

New theory hints at more efficient way to develop quantum algorithms
A new theory could bring a way to make quantum algorithm development less of an accidental process, say Purdue University scientists.

AI as good as the average radiologist in identifying breast cancer
Researchers at Karolinska Institutet and Karolinska University Hospital in Sweden have compared the ability of three different artificial intelligence (AI) algorithms to identify breast cancer based on previously taken mammograms.

Context reduces racial bias in hate speech detection algorithms
When it comes to accurately flagging hate speech on social media, context matters, says a new USC study aimed at reducing errors that could amplify racial bias.

Researchers discover algorithms and neural circuit mechanisms of escape responses
Prof. WEN Quan from School of Life Sciences, University of Science and Technology of China (USTC) of the Chinese Academy of Sciences (CAS) has proposed the algorithms and circuit mechanisms for the robust and flexible motor states of nematodes during escape responses.

Lightning fast algorithms can lighten the load of 3D hologram generation
Tokyo, Japan - Researchers from Tokyo Metropolitan University have developed a new way of calculating simple holograms for heads-up displays (HUDs) and near-eye displays (NEDs).

Read More: Algorithms News and Algorithms Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to