Nav: Home

New machine learning algorithms offer safety and fairness guarantees

November 21, 2019

AMHERST, Mass. - Seventy years ago, science fiction writer Isaac Asimov imagined a world where robots would serve humans in countless ways, and he equipped them with built-in safeguards ¬now known as Asimov's Three Laws of Robotics, to prevent them, among other goals, from ever harming a person.

Guaranteeing safe and fair machine behavior is still an issue today, says machine learning researcher and lead author Philip Thomas at the University of Massachusetts Amherst. "When someone applies a machine learning algorithm, it's hard to control its behavior," he points out. This risks undesirable outcomes from algorithms that direct everything from self-driving vehicles to insulin pumps to criminal sentencing, say he and co-authors.

Writing in Science, Thomas and his colleagues Yuriy Brun, Andrew Barto and graduate student Stephen Giguere at UMass Amherst, Bruno Castro da Silva at the Federal University of Rio Grande del Sol, Brazil, and Emma Brunskill at Stanford University this week introduce a new framework for designing machine learning algorithms that make it easier for users of the algorithm to specify safety and fairness constraints.

"We call algorithms created with our new framework 'Seldonian' after Asimov's character Hari Seldon," Thomas explains. "If I use a Seldonian algorithm for diabetes treatment, I can specify that undesirable behavior means dangerously low blood sugar, or hypoglycemia. I can say to the machine, 'while you're trying to improve the controller in the insulin pump, don't make changes that would increase the frequency of hypoglycemia.' Most algorithms don't give you a way to put this type of constraint on behavior; it wasn't included in early designs."

"But making it easier to ensure fairness and avoid harm is becoming increasingly important as machine learning algorithms impact our lives more and more," he says.

However, "a recent paper listed 21 different definitions of fairness in machine learning. It's important that we allow the user to select the definition that is appropriate for their intended application," he adds. "The interface that comes with a Seldonian algorithm allows the user to do just this: to define what 'undesirable behavior' means for their application."

In Asimov's Foundation series, Seldon is in the same universe as his Robot series. Thomas explains, "Everything has fallen apart, the galactic empire is collapsing, partly because the Three Laws of Robotics require certainty. With that level of safety required, robots are paralyzed with indecision because they cannot act with certainty and guarantee that no human will be harmed by their actions."

Seldon proposes fixing this by turning to reasoning probabilistically about safety. "That's a good fit to what we're doing, Thomas says. The new approach he and colleagues provide allows for probabilistic constraints and requires the algorithm to specify ways the user can tell it what to constrain. He says, "The framework is a tool for the machine learning researcher. It guides them toward creating algorithms that are easier for users to apply responsibly to real-world problems."

To test the new framework, they applied it to predict grade point averages in a data set of 43,000 students in Brazil by creating a Seldonian algorithm with constraints. It successfully avoided several types of undesirable gender bias. In another test, they show how an algorithm could improve the controller in an insulin pump while guaranteeing that it would not increase the frequency of hypoglycemia.

Thomas says, "We believe there's massive room for improvement in this area. Even with our algorithms made of simple components, we obtained impressive results. We hope that machine learning researchers will go on to develop new and more sophisticated algorithms using our framework, which can be used responsibly for applications where machine learning used to be considered too risky. It's a call to other researchers to conduct research in this space."

University of Massachusetts Amherst

Related Behavior Articles:

How synaptic changes translate to behavior changes
Learning changes behavior by altering many connections between brain cells in a variety of ways all at the same time, according to a study of sea slugs recently published in JNeurosci.
I won't have what he's having: The brain and socially motivated behavior
Monkeys devalue rewards when they anticipate that another monkey will get them instead.
Unlocking animal behavior through motion
Using physics to study different types of animal motion, such as burrowing worms or flying flocks, can reveal how animals behave in different settings.
AI to help monitor behavior
Algorithms based on artificial intelligence do better at supporting educational and clinical decision-making, according to a new study.
Increasing opportunities for sustainable behavior
To mitigate climate change and safeguard ecosystems, we need to make drastic changes in our consumption and transport behaviors.
Predicting a protein's behavior from its appearance
Researchers at EPFL have developed a new way to predict a protein's interactions with other proteins and biomolecules, and its biochemical activity, merely by observing its surface.
Spirituality affects the behavior of mortgagers
According to Olga Miroshnichenko, a Sc.D in Economics, and a Professor at the Department of Economics and Finance, Tyumen State University, morals affect the thinking of mortgage payers and help them avoid past due payments.
Asking if behavior can be changed on climate crisis
One of the more complex problems facing social psychologists today is whether any intervention can move people to change their behavior about climate change and protecting the environment for the sake of future generations.
Is Instagram behavior motivated by a desire to belong?
Does a desire to belong and perceived social support drive a person's frequency of Instagram use?
A 3D view of climatic behavior at the third pole
Research across several areas of the 'Third Pole' -- the high-mountain region centered on the Tibetan Plateau -- shows a seasonal cycle in how near-surface temperature changes with elevation.
More Behavior News and Behavior Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Processing The Pandemic
Between the pandemic and America's reckoning with racism and police brutality, many of us are anxious, angry, and depressed. This hour, TED Fellow and writer Laurel Braitman helps us process it all.
Now Playing: Science for the People

#568 Poker Face Psychology
Anyone who's seen pop culture depictions of poker might think statistics and math is the only way to get ahead. But no, there's psychology too. Author Maria Konnikova took her Ph.D. in psychology to the poker table, and turned out to be good. So good, she went pro in poker, and learned all about her own biases on the way. We're talking about her new book "The Biggest Bluff: How I Learned to Pay Attention, Master Myself, and Win".
Now Playing: Radiolab

Invisible Allies
As scientists have been scrambling to find new and better ways to treat covid-19, they've come across some unexpected allies. Invisible and primordial, these protectors have been with us all along. And they just might help us to better weather this viral storm. To kick things off, we travel through time from a homeless shelter to a military hospital, pondering the pandemic-fighting power of the sun. And then, we dive deep into the periodic table to look at how a simple element might actually be a microbe's biggest foe. This episode was reported by Simon Adler and Molly Webster, and produced by Annie McEwen and Pat Walters. Support Radiolab today at