Nav: Home

New machine learning algorithms offer safety and fairness guarantees

November 21, 2019

AMHERST, Mass. - Seventy years ago, science fiction writer Isaac Asimov imagined a world where robots would serve humans in countless ways, and he equipped them with built-in safeguards ¬now known as Asimov's Three Laws of Robotics, to prevent them, among other goals, from ever harming a person.

Guaranteeing safe and fair machine behavior is still an issue today, says machine learning researcher and lead author Philip Thomas at the University of Massachusetts Amherst. "When someone applies a machine learning algorithm, it's hard to control its behavior," he points out. This risks undesirable outcomes from algorithms that direct everything from self-driving vehicles to insulin pumps to criminal sentencing, say he and co-authors.

Writing in Science, Thomas and his colleagues Yuriy Brun, Andrew Barto and graduate student Stephen Giguere at UMass Amherst, Bruno Castro da Silva at the Federal University of Rio Grande del Sol, Brazil, and Emma Brunskill at Stanford University this week introduce a new framework for designing machine learning algorithms that make it easier for users of the algorithm to specify safety and fairness constraints.

"We call algorithms created with our new framework 'Seldonian' after Asimov's character Hari Seldon," Thomas explains. "If I use a Seldonian algorithm for diabetes treatment, I can specify that undesirable behavior means dangerously low blood sugar, or hypoglycemia. I can say to the machine, 'while you're trying to improve the controller in the insulin pump, don't make changes that would increase the frequency of hypoglycemia.' Most algorithms don't give you a way to put this type of constraint on behavior; it wasn't included in early designs."

"But making it easier to ensure fairness and avoid harm is becoming increasingly important as machine learning algorithms impact our lives more and more," he says.

However, "a recent paper listed 21 different definitions of fairness in machine learning. It's important that we allow the user to select the definition that is appropriate for their intended application," he adds. "The interface that comes with a Seldonian algorithm allows the user to do just this: to define what 'undesirable behavior' means for their application."

In Asimov's Foundation series, Seldon is in the same universe as his Robot series. Thomas explains, "Everything has fallen apart, the galactic empire is collapsing, partly because the Three Laws of Robotics require certainty. With that level of safety required, robots are paralyzed with indecision because they cannot act with certainty and guarantee that no human will be harmed by their actions."

Seldon proposes fixing this by turning to reasoning probabilistically about safety. "That's a good fit to what we're doing, Thomas says. The new approach he and colleagues provide allows for probabilistic constraints and requires the algorithm to specify ways the user can tell it what to constrain. He says, "The framework is a tool for the machine learning researcher. It guides them toward creating algorithms that are easier for users to apply responsibly to real-world problems."

To test the new framework, they applied it to predict grade point averages in a data set of 43,000 students in Brazil by creating a Seldonian algorithm with constraints. It successfully avoided several types of undesirable gender bias. In another test, they show how an algorithm could improve the controller in an insulin pump while guaranteeing that it would not increase the frequency of hypoglycemia.

Thomas says, "We believe there's massive room for improvement in this area. Even with our algorithms made of simple components, we obtained impressive results. We hope that machine learning researchers will go on to develop new and more sophisticated algorithms using our framework, which can be used responsibly for applications where machine learning used to be considered too risky. It's a call to other researchers to conduct research in this space."

University of Massachusetts Amherst

Related Behavior Articles:

I won't have what he's having: The brain and socially motivated behavior
Monkeys devalue rewards when they anticipate that another monkey will get them instead.
Unlocking animal behavior through motion
Using physics to study different types of animal motion, such as burrowing worms or flying flocks, can reveal how animals behave in different settings.
AI to help monitor behavior
Algorithms based on artificial intelligence do better at supporting educational and clinical decision-making, according to a new study.
Increasing opportunities for sustainable behavior
To mitigate climate change and safeguard ecosystems, we need to make drastic changes in our consumption and transport behaviors.
Predicting a protein's behavior from its appearance
Researchers at EPFL have developed a new way to predict a protein's interactions with other proteins and biomolecules, and its biochemical activity, merely by observing its surface.
Spirituality affects the behavior of mortgagers
According to Olga Miroshnichenko, a Sc.D in Economics, and a Professor at the Department of Economics and Finance, Tyumen State University, morals affect the thinking of mortgage payers and help them avoid past due payments.
Asking if behavior can be changed on climate crisis
One of the more complex problems facing social psychologists today is whether any intervention can move people to change their behavior about climate change and protecting the environment for the sake of future generations.
Is Instagram behavior motivated by a desire to belong?
Does a desire to belong and perceived social support drive a person's frequency of Instagram use?
A 3D view of climatic behavior at the third pole
Research across several areas of the 'Third Pole' -- the high-mountain region centered on the Tibetan Plateau -- shows a seasonal cycle in how near-surface temperature changes with elevation.
Witnessing uncivil behavior
When people witness poor customer service, a manager's intervention can help reduce hostility toward the company or brand, according to WSU research.
More Behavior News and Behavior Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Clint Smith
The killing of George Floyd by a police officer has sparked massive protests nationwide. This hour, writer and scholar Clint Smith reflects on this moment, through conversation, letters, and poetry.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Dispatch 6: Strange Times
Covid has disrupted the most basic routines of our days and nights. But in the middle of a conversation about how to fight the virus, we find a place impervious to the stalled plans and frenetic demands of the outside world. It's a very different kind of front line, where urgent work means moving slow, and time is marked out in tiny pre-planned steps. Then, on a walk through the woods, we consider how the tempo of our lives affects our minds and discover how the beats of biology shape our bodies. This episode was produced with help from Molly Webster and Tracie Hunte. Support Radiolab today at