Nav: Home

Ethics and AI: An unethical optimization principle

June 30, 2020

Artificial intelligence (AI) is increasingly deployed around us and may have large potential benefits. But there are growing concerns about the unethical use of AI. Professor Anthony Davison, who holds the Chair of Statistics at EPFL, and colleagues in the UK, have tackled these questions from a mathematical point of view, focusing on commercial AI that seek to maximize profits.

One example is an insurance company using AI to find a strategy for deciding premiums for potential customers. The AI will choose from many potential strategies, some of which may be discriminatory or may otherwise misuse customer data in ways that later lead to severe penalties for the company. Ideally, unethical strategies such as these would be removed from the space of potential strategies beforehand, but the AI does not have a moral sense, so it cannot distinguish between ethical and unethical strategies.

In work published in Royal Society Open Science on 1 July 2020, Davison and his co-authors Heather Battey (Imperial College London), Nicholas Beale (Sciteb Limited) and Robert MacKay (University of Warwick), show that an AI is likely to pick an unethical strategy in many situations. They formulate their results as an "Unethical Optimization Principle":

If an AI aims to maximize risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk.

This principle can help risk managers, regulators or others to detect the unethical strategies that might be hidden in a large strategy space. In an ideal world one would configure the AI to avoid unethical strategies, but this may be impossible because they cannot be specified in advance. In order to guide the use of the AI, the article suggests how to estimate the proportion of unethical strategies and the distribution of the most profitable strategies.

"Our work can be used to help regulators, compliance staff and others to find problematic strategies that might be hidden in a large strategy space. Such a space can be expected to contain disproportionately many unethical strategies, inspection of which should show where problems are likely to arise and thus suggest how the AI search algorithm should be modified to avoid them," says Professor Davison. "It also suggests that it may be necessary to re-think the way AI operates in very large strategy spaces, so that unethical outcomes are explicitly rejected during the learning process."

Professor Wendy Hall of the University of Southampton, known worldwide for her work on the potential practical benefits and problems brought by AI, said: "This is a really important paper. It shows that we can't just rely on AI systems to act ethically because their objectives seem ethically neutral. On the contrary, under mild conditions, an AI system will disproportionately find unethical solutions unless it is carefully designed to avoid them.

"The tremendous potential benefits of AI will only be realised properly if ethical behaviour is designed in from the ground up, taking account of this Unethical Optimisation Principle from a diverse set of perspectives. Encouragingly, this Principle can also be used to help find ethical problems with existing systems which can then be addressed by better design."

Beale N, Battey H, Davison AC, MacKay RS. (2020) An unethical optimization principle. R. Soc. Open Sci. 7: 200462. DOI: 10.1098/rsos.200462

Ecole Polytechnique Fédérale de Lausanne

Related Space Articles:

Space to grow, or grow in space -- how vertical farms could be ready to take-off
Vertical farms with their soil-free, computer-controlled environments may sound like sci-fi.
Space lettuce
Astronauts have now managed to grow lettuce inside specially designed chambers on the International Space Station.
A filament fit for space -- silk is proven to thrive in outer space temperatures
Scientists from the universities of Oxford, Shanghai and Beijing who discovered that natural silks get stronger the colder they get, have finally solved the puzzle of why.
Detecting bacteria in space
A new genomic approach provides a glimpse into the diverse bacterial ecosystem on the International Space Station.
Grease in space
The galaxy is rich in grease-like molecules, according to an Australian-Turkish team.
Surgery in space
With renewed public interest in manned space exploration comes the potential need to diagnose and treat medical issues encountered by future space travelers.
Viruses are everywhere, maybe even in space
Viruses are the most abundant and one of the least understood biological entities on Earth.
Space program should focus on Mars, says editor of New Space
The US space exploration program should continue to focus on robotic sample recovery and human missions to Mars, says Scott Hubbard, Editor-in-Chief of New Space.
Fireworks in space
Some of the most exciting things that we've seen from looking at gene expression in space is that we really see an explosion, like fireworks taking off, as soon as the human body gets into space.
NASA mission surfs through waves in space to understand space weather
NASA's Van Allen Probes have observed a new population of space sound waves, called plasmaspheric hiss, which are important in removing high-energy particles from around Earth that can damage satellites.
More Space News and Space Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Warped Reality
False information on the internet makes it harder and harder to know what's true, and the consequences have been devastating. This hour, TED speakers explore ideas around technology and deception. Guests include law professor Danielle Citron, journalist Andrew Marantz, and computer scientist Joy Buolamwini.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

How to Win Friends and Influence Baboons
Baboon troops. We all know they're hierarchical. There's the big brutish alpha male who rules with a hairy iron fist, and then there's everybody else. Which is what Meg Crofoot thought too, before she used GPS collars to track the movements of a troop of baboons for a whole month. What she and her team learned from this data gave them a whole new understanding of baboon troop dynamics, and, moment to moment, who really has the power.  This episode was reported and produced by Annie McEwen. Support Radiolab by becoming a member today at