Ethics and AI: An unethical optimization principle

June 30, 2020

Artificial intelligence (AI) is increasingly deployed around us and may have large potential benefits. But there are growing concerns about the unethical use of AI. Professor Anthony Davison, who holds the Chair of Statistics at EPFL, and colleagues in the UK, have tackled these questions from a mathematical point of view, focusing on commercial AI that seek to maximize profits.

One example is an insurance company using AI to find a strategy for deciding premiums for potential customers. The AI will choose from many potential strategies, some of which may be discriminatory or may otherwise misuse customer data in ways that later lead to severe penalties for the company. Ideally, unethical strategies such as these would be removed from the space of potential strategies beforehand, but the AI does not have a moral sense, so it cannot distinguish between ethical and unethical strategies.

In work published in Royal Society Open Science on 1 July 2020, Davison and his co-authors Heather Battey (Imperial College London), Nicholas Beale (Sciteb Limited) and Robert MacKay (University of Warwick), show that an AI is likely to pick an unethical strategy in many situations. They formulate their results as an "Unethical Optimization Principle":

If an AI aims to maximize risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk.

This principle can help risk managers, regulators or others to detect the unethical strategies that might be hidden in a large strategy space. In an ideal world one would configure the AI to avoid unethical strategies, but this may be impossible because they cannot be specified in advance. In order to guide the use of the AI, the article suggests how to estimate the proportion of unethical strategies and the distribution of the most profitable strategies.

"Our work can be used to help regulators, compliance staff and others to find problematic strategies that might be hidden in a large strategy space. Such a space can be expected to contain disproportionately many unethical strategies, inspection of which should show where problems are likely to arise and thus suggest how the AI search algorithm should be modified to avoid them," says Professor Davison. "It also suggests that it may be necessary to re-think the way AI operates in very large strategy spaces, so that unethical outcomes are explicitly rejected during the learning process."

Professor Wendy Hall of the University of Southampton, known worldwide for her work on the potential practical benefits and problems brought by AI, said: "This is a really important paper. It shows that we can't just rely on AI systems to act ethically because their objectives seem ethically neutral. On the contrary, under mild conditions, an AI system will disproportionately find unethical solutions unless it is carefully designed to avoid them.

"The tremendous potential benefits of AI will only be realised properly if ethical behaviour is designed in from the ground up, taking account of this Unethical Optimisation Principle from a diverse set of perspectives. Encouragingly, this Principle can also be used to help find ethical problems with existing systems which can then be addressed by better design."

Beale N, Battey H, Davison AC, MacKay RS. (2020) An unethical optimization principle. R. Soc. Open Sci. 7: 200462. DOI: 10.1098/rsos.200462

Ecole Polytechnique Fédérale de Lausanne

Related Space Articles from Brightsurf:

Space to grow, or grow in space -- how vertical farms could be ready to take-off
Vertical farms with their soil-free, computer-controlled environments may sound like sci-fi.

Space lettuce
Astronauts have now managed to grow lettuce inside specially designed chambers on the International Space Station.

A filament fit for space -- silk is proven to thrive in outer space temperatures
Scientists from the universities of Oxford, Shanghai and Beijing who discovered that natural silks get stronger the colder they get, have finally solved the puzzle of why.

Detecting bacteria in space
A new genomic approach provides a glimpse into the diverse bacterial ecosystem on the International Space Station.

Grease in space
The galaxy is rich in grease-like molecules, according to an Australian-Turkish team.

Surgery in space
With renewed public interest in manned space exploration comes the potential need to diagnose and treat medical issues encountered by future space travelers.

Viruses are everywhere, maybe even in space
Viruses are the most abundant and one of the least understood biological entities on Earth.

Space program should focus on Mars, says editor of New Space
The US space exploration program should continue to focus on robotic sample recovery and human missions to Mars, says Scott Hubbard, Editor-in-Chief of New Space.

Fireworks in space
Some of the most exciting things that we've seen from looking at gene expression in space is that we really see an explosion, like fireworks taking off, as soon as the human body gets into space.

NASA mission surfs through waves in space to understand space weather
NASA's Van Allen Probes have observed a new population of space sound waves, called plasmaspheric hiss, which are important in removing high-energy particles from around Earth that can damage satellites.

Read More: Space News and Space Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to