Nav: Home

Measuring AI's ability to learn is difficult

January 17, 2019

Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study from the University of Waterloo has found.

In a study published in Nature Machine Intelligence, Waterloo researchers found that contrary to conventional wisdom, there can be no exact method for deciding whether a given problem may be successfully solved by machine learning tools.

"We have to proceed with caution," said Shai Ben-David, lead author of the study and a professor in Waterloo's School of Computer Science. "There is a big trend of tools that are very successful, but nobody understands why they are successful, and nobody can provide guarantees that they will continue to be successful.

"In situations where just a yes or no answer is required, we know exactly what can or cannot be done by machine learning algorithms. However, when it comes to more general setups, we can't distinguish learnable from un-learnable tasks."

In the study, Ben-David and his colleagues considered a learning model called estimating the maximum (EMX), which captures many common machine learning tasks. For example, tasks like identifying the best place to locate a set of distribution facilities to optimize their accessibility for future expected consumers. The research found that no mathematical method would ever be able to tell, given a task in that model, whether an AI-based tool could handle that task or not.

"This finding comes as a surprise to the research community since it has long been believed that once a precise description of a task is provided, it can then be determined whether machine learning algorithms will be able to learn and carry out that task," said Ben-David.
-end-
The study, Learnability can be Undecidable, was co-authored by Ben-David, Pavel Hrubeš from the Institute of Mathematics of the Academy of Sciences in the Czech Republic, Shay Morgan from the Department of Computer Science, Princeton University, Amir Shpilka, Department of Computer Science, Tel Aviv University, and Amir Yehudayoff from the Department of Mathematics, Technion-IIT.

University of Waterloo

Related Artificial Intelligence Articles:

Artificial intelligence improves biomedical imaging
ETH researchers use artificial intelligence to improve quality of images recorded by a relatively new biomedical imaging method.
Evolution of learning is key to better artificial intelligence
Researchers at Michigan State University say that true, human-level intelligence remains a long way off, but their new paper published in The American Naturalist explores how computers could begin to evolve learning in the same way as natural organisms did -- with implications for many fields, including artificial intelligence.
Artificial intelligence probes dark matter in the universe
A team of physicists and computer scientists at ETH Zurich has developed a new approach to the problem of dark matter and dark energy in the universe.
Artificial intelligence used to recognize primate faces in the wild
Scientists at the University of Oxford have developed new artificial intelligence software to recognize and track the faces of individual chimpanzees in the wild.
The brain inspires a new type of artificial intelligence
Using advanced experiments on neuronal cultures and large scale simulations, scientists at Bar-Ilan University have demonstrated a new type of ultrafast artifical intelligence algorithms -- based on the very slow brain dynamics -- which outperform learning rates achieved to date by state-of-the-art learning algorithms.
More Artificial Intelligence News and Artificial Intelligence Current Events

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Erasing The Stigma
Many of us either cope with mental illness or know someone who does. But we still have a hard time talking about it. This hour, TED speakers explore ways to push past — and even erase — the stigma. Guests include musician and comedian Jordan Raskopoulos, neuroscientist and psychiatrist Thomas Insel, psychiatrist Dixon Chibanda, anxiety and depression researcher Olivia Remes, and entrepreneur Sangu Delle.
Now Playing: Science for the People

#537 Science Journalism, Hold the Hype
Everyone's seen a piece of science getting over-exaggerated in the media. Most people would be quick to blame journalists and big media for getting in wrong. In many cases, you'd be right. But there's other sources of hype in science journalism. and one of them can be found in the humble, and little-known press release. We're talking with Chris Chambers about doing science about science journalism, and where the hype creeps in. Related links: The association between exaggeration in health related science news and academic press releases: retrospective observational study Claims of causality in health news: a randomised trial This...