Nav: Home

Measuring AI's ability to learn is difficult

January 17, 2019

Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study from the University of Waterloo has found.

In a study published in Nature Machine Intelligence, Waterloo researchers found that contrary to conventional wisdom, there can be no exact method for deciding whether a given problem may be successfully solved by machine learning tools.

"We have to proceed with caution," said Shai Ben-David, lead author of the study and a professor in Waterloo's School of Computer Science. "There is a big trend of tools that are very successful, but nobody understands why they are successful, and nobody can provide guarantees that they will continue to be successful.

"In situations where just a yes or no answer is required, we know exactly what can or cannot be done by machine learning algorithms. However, when it comes to more general setups, we can't distinguish learnable from un-learnable tasks."

In the study, Ben-David and his colleagues considered a learning model called estimating the maximum (EMX), which captures many common machine learning tasks. For example, tasks like identifying the best place to locate a set of distribution facilities to optimize their accessibility for future expected consumers. The research found that no mathematical method would ever be able to tell, given a task in that model, whether an AI-based tool could handle that task or not.

"This finding comes as a surprise to the research community since it has long been believed that once a precise description of a task is provided, it can then be determined whether machine learning algorithms will be able to learn and carry out that task," said Ben-David.
-end-
The study, Learnability can be Undecidable, was co-authored by Ben-David, Pavel Hrubeš from the Institute of Mathematics of the Academy of Sciences in the Czech Republic, Shay Morgan from the Department of Computer Science, Princeton University, Amir Shpilka, Department of Computer Science, Tel Aviv University, and Amir Yehudayoff from the Department of Mathematics, Technion-IIT.

University of Waterloo

Related Artificial Intelligence Articles:

Hacking the human brain -- lab-made synapses for artificial intelligence
One of the greatest challenges facing artificial intelligence development is understanding the human brain and figuring out how to mimic it.
Artificial intelligence predicts patient lifespans
A computer's ability to predict a patient's lifespan simply by looking at images of their organs is a step closer to becoming a reality, thanks to new research led by the University of Adelaide.
Building a better 'bot': Artificial intelligence helps human groups
Artificial intelligence doesn't have to be super-sophisticated to make a difference in people's lives, according to a new Yale University study.
Artificial intelligence may help diagnose tuberculosis in remote areas
Researchers are training artificial intelligence models to identify tuberculosis (TB) on chest X-rays, which may help screening and evaluation efforts in TB-prevalent areas with limited access to radiologists, according to a new study.
Biased bots: Human prejudices sneak into artificial intelligence systems
In debates over the future of artificial intelligence, many experts think of the new systems as coldly logical and objectively rational.
Canada funds $125 million Pan-Canadian Artificial Intelligence Strategy
The Government of Canada is funding a Pan-Canadian Artificial Intelligence Strategy for research and talent that will cement Canada's position as a world leader in AI.
Artificial intelligence and robots to make offshore windfarms safer and cheaper
The University of Manchester is leading a consortium to investigate advanced technologies, including robotics and artificial intelligence, for the operation and maintenance of offshore windfarms.
Artificial intelligence virtual consultant helps deliver better patient care
Interventional radiologists at the University of California at Los Angeles (UCLA) are using technology found in self-driving cars to power a machine learning application that helps guide patients' interventional radiology care, according to research presented today at the Society of Interventional Radiology's 2017 Annual Scientific Meeting.
Artificial intelligence in quantum systems, too
The UPV/EHU's Department of Physical Chemistry has conducted world-class research in physics and quantum computation.
Artificial intelligence enters the nutraceutical industry
Life Extension (LE) launched a new line of nutraceuticals called GEROPROTECTTM, and the first product in the series called Ageless Cell combines some of the natural compounds that were shortlisted by Insilico Medicine's algorithms and are generally recognized as safe (GRAS).

Related Artificial Intelligence Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Climate Crisis
There's no greater threat to humanity than climate change. What can we do to stop the worst consequences? This hour, TED speakers explore how we can save our planet and whether we can do it in time. Guests include climate activist Greta Thunberg, chemical engineer Jennifer Wilcox, research scientist Sean Davis, food innovator Bruce Friedrich, and psychologist Per Espen Stoknes.
Now Playing: Science for the People

#527 Honey I CRISPR'd the Kids
This week we're coming to you from Awesome Con in Washington, D.C. There, host Bethany Brookshire led a panel of three amazing guests to talk about the promise and perils of CRISPR, and what happens now that CRISPR babies have (maybe?) been born. Featuring science writer Tina Saey, molecular biologist Anne Simon, and bioethicist Alan Regenberg. A Nobel Prize winner argues banning CRISPR babies won’t work Geneticists push for a 5-year global ban on gene-edited babies A CRISPR spin-off causes unintended typos in DNA News of the first gene-edited babies ignited a firestorm The researcher who created CRISPR twins defends...