Nav: Home

Framework improves 'continual learning' for artificial intelligence

May 15, 2019

Researchers have developed a new framework for deep neural networks that allows artificial intelligence (AI) systems to better learn new tasks while "forgetting" less of what it has learned regarding previous tasks. The researchers have also demonstrated that using the framework to learn a new task can make the AI better at performing previous tasks, a phenomenon called backward transfer.

"People are capable of continual learning; we learn new tasks all the time, without forgetting what we already know," says Tianfu Wu, an assistant professor of electrical and computer engineering at NC State and co-author of a paper on the work. "To date, AI systems using deep neural networks have not been very good at this."

"Deep neural network AI systems are designed for learning narrow tasks," says Xilai Li, a co-lead author of the paper and a Ph.D. candidate at NC State. "As a result, one of several things can happen when learning new tasks. Systems can forget old tasks when learning new ones, which is called catastrophic forgetting. Systems can forget some of the things they knew about old tasks, while not learning to do new ones as well. Or systems can fix old tasks in place while adding new tasks - which limits improvement and quickly leads to an AI system that is too large to operate efficiently. Continual learning, also called lifelong-learning or learning-to-learn, is trying to address the issue."

"We have proposed a new framework for continual learning, which decouples network structure learning and model parameter learning," says Yingbo Zhou, co-lead author of the paper and a research scientist at Salesforce Research. "We call it the Learn to Grow framework. In experimental testing, we've found that it outperforms previous approaches to continual learning."

To understand the Learn to Grow framework, think of deep neural networks as a pipe filled with multiple layers. Raw data goes into the top of the pipe, and task outputs come out the bottom. Every "layer" in the pipe is a computation that manipulates the data in order to help the network accomplish its task, such as identifying objects in a digital image. There are multiple ways of arranging the layers in the pipe, which correspond to different "architectures" of the network.

When asking a deep neural network to learn a new task, the Learn to Grow framework begins by conducting something called an explicit neural architecture optimization via search. What this means is that as the network comes to each layer in its system, it can decide to do one of four things: skip the layer; use the layer in the same way that previous tasks used it; attach a lightweight adapter to the layer, which modifies it slightly; or create an entirely new layer.

This architecture optimization effectively lays out the best topology, or series of layers, needed to accomplish the new task. Once this is complete, the network uses the new topology to train itself on how to accomplish the task - just like any other deep learning AI system.

"We've run experiments using several datasets, and what we've found is that the more similar a new task is to previous tasks, the more overlap there is in terms of the existing layers that are kept to perform the new task," Li says. "What is more interesting is that, with the optimized - or "learned" topology - a network trained to perform new tasks forgets very little of what it needed to perform the older tasks, even if the older tasks were not similar."

The researchers also ran experiments comparing the Learn to Grow framework's ability to learn new tasks to several other continual learning methods, and found that the Learn to Grow framework had better accuracy when completing new tasks.

To test how much each network may have forgotten when learning the new task, the researchers then tested each system's accuracy at performing the older tasks - and the Learn to Grow framework again outperformed the other networks.

"In some cases, the Learn to Grow framework actually got better at performing the old tasks," says Caiming Xiong, the research director of Salesforce Research and a co-author of the work. "This is called backward transfer, and occurs when you find that learning a new task makes you better at an old task. We see this in people all the time; not so much with AI."
-end-
The paper, "Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting," will be presented at the 36th International Conference on Machine Learning, being held June 9-15 in Long Beach, California. Co-lead authors of the paper are Xilai Li, a Ph.D. student at NC State, and Yingbo Zhou of Salesforce Research. The paper was co-authored by Richard Socher and Caiming Xiong of Salesforce Research.

The work was done with support from the U.S. Army Research Office under grants W911NF1810295 and W911NF1810209; and from the National Science Foundation, under grant 1822477. Part of the work was done while Li was a summer intern at Salesforce AI Research.

North Carolina State University

Related Learning Articles:

Learning with light: New system allows optical 'deep learning'
A team of researchers at MIT and elsewhere has come up with a new approach to complex computations, using light instead of electricity.
Mount Sinai study reveals how learning in the present shapes future learning
The prefrontal cortex shapes memory formation by modulating hippocampal encoding.
Better learning through zinc?
Zinc is a vital micronutrient involved in many cellular processes: For example, in learning and memory processes, it plays a role that is not yet understood.
Deep learning and stock trading
A study undertaken by researchers at the School of Business and Economics at Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) has shown that computer programs that algorithms based on artificial intelligence are able to make profitable investment decisions.
Learning makes animals intelligent
The fact that animals can use tools, have self-control and certain expectations of life can be explained with the help of a new learning model for animal behavior.
Learning Morse code without trying
Researchers at the Georgia Institute of Technology have developed a system that teaches people Morse code within four hours using a series of vibrations felt near the ear.
The adolescent brain is adapted to learning
Teenagers are often portrayed as seeking immediate gratification, but new work suggests that their sensitivity to reward could be part of an evolutionary adaptation to learn from their environment.
The brain watched during language learning
Researchers from Nijmegen, the Netherlands, have for the first time captured images of the brain during the initial hours and days of learning a new language.
Learning in the absence of external feedback
Rewards act as external factors that influence and reinforce learning processes.
New learning procedure for neural networks
Neural networks learn to link temporally dispersed stimuli.

Related Learning Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Digital Manipulation
Technology has reshaped our lives in amazing ways. But at what cost? This hour, TED speakers reveal how what we see, read, believe — even how we vote — can be manipulated by the technology we use. Guests include journalist Carole Cadwalladr, consumer advocate Finn Myrstad, writer and marketing professor Scott Galloway, behavioral designer Nir Eyal, and computer graphics researcher Doug Roble.
Now Playing: Science for the People

#530 Why Aren't We Dead Yet?
We only notice our immune systems when they aren't working properly, or when they're under attack. How does our immune system understand what bits of us are us, and what bits are invading germs and viruses? How different are human immune systems from the immune systems of other creatures? And is the immune system so often the target of sketchy medical advice? Those questions and more, this week in our conversation with author Idan Ben-Barak about his book "Why Aren't We Dead Yet?: The Survivor’s Guide to the Immune System".