Nav: Home

Analytic technique could allow neural networks to run on cellphones

July 18, 2017

In recent years, the best-performing artificial-intelligence systems -- in areas such as autonomous driving, speech recognition, computer vision, and automatic translation -- have come courtesy of software systems known as neural networks.

But neural networks take up a lot of memory and consume a lot of power, so they usually run on servers in the cloud, which receive data from desktop or mobile devices and then send back their analyses.

Last year, MIT associate professor of electrical engineering and computer science Vivienne Sze and colleagues unveiled a new, energy-efficient computer chip optimized for neural networks, which could enable powerful artificial-intelligence systems to run locally on mobile devices.

Now, Sze and her colleagues have approached the same problem from the opposite direction, with a battery of techniques for designing more energy-efficient neural networks. First, they developed an analytic method that can determine how much power a neural network will consume when run on a particular type of hardware. Then they used the method to evaluate new techniques for paring down neural networks so that they'll run more efficiently on handheld devices.

The researchers describe the work in a paper they're presenting next week at the Computer Vision and Pattern Recognition Conference. In the paper, they report that the methods offered as much as a 73 percent reduction in power consumption over the standard implementation of neural networks, and as much as a 43 percent reduction over the best previous method for paring the networks down.

Energy evaluator

Loosely based on the anatomy of the brain, neural networks consist of thousands or even millions of simple but densely interconnected information-processing nodes, usually organized into layers. Different types of networks vary according to their number of layers, the number of connections between the nodes, and the number of nodes in each layer.

The connections between nodes have "weights" associated with them, which determine how much a given node's output will contribute to the next node's computation. During training, in which the network is presented with examples of the computation it's learning to perform, those weights are continually readjusted, until the output of the network's last layer consistently corresponds with the result of the computation.

"The first thing we did was develop an energy-modeling tool that accounts for data movement, transactions, and data flow," Sze says. "If you give it a network architecture and the value of its weights, it will tell you how much energy this neural network will take. One of the questions that people had is 'Is it more energy efficient to have a shallow network and more weights or a deeper network with fewer weights?' This tool gives us better intuition as to where the energy is going, so that an algorithm designer could have a better understanding and use this as feedback. The second thing we did is that, now that we know where the energy is actually going, we started to use this model to drive our design of energy-efficient neural networks."

In the past, Sze explains, researchers attempting to reduce neural networks' power consumption used a technique called "pruning." Low-weight connections between nodes contribute very little to a neural network's final output, so many of them can be safely eliminated, or pruned.

Principled pruning

With the aid of their energy model, Sze and her colleagues -- first author Tien-Ju Yang and Yu-Hsin Chen, both graduate students in electrical engineering and computer science -- varied this approach. Although cutting even a large number of low-weight connections can have little effect on a neural net's output, cutting all of them probably would, so pruning techniques must have some mechanism for deciding when to stop.

The MIT researchers thus begin pruning those layers of the network that consume the most energy. That way, the cuts translate to the greatest possible energy savings. They call this method "energy-aware pruning."

Weights in a neural network can be either positive or negative, so the researchers' method also looks for cases in which connections with weights of opposite sign tend to cancel each other out. The inputs to a given node are the outputs of nodes in the layer below, multiplied by the weights of their connections. So the researchers' method looks not only at the weights but also at the way the associated nodes handle training data. Only if groups of connections with positive and negative weights consistently offset each other can they be safely cut. This leads to more efficient networks with fewer connections than earlier pruning methods did.
-end-
Additional background

PAPER: Designing energy-efficient convolutional neural networks using energy-aware pruning

https://arxiv.org/pdf/1611.05128.pdf

ARCHIVE: Peering into neural networks

http://news.mit.edu/2017/inner-workings-neural-networks-visual-data-0630

ARCHIVE: New system allows optical "deep learning"

http://news.mit.edu/2017/new-system-allows-optical-deep-learning-0612

ARCHIVE: Explained: Neural networks

http://news.mit.edu/2017/explained-neural-networks-deep-learning-0414

ARCHIVE: Energy-friendly chip can perform powerful artificial-intelligence tasks

http://news.mit.edu/2016/neural-chip-artificial-intelligence-mobile-devices-0203

Massachusetts Institute of Technology

Related Electrical Engineering Articles:

3D-printed plastics with high performance electrical circuits
Rutgers engineers have embedded high performance electrical circuits inside 3D-printed plastics, which could lead to smaller and versatile drones and better-performing small satellites, biomedical implants and smart structures.
In and out with 10-minute electrical vehicle recharge
Electric vehicle owners may soon be able to pull into a fueling station, plug their car in, go to the restroom, get a cup of coffee and in 10 minutes, drive out with a fully charged battery, according to a team of engineers.
Electrical stimulation aids in spinal fusion
Spine surgeons in the U.S. perform more than 400,000 spinal fusions each year as a way to ease back pain and prevent vertebrae in the spine from wiggling around and doing more damage.
The effectiveness of electrical stimulation in producing spinal fusion
Researchers from The Johns Hopkins University School of Medicine performed a systematic review and meta-analysis of published data on the effect of electrical stimulation therapies on spinal fusion.
Fat pumps generate electrical power
A previously unknown electrical current develops in the body's cells when the vital fat pump function of the flippases transfers ('flips') lipids from the outer to the inner layer of the body's cell membranes.
UCI electrical engineering team develops 'beyond 5G' wireless transceiver
An end-to-end transmitter-receiver created by engineers in UCI's Nanoscale Communication Integrated Circuits Labs, is a 4.4-millimeter-square silicon chip that is capable of processing digital signals with significantly greater speed and energy efficiency because of its unique digital-analog architecture.
How electrical stimulation reorganizes the brain
Recordings of neural activity during therapeutic stimulation can be used to predict subsequent changes in brain connectivity, according to a study of epilepsy patients published in JNeurosci.
Electrical signals kick off flatworm regeneration
In a study publishing March 5 in Biophysical Journal, scientists report that electrical activity is the first known step in the tissue-regeneration process of planarian flatworms, starting before the earliest known genetic machinery kicks in and setting off the downstream activities of gene transcription needed to construct new heads or tails.
Electrical activity in prostate cancer cells
Experts from the universities of Bath and Seville have carried out a series of experiments with which, for the first time, they have been able to characterize the normal electrical activity in PC-3 prostate cancer cells in real time, with a resulting low-frequency electrical pattern between 0.1 and 10 Hertz.
Toward a secure electrical grid
Professor João Hespanha suggests a way to protect autonomous grids from potentially crippling GPS spoofing attacks.
More Electrical Engineering News and Electrical Engineering Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Reinvention
Change is hard, but it's also an opportunity to discover and reimagine what you thought you knew. From our economy, to music, to even ourselves–this hour TED speakers explore the power of reinvention. Guests include OK Go lead singer Damian Kulash Jr., former college gymnastics coach Valorie Kondos Field, Stockton Mayor Michael Tubbs, and entrepreneur Nick Hanauer.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Dispatch 6: Strange Times
Covid has disrupted the most basic routines of our days and nights. But in the middle of a conversation about how to fight the virus, we find a place impervious to the stalled plans and frenetic demands of the outside world. It's a very different kind of front line, where urgent work means moving slow, and time is marked out in tiny pre-planned steps. Then, on a walk through the woods, we consider how the tempo of our lives affects our minds and discover how the beats of biology shape our bodies. This episode was produced with help from Molly Webster and Tracie Hunte. Support Radiolab today at Radiolab.org/donate.