Army researchers develop efficient distributed deep learning

February 18, 2020

ADELPHI, Md. (Feb. 18, 2020) -- A new algorithm is enabling deep learning that is more collaborative and communication-efficient than traditional methods.

Army researchers developed algorithms that facilitate distributed, decentralized and collaborative learning capabilities among devices, avoiding the need to pool all data at a central server for learning.

"There has been an exponential growth in the amount of data collected and stored locally on individual smart devices," said Dr. Jemin George, an Army scientist at the U.S. Army Combat Capabilities Development Command's Army Research Laboratory. "Numerous research efforts as well as businesses have focused on applying machine learning to extract value from such massive data to provide data-driven insights, decisions and predictions."

However, none of these efforts address any of the issues associated with applying machine learning to a contested, congested and constrained battlespace, George said. These battlespace constraints become more apparent when the devices are using deep learning algorithms for decision-making due to the heavy computational costs in terms of learning time and processing power.

"This research tries to address some of the challenges of applying machine learning, or deep learning, in military environments," said Dr. Prudhvi Gurram, a scientist who contributed to this research. "Early indications and warnings of threats enhance situational awareness and contribute to how the Army evolves and adapts to defeat adversarial threats."

The researchers presented their findings at the 34th Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence in New York. A pre-print version of the paper is online (see Related Links below).

In an earlier study (see Related Links below), the researchers demonstrated that the distributed deep learning algorithms can yield the same performance as the typical centralized learning algorithms without aggregating the data at a single, central location, while decreasing the learning time linearly with the number of devices or agents involved in distributed learning.

"Distributed learning algorithms typically require numerous rounds of communication among the agents or devices involved in the learning process to share their current model with the rest of the network," George said. "This presents several communication challenges."

The Army researchers developed a new technique to significantly decrease the communication overhead, by up to 70% in certain scenarios, without sacrificing the learning rate or performance accuracy.

The researchers developed a triggering mechanism, which allowed the individual agents to communicate their model with their neighbors only if it has significantly changed since it was last transmitted. Though this significantly decreases the communication interaction among the agents, it does not affect the overall learning rate or the performance accuracy of the final learned model, George said.

Army researchers are investigating how this research can be applied to the Internet of Battlefield Things, incorporating quantized and compressed communication schemes to the current algorithm to further reduce the communication overhead.

The Army's modernization priorities include next-generation computer networks (see Related Links below), which enable the Army to deliver leader-approved technology capabilities to warfighters at the best possible return on investment for the Army.

Future efforts will evaluate the algorithm behavior on larger, military-relevant datasets using the computing resources available through the U.S. Army AI Innovation Institute, with the algorithm expected to transition to run on edge devices, George said.
-end-
Related Links:

Distributed Deep Learning with Event-Triggered Communication https://arxiv.org/abs/1909.05020

Distributed Stochastic Gradient Method for Non-Convex Problems with Applications in Supervised Learning https://arxiv.org/abs/1908.06693

Army Network https://www.army.mil/standto/archive_2018-03-08/

CCDC Army Research Laboratory is an element of the U.S. Army Combat Capabilities Development Command. As the Army's corporate research laboratory, ARL discovers, innovates and transitions science and technology to ensure dominant strategic land power. Through collaboration across the command's core technical competencies, CCDC leads in the discovery, development and delivery of the technology-based capabilities required to make Soldiers more lethal to win our nation's wars and come home safely. CCDC is a major subordinate command of the U.S. Army Futures Command.

U.S. Army Research Laboratory

Related Learning Articles from Brightsurf:

Learning the language of sugars
We're told not to eat too much sugar, but in reality, all of our cells are covered in sugar molecules called glycans.

When learning on your own is not enough
We make decisions based on not only our own learning experience, but also learning from others.

Learning more about particle collisions with machine learning
A team of Argonne scientists has devised a machine learning algorithm that calculates, with low computational time, how the ATLAS detector in the Large Hadron Collider would respond to the ten times more data expected with a planned upgrade in 2027.

Getting kids moving, and learning
Children are set to move more, improve their skills, and come up with their own creative tennis games with the launch of HomeCourtTennis, a new initiative to assist teachers and coaches with keeping kids active while at home.

How expectations influence learning
During learning, the brain is a prediction engine that continually makes theories about our environment and accurately registers whether an assumption is true or not.

Technology in higher education: learning with it instead of from it
Technology has shifted the way that professors teach students in higher education.

Learning is optimized when we fail 15% of the time
If you're always scoring 100%, you're probably not learning anything new.

School spending cuts triggered by great recession linked to sizable learning losses for learning losses for students in hardest hit areas
Substantial school spending cuts triggered by the Great Recession were associated with sizable losses in academic achievement for students living in counties most affected by the economic downturn, according to a new study published today in AERA Open, a peer-reviewed journal of the American Educational Research Association.

Lessons in learning
A new Harvard study shows that, though students felt like they learned more from traditional lectures, they actually learned more when taking part in active learning classrooms.

Learning to look
A team led by JGI scientists has overhauled the perception of inovirus diversity.

Read More: Learning News and Learning Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.