Nav: Home

Synergy emergence in deep reinforcement motor learning

March 19, 2020

Human motor control has always been efficient at executing complex movements naturally, efficiently, and without much thought involved. This is because of the existence of motor synergy in the central nervous system (CNS). Motor synergy allows the CNS to use a smaller set of variables to control a large group of muscles; thereby simplifying the control over coordinated and complex movements.

Now, researchers at Tohoku University have observed a similar concept in robotic agents using deep reinforcement learning (DRL) algorithms.

DRL allows robotic agents to learn the best action possible in their virtual environment. It allows complex robotic tasks to be solved whilst minimalising manual operations and achieving peak performance. Classical algorithms, on the other hand, require manual intervention to find specific solutions for every new task that appears.

However, applying motor synergy from the human world to the robotic world is no small task. Even though many studies support the employment of motor synergy in human and animal motor control, the background process is still largely unknown.

In the current study, researchers from Tohoku University utilised two DRL algorithms on walking robotic agents known as HalfCheetah and FullCheetah. The two algorithms were TD3, a classical DRL, and SAC, a high-performing DRL.

The two robotic agents were tasked with running forward as far as possible within a given time. In total, the robotic agents completed 3 million steps. Synergy information was not used vis-à-vis the DRLs but the robotic agents demonstrated the emergence of motor synergy throughout their movements.

Mitsuhiro Hayashibe, Tohoku University professor and co-author of the study, notes, "We first confirmed in a quantitative way that motor synergy can emerge even in deep learning as humans do." Professor Hayashibe adds, "After employing deep learning, the robotic agents improved their motor performances while limiting energy consumption by employing motor synergy."

Going forward, the researchers aim to explore more task with different body models to further confirm their findings.
-end-


Tohoku University

Related Central Nervous System Articles:

Zika combats advanced-stage central nervous system tumors in dogs
The viral therapy was tested in three elderly animals with spontaneous brain tumors by a group affiliated with the FAPESP-funded Human Genome and Stem Cell Research Center.
Fewer scars in the central nervous system
Researchers have discovered the influence of the coagulation factor fibrinogen on the damaged brain.
Mayo Clinic discovers a molecular switch for repairing central nervous system disorders
A molecular switch has the ability to turn on a substance in animals that repairs neurological damage in disorders such as multiple sclerosis (MS), Mayo Clinic researchers discovered.
Discovery concerning the nervous system overturns a previous theory
It appears that when our nervous system is developing, only the most viable neurons survive, while immature neurons are weeded out and die.
Nanocapsule reaches cancer that has spread to central nervous system in mice
Researchers developed a drug delivery system that can break through the blood-brain barrier in mice.
Persistent HIV in central nervous system linked to cognitive impairment
Many people with HIV on antiretroviral therapy (ART) have viral genetic material in the cells of their cerebrospinal fluid, and these individuals are more likely to experience memory and concentration problems, according to a NIAID-funded study published in the Journal of Clinical Investigation.
First step to induce self-repair in the central nervous system
Injured axons instruct Schwann cells to build specialized actin spheres to break down and remove axon fragments, thereby starting the regeneration process.
First complete wiring diagram of an animal's nervous system
In a study published online today in Nature, researchers at Albert Einstein College of Medicine describe the first complete wiring diagram of the nervous system of an animal, the roundworm Caenorhabditis elegans, used by scientists worldwide as a model organism.
Scientists unlock new role for nervous system in regeneration
Biologists have developed a computational model of flatworm regeneration to answer an important question in regeneration research - what are the signals that determine the rebuilding of specific anatomical structures?
Microglia, immune cells of the central nervous system, shown to regulate neuroinflammation
A research team at Massachusetts Eye and Ear has shown that microglia, the immune cells of the central nervous system -- including the retina -- serve as 'gatekeepers' of neuroinflammation.
More Central Nervous System News and Central Nervous System Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Processing The Pandemic
Between the pandemic and America's reckoning with racism and police brutality, many of us are anxious, angry, and depressed. This hour, TED Fellow and writer Laurel Braitman helps us process it all.
Now Playing: Science for the People

#568 Poker Face Psychology
Anyone who's seen pop culture depictions of poker might think statistics and math is the only way to get ahead. But no, there's psychology too. Author Maria Konnikova took her Ph.D. in psychology to the poker table, and turned out to be good. So good, she went pro in poker, and learned all about her own biases on the way. We're talking about her new book "The Biggest Bluff: How I Learned to Pay Attention, Master Myself, and Win".
Now Playing: Radiolab

Invisible Allies
As scientists have been scrambling to find new and better ways to treat covid-19, they've come across some unexpected allies. Invisible and primordial, these protectors have been with us all along. And they just might help us to better weather this viral storm. To kick things off, we travel through time from a homeless shelter to a military hospital, pondering the pandemic-fighting power of the sun. And then, we dive deep into the periodic table to look at how a simple element might actually be a microbe's biggest foe. This episode was reported by Simon Adler and Molly Webster, and produced by Annie McEwen and Pat Walters. Support Radiolab today at Radiolab.org/donate.