Optimizing neural networks on a brain-inspired computer

July 22, 2020

Many computational properties are maximized when the dynamics of a network are at a "critical point", a state where systems can quickly change their overall characteristics in fundamental ways, transitioning e.g. between order and chaos or stability and instability. Therefore, the critical state is widely assumed to be optimal for any computation in recurrent neural networks, which are used in many AI applications.

Researchers from the HBP partner Heidelberg University and the Max-Planck-Institute for Dynamics and Self-Organization challenged this assumption by testing the performance of a spiking recurrent neural network on a set of tasks with varying complexity at - and away from critical dynamics. They instantiated the network on a prototype of the analog neuromorphic BrainScaleS-2 system. BrainScaleS is a state-of-the-art brain-inspired computing system with synaptic plasticity implemented directly on the chip. It is one of two neuromorphic systems currently under development within the European Human Brain Project.

First, the researchers showed that the distance to criticality can be easily adjusted in the chip by changing the input strength, and then demonstrated a clear relation between criticality and task-performance. The assumption that criticality is beneficial for every task was not confirmed: whereas the information-theoretic measures all showed that network capacity was maximal at criticality, only the complex, memory intensive tasks profited from it, while simple tasks actually suffered. The study thus provides a more precise understanding of how the collective network state should be tuned to different task requirements for optimal performance.

Mechanistically, the optimal working point for each task can be set very easily under homeostatic plasticity by adapting the mean input strength. The theory behind this mechanism was developed very recently at the Max Planck Institute. "Putting it to work on neuromorphic hardware shows that these plasticity rules are very capable in tuning network dynamics to varying distances from criticality", says senior author Viola Priesemann, group leader at MPIDS. Thereby tasks of varying complexity can be solved optimally within that space.

The finding may also explain why biological neural networks operate not necessarily at criticality, but in the dynamically rich vicinity of a critical point, where they can tune their computation properties to task requirements. Furthermore, it establishes neuromorphic hardware as a fast and scalable avenue to explore the impact of biological plasticity rules on neural computation and network dynamics.

"As a next step, we now study and characterize the impact of the spiking network's working point on classifying artificial and real-world spoken words", says first author Benjamin Cramer of Heidelberg University.

Human Brain Project

Related Task Articles from Brightsurf:

People with less body response to stress task had more PTSD signs after COVID-19 began
People who did not have a large heart rate response to a stress task surprised researchers later -- after the onset of the COVID-19 pandemic -- when they showed more symptoms of post-traumatic stress disorder related to the crisis than others who also did the stress task and COVID-19 stress ratings.

Task force examines role of mobile health technology in COVID-19 pandemic
An international task force, including two University of Massachusetts Amherst computer scientists, concludes in new research that mobile health (mHealth) technologies are a viable option to monitor COVID-19 patients at home and predict which ones will need medical intervention.

The most important task for a PTSD service dog for veterans is disrupting anxiety
Science has shown that service dogs can benefit some veterans with PTSD.

Task force recommends changes to increase African-American physics and astronomy students
Due to long-term and systemic issues leading to the consistent exclusion of African-Americans in physics and astronomy, a task force is recommending sweeping changes and calling for awareness into the number and experiences of African-American students studying the fields.

Chimpanzees more likely to share tools, teach skills when task is complex
A new study finds that chimpanzees that use a multi-step process and complex tools to gather termites are more likely to share tools with novices.

Task force provides insights and direction on cell-based therapies
A new report published in the Journal of Bone and Mineral Research and concurrently in the Journal of Orthopaedic Research highlights the latest advances in cell-based therapies for the treatment of disorders of the musculoskeletal system, such as arthritis and osteoporosis, and it identifies key unanswered questions that should be addressed through ongoing research.

New model more accurately predicts choices in classic decision-making task
A new mathematical model that predicts which choices people will make in the Iowa Gambling Task, a task used for the past 25 years to study decision-making, outperforms previously developed models.

Dehydration alters human brain shape and activity, slackens task performance
Sweating up a storm gardening? This is what your brain might be doing: Reasonably customary dehydration led to shape changes inside the brains of test persons in a new study.

Monkeys' brains synchronize as they collaborate to perform a motor task
Scientists have previously shown that when one animal watches another performing a motor task, such as reaching for food, mirror neurons in the motor cortex of the observer's brain start firing as though the observer were also reaching for food.

Results from new global health task shifting trial surprise researchers
One of the largest stroke rehabilitation trials ever undertaken has revealed family-led rehabilitation is ineffective.

Read More: Task News and Task Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.