Nav: Home

Numenta brings brain theory to machine learning

November 14, 2016

REDWOOD CITY, CA - November 14, 2016-- Numerous proposals have been offered for how intelligent machines might learn sequences of patterns, which is believed to be an essential component of any intelligent system. Researchers at Numenta Inc. have published a new study, "Continuous Online Sequence Learning with an Unsupervised Neural Network Model," which compares their biologically-derived HTM sequence memory to traditional machine learning algorithms.

The paper has been published in MIT Press Journal's Neural Computation 28, 2474-2504 (2016). You can read and download the paper here.

Authored by Numenta researchers Yuwei Cui, Subutai Ahmad, and Jeff Hawkins, the new paper serves as a companion piece to Numenta's breakthrough research offered in "Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex," which appeared in Frontiers in Neural Circuits, in March 2016.

The earlier paper described a biological theory of how networks of neurons in the neocortex learn sequences. In this paper, the authors demonstrate how this theory, HTM sequence memory, can be applied to sequence learning and prediction of streaming data.

"Our primary goal at Numenta is to understand, in detail, how the neocortex works. We believe the principles we learn from the brain will be essential for creating intelligent machines, so a second part of our mission is to bridge the two worlds of neuroscience and AI. This new work demonstrates progress towards that goal," Hawkins commented.

In the new paper, HTM sequence memory is compared with four popular statistical and machine learning techniques: ARIMA, a statistical method for time-series forecasting (Durbin & Koopman 2012); extreme learning machine (ELM), a feedforward network with sequential online learning (Huang, Zhu, & Siew, 2006); and two recurrent networks, long-short term memory (LSTM) (Hochreiter and Schmidhuber 1997) and echo state networks (ESN) (Jaeger and Hass 2004).

The results in this paper show that HTM sequence memory achieves comparable prediction accuracy to these other techniques. However, the HTM model also exhibits several properties that are critical for streaming data applications including:
  • Continuous online learning
  • Ability to make multiple simultaneous predictions
  • Robustness to sensor noise and fault tolerance
  • Good performance without task-specific tuning
"Many existing machine learning techniques demonstrate some of these properties," Cui noted, "but a truly powerful system for streaming analytics should have all of them."

The HTM sequence memory algorithm is something that machine learning experts can test and incorporate into a broad range of applications. In keeping with Numenta's open research philosophy, the source code for replicating the graphs in the paper can be found here. Numenta also welcome questions and discussion about the paper on the HTM Forum or by contacting the authors directly.
  • Yuwei Cui, Subutai Ahmad, Jeff Hawkins (2016) Continuous Online Sequence Learning with an Unsupervised Neural Network Model. Neural Computation 28(11), 2474-2504. doi:10.1162/NECO_a_00893

  • Hawkins, J., and Ahmad, S. (2016). Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex. Front. Neural Circuits 10. doi:10.3389/fncir.2016.00023

About Neural Computation

Neural Computation disseminates important, multidisciplinary research results in a field that attracts psychologists, physicists, computer scientists, neuroscientists, and artificial intelligence investigators, among others. For researchers looking at the scientific and engineering challenges of understanding the brain and building computers, Neural Computation highlights common problems and techniques in modeling the brain, and in the design and construction of neurally-inspired information processing systems.

About Numenta

Founded in 2005, Numenta develops theory, software technology, and applications all based on reverse engineering the neocortex. Laying the groundwork for the new era of machine intelligence, this technology is ideal for analysis of continuously streaming data and excels at modeling and predicting patterns in data. Numenta has also developed a suite of products and demonstration applications that utilize its flexible and generalizable Hierarchical Temporal Memory (HTM) learning algorithms to provide solutions that encompass the fields of machine generated data, human behavioral modeling, geo-location processing, semantic understanding and sensory-motor control. In addition, Numenta has created NuPIC (Numenta Platform for Intelligent Computing) as an open source project. Numenta is based in Redwood City, California. Connect with Numenta on Twitter, Facebook, Google+ and LinkedIn.

Krause Taylor Associates

Related Machine Learning Articles:

Using machine learning to predict pediatric brain injury
When newborn babies or children with heart or lung distress are struggling to survive, doctors often turn to a form of life support that uses artificial lungs.
Cannabis data lacking, but machine learning could help
Everyone's heard of THC and CBD. But many other active compounds in cannabis interact to influence its effects.
Fifty new planets confirmed in machine learning first
Fifty potential planets have had their existence confirmed by a new machine learning algorithm developed by University of Warwick scientists.
Machine learning peeks into nano-aquariums
In the nanoworld, tiny particles such as proteins appear to dance as they transform and assemble to perform various tasks while suspended in a liquid.
Applying machine learning to biomedical science
Dr Pengyi Yang and colleagues from the University of Sydney have brought together the latest developments in applications of machine learning in biomedical science, showing that new techniques are combining ensemble methods with deep learning, with potential applications in cancer research and better understanding viruses.
Study: Machine learning can predict market behavior
Machine learning can assess the effectiveness of mathematical tools used to predict the movements of financial markets, according to new Cornell research based on the largest dataset ever used in this area.
Machine learning research may help find new tungsten deposits in SW England
Geologists have developed a machine learning technique that highlights the potential for further deposits of the critical metal tungsten in SW England.
Machine learning finds a surprising early galaxy
New results achieved by combining big data captured by the Subaru Telescope and the power of machine learning have discovered a galaxy with an extremely low oxygen abundance of 1.6% solar abundance, breaking the previous record of the lowest oxygen abundance.
Machine learning accurately predicts who's who in the health care workforce
Until recently, economists, policy makers and workforce experts have relied on outdated and inaccurate snapshots of the US physician workforce, making it especially difficult to predict the need and availability of health care services across the country.
Learning more about particle collisions with machine learning
A team of Argonne scientists has devised a machine learning algorithm that calculates, with low computational time, how the ATLAS detector in the Large Hadron Collider would respond to the ten times more data expected with a planned upgrade in 2027.
More Machine Learning News and Machine Learning Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Power Of Spaces
How do spaces shape the human experience? In what ways do our rooms, homes, and buildings give us meaning and purpose? This hour, TED speakers explore the power of the spaces we make and inhabit. Guests include architect Michael Murphy, musician David Byrne, artist Es Devlin, and architect Siamak Hariri.
Now Playing: Science for the People

#576 Science Communication in Creative Places
When you think of science communication, you might think of TED talks or museum talks or video talks, or... people giving lectures. It's a lot of people talking. But there's more to sci comm than that. This week host Bethany Brookshire talks to three people who have looked at science communication in places you might not expect it. We'll speak with Mauna Dasari, a graduate student at Notre Dame, about making mammals into a March Madness match. We'll talk with Sarah Garner, director of the Pathologists Assistant Program at Tulane University School of Medicine, who takes pathology instruction out of...
Now Playing: Radiolab

What If?
There's plenty of speculation about what Donald Trump might do in the wake of the election. Would he dispute the results if he loses? Would he simply refuse to leave office, or even try to use the military to maintain control? Last summer, Rosa Brooks got together a team of experts and political operatives from both sides of the aisle to ask a slightly different question. Rather than arguing about whether he'd do those things, they dug into what exactly would happen if he did. Part war game part choose your own adventure, Rosa's Transition Integrity Project doesn't give us any predictions, and it isn't a referendum on Trump. Instead, it's a deeply illuminating stress test on our laws, our institutions, and on the commitment to democracy written into the constitution. This episode was reported by Bethel Habte, with help from Tracie Hunte, and produced by Bethel Habte. Jeremy Bloom provided original music. Support Radiolab by becoming a member today at     You can read The Transition Integrity Project's report here.