Carnegie Mellon theory of visual computation reveals how brain makes sense of natural scenes

November 19, 2008

PITTSBURGH--Computational neuroscientists at Carnegie Mellon University have developed a computational model that provides insight into the function of the brain's visual cortex and the information processing that enables people to perceive contours and surfaces, and understand what they see in the world around them.

A type of visual neuron known as simple cells can detect lines, or edges, but the computation they perform is insufficient to make sense of natural scenes, said Michael S. Lewicki, associate professor in Carnegie Mellon's Computer Science Department and the Center for the Neural Basis of Cognition. Edges often are obscured by variations in the foreground and background surfaces within the scene, he said, so more sophisticated processing is necessary to understand the complete picture. But little is known about how the visual system accomplishes this feat.

In a paper published online by the journal Nature, Lewicki and his graduate student, Yan Karklin, outline their computational model of this visual processing. The model employs an algorithm that analyzes the myriad patterns that compose natural scenes and statistically characterizes those patterns to determine which patterns are most likely associated with each other.

The bark of a tree, for instance, is composed of a multitude of different local image patterns, but the computational model can determine that all these local images represent bark and are all part of the same tree, as well as determining that those same patches are not part of a bush in the foreground or the hill behind it.

"Our model takes a statistical approach to making these generalizations about each patch in the image," said Lewicki, who currently is on sabbatical at the Institute for Advanced Study in Berlin. "We don't know if the visual system computes exactly in this way, but it is behaving as if it is."

Lewicki and Karklin report that the response of their model neurons to images used in physiological experiments matches well with the response of neurons in higher visual stages. These "complex cells," so-called for their more complex response properties, have been extensively studied, but the role they play in visual processing has been elusive. "We were astonished that the model reproduced so many of the properties of these cells just as a result of solving this computational problem," Lewicki said.

The human brain makes these interpretations of visual stimuli effortlessly, but computer scientists have long struggled to program computers to do the same. "We don't have computer vision algorithms that function as well as the brain," Lewicki said, so computers often have trouble recognizing objects, understanding their three-dimensional nature and appreciating how the objects they see are juxtaposed across a landscape. A deeper understanding of how the brain perceives the world could translate into improved computer vision systems.

In the meantime, the functional explanation of complex cells suggested by the computer model will enable scientists to develop new ways of investigating the visual system and other brain areas. "It's still a theory, after all, so naturally you want to test it further," Lewicki noted. But if the model is confirmed, it could establish a new paradigm for how we derive the general from the specific.
Karklin, who earned his Ph.D. in computational neuroscience, machine learning and computer science in 2007, is now a post-doctoral fellow at New York University.

The Computer Science Department is a division of Carnegie Mellon's School of Computer Science. The Center for the Neural Basis of Cognition is a joint project of Carnegie Mellon and the University of Pittsburgh.

About Carnegie Mellon: Carnegie Mellon is a private research university with a distinctive mix of programs in engineering, computer science, robotics, business, public policy, science and social science, fine arts and the humanities. More than 10,000 undergraduate and graduate students receive an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration, and innovation. A small student-to-faculty ratio provides an opportunity for close interaction between students and professors. While technology is pervasive on its 144-acre Pittsburgh campus, Carnegie Mellon is also distinctive among leading research universities for the world-renowned programs in its College of Fine Arts. A global university, Carnegie Mellon has campuses in Silicon Valley, Calif., and Qatar, and programs in Asia, Australia and Europe. For more, see

Carnegie Mellon University

Related Brain Articles from Brightsurf:

Glioblastoma nanomedicine crosses into brain in mice, eradicates recurring brain cancer
A new synthetic protein nanoparticle capable of slipping past the nearly impermeable blood-brain barrier in mice could deliver cancer-killing drugs directly to malignant brain tumors, new research from the University of Michigan shows.

Children with asymptomatic brain bleeds as newborns show normal brain development at age 2
A study by UNC researchers finds that neurodevelopmental scores and gray matter volumes at age two years did not differ between children who had MRI-confirmed asymptomatic subdural hemorrhages when they were neonates, compared to children with no history of subdural hemorrhage.

New model of human brain 'conversations' could inform research on brain disease, cognition
A team of Indiana University neuroscientists has built a new model of human brain networks that sheds light on how the brain functions.

Human brain size gene triggers bigger brain in monkeys
Dresden and Japanese researchers show that a human-specific gene causes a larger neocortex in the common marmoset, a non-human primate.

Unique insight into development of the human brain: Model of the early embryonic brain
Stem cell researchers from the University of Copenhagen have designed a model of an early embryonic brain.

An optical brain-to-brain interface supports information exchange for locomotion control
Chinese researchers established an optical BtBI that supports rapid information transmission for precise locomotion control, thus providing a proof-of-principle demonstration of fast BtBI for real-time behavioral control.

Transplanting human nerve cells into a mouse brain reveals how they wire into brain circuits
A team of researchers led by Pierre Vanderhaeghen and Vincent Bonin (VIB-KU Leuven, Université libre de Bruxelles and NERF) showed how human nerve cells can develop at their own pace, and form highly precise connections with the surrounding mouse brain cells.

Brain scans reveal how the human brain compensates when one hemisphere is removed
Researchers studying six adults who had one of their brain hemispheres removed during childhood to reduce epileptic seizures found that the remaining half of the brain formed unusually strong connections between different functional brain networks, which potentially help the body to function as if the brain were intact.

Alcohol byproduct contributes to brain chemistry changes in specific brain regions
Study of mouse models provides clear implications for new targets to treat alcohol use disorder and fetal alcohol syndrome.

Scientists predict the areas of the brain to stimulate transitions between different brain states
Using a computer model of the brain, Gustavo Deco, director of the Center for Brain and Cognition, and Josephine Cruzat, a member of his team, together with a group of international collaborators, have developed an innovative method published in Proceedings of the National Academy of Sciences on Sept.

Read More: Brain News and Brain Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to