Hands that see, eyes that feel? Brain study reveals the mathematics of identifying objects

May 29, 2019

NEW YORK -- From a child snapping Legos together to a pickpocket plucking a wallet from your bag, our brains have a remarkable ability to spot new objects and figure out how to manipulate them. Scientists have long believed that the brain accomplishes this by methodically interpreting visual and textural cues, such as an object's edges or boundaries. But a new study suggests that the human brain requires only a tiny bit of information, as well as its previous experience, to calculate a complete mental representation of a new object. These results help to explain the mental mathematics that enable us to easily know what a novel object looks like simply by touching it, or the way an object feels from sight alone.

This study, led by researchers at Columbia University, the University of Cambridge and the Central European University and reported in the journal eLife, illustrates the brain's natural power to learn quickly and generalize.

"Our brains' ability to single out one object from many by touch -- the way pickpockets use their fingers to hone in on a wallet deep inside a purse -- is a broadly used skill, and key to our ability to interact with the world," said Daniel Wolpert, PhD, a principal investigator at Columbia's Mortimer B. Zuckerman Mind Brain Behavior Institute and the study's co-senior author. "Our latest study exemplifies the brain's knack for performing mathematics to infer an object's identity."

Nearly 40 years ago, scientists trying to understand how we identify individual objects proposed that the edges or boundaries of each item allow us to distinguish one object from the next. But Dr. Wolpert and the research team hypothesized that this explanation did not tell the whole story and was perhaps a smaller part of a much larger, more generalized principle of how the brain infers properties about its surroundings.

"We wondered whether our brains could do more, with less," said Máté Lengyel, PhD, professor of computational neuroscience from the University of Cambridge, a research fellow at the Central European University and one of the study's senior authors. "Perhaps they don't need to acquire and analyze boundary information systematically in order to identify an object, but can instead work out an object's identity by performing clever statistical analyses that also incorporate memories and experiences,"

These "clever statistical analyses," Dr. Lengyel and his colleagues thought, could allow our brains to not only immediately identify objects we've encountered before, but also predict key properties of new objects we come across.

To test their object-identification hypothesis, the researchers developed a simple computer game where players watched jigsaw-like puzzle pieces stuck together in various combinations on a screen. Their task, which they generally did with ease, was to recognize which combinations appeared together more frequently.

"The players' brains were able to gather visual information about the pairs of puzzle pieces, such as which pieces were most often found together, or which looked easiest to pull apart," said Dr. Wolpert, who is also a professor of neuroscience at Columbia's Vagelos College of Physicians and Surgeons.

A second set of participants was given similar scenes of puzzle pieces stuck together. But instead of using their eyes to work out which pieces went together as a pair, they used their sense of touch. Scientists call this form of identification haptic perception.

The players grasped two handles -- mounted to a robotic contraption that controlled the forces the player experienced when ripping apart different sets of puzzle pieces. Some combinations were easy to pull apart; others, more difficult.

After providing visual or haptic information to each of the two groups of players, the research team swapped the groups; the visual group now evaluated puzzle pieces using their sense of touch, and the haptic group now used their sense of vision.

"Our results of this swap revealed players' adeptness to take the knowledge they had learned from one of the two modalities -- either visual or haptic -- and extrapolate to the other modality," said Dr. Lengyel.

These findings challenge the conventional view of how our brains extract and learn about our environment. Even when faced with a minimal amount of source material -- a small amount of statistics about how often two objects appeared together, or how much force it took to pull them apart -- the human brain could make powerful inferences and connections.

"Our study shows that our brains are wondrously adept at generalizing from one modality, such as vision, to another, such as touch," said Dr. Wolpert. "This may be because our brains have calculated a statistical understanding of how objects behave based on our previous experiences. This study further reveals that the computations our brains perform are sufficiently powerful for achieving a multitude of cognitive feats -- whether it be picking someone's pocket or imagining the feel of a leather purse in a window display."
This paper is titled "Unimodal statistical learning produces multimodal object-like representations." Additional contributors include first author Gábor Lengyel, Goda Zalayte, Alexandros Pantelides, James N. Ingram, PhD, and co-senior author Józef Fiser, PhD.

This research was supported by the European Research Council (Consolidator Grant ERC-2016-COG/726090), the Royal Society (Noreen Murray Professorship in Neurobiology RP120142), the Seventh Framework Programme (Marie Curie CIG 618918), the Wellcome Trust (New Investigator Award 095621/Z/11/Z; Senior Investigator Award 097803/Z/11/Z) and the National Institutes of Health (R21 HD088731).

The authors report no financial or other conflicts of interest.

Columbia University's Mortimer B. Zuckerman Mind Brain Behavior Institute brings together an extraordinary group of world-class scientists and scholars to pursue the most urgent and exciting challenge of our time: understanding the brain and mind. A deeper understanding of the brain promises to transform human health and society. From effective treatments for disorders like Alzheimer's, Parkinson's, depression and autism to advances in fields as fundamental as computer science, economics, law, the arts and social policy, the potential for humanity is staggering. To learn more, visit: zuckermaninstitute.columbia.edu.

The Zuckerman Institute at Columbia University

Related Brain Articles from Brightsurf:

Glioblastoma nanomedicine crosses into brain in mice, eradicates recurring brain cancer
A new synthetic protein nanoparticle capable of slipping past the nearly impermeable blood-brain barrier in mice could deliver cancer-killing drugs directly to malignant brain tumors, new research from the University of Michigan shows.

Children with asymptomatic brain bleeds as newborns show normal brain development at age 2
A study by UNC researchers finds that neurodevelopmental scores and gray matter volumes at age two years did not differ between children who had MRI-confirmed asymptomatic subdural hemorrhages when they were neonates, compared to children with no history of subdural hemorrhage.

New model of human brain 'conversations' could inform research on brain disease, cognition
A team of Indiana University neuroscientists has built a new model of human brain networks that sheds light on how the brain functions.

Human brain size gene triggers bigger brain in monkeys
Dresden and Japanese researchers show that a human-specific gene causes a larger neocortex in the common marmoset, a non-human primate.

Unique insight into development of the human brain: Model of the early embryonic brain
Stem cell researchers from the University of Copenhagen have designed a model of an early embryonic brain.

An optical brain-to-brain interface supports information exchange for locomotion control
Chinese researchers established an optical BtBI that supports rapid information transmission for precise locomotion control, thus providing a proof-of-principle demonstration of fast BtBI for real-time behavioral control.

Transplanting human nerve cells into a mouse brain reveals how they wire into brain circuits
A team of researchers led by Pierre Vanderhaeghen and Vincent Bonin (VIB-KU Leuven, Université libre de Bruxelles and NERF) showed how human nerve cells can develop at their own pace, and form highly precise connections with the surrounding mouse brain cells.

Brain scans reveal how the human brain compensates when one hemisphere is removed
Researchers studying six adults who had one of their brain hemispheres removed during childhood to reduce epileptic seizures found that the remaining half of the brain formed unusually strong connections between different functional brain networks, which potentially help the body to function as if the brain were intact.

Alcohol byproduct contributes to brain chemistry changes in specific brain regions
Study of mouse models provides clear implications for new targets to treat alcohol use disorder and fetal alcohol syndrome.

Scientists predict the areas of the brain to stimulate transitions between different brain states
Using a computer model of the brain, Gustavo Deco, director of the Center for Brain and Cognition, and Josephine Cruzat, a member of his team, together with a group of international collaborators, have developed an innovative method published in Proceedings of the National Academy of Sciences on Sept.

Read More: Brain News and Brain Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.