York University vision scientists disprove 60-year-old perception theory

October 24, 2019

TORONTO, October 24, 2019-Vision researchers at York University have disproved a long-standing theory of how the human vision system processes images, using computational models and human experiments.

A team led by John Tsotsos, professor in the Department of Electrical Engineering and Computer Science at the Lassonde School of Engineering, found that the human brain does not select interesting portions of an image to process preferentially, as the highly influential 1958 theory of Donald Broadbent proposed.

For psychologist Broadbent, the interesting image portions are those that have relevance to why you are looking at a scene in the first place, or are novel items that immediately grab our attention. Broadbent's Theory of Early Selection, which has a modern counterpart in the Saliency Map Theory of Christof Koch and Shimon Ullman published in 1985, claims that these interesting regions are processed by the brain one at a time, in order of their salience, which is a numerical score of how interesting a region is. There are now hundreds of saliency algorithms, rooted in the work of Koch and Ullman, to accomplish such a ranking.

Tsotsos' team found however, that salience is not needed at all for the simple task of quickly deciding what an image depicts. Moreover, none of the current algorithms within artificial intelligence (AI) for this task come close to matching human performance, which is remarkably good. On the other hand, salience computation does play a primary role in determining where humans move their eyes, and it is eye movement that selects portions of a scene to process next.

"Our study looks at this for vision and tests the leading algorithms that compute the saliency measure and asks the question 'are those algorithms performing at the same level as humans do on these images'? For example, if the task is to determine if there is a cat in a scene, does the saliency algorithm pick out the cat correctly? The study showed that these algorithms are far from doing as well as humans," said Tsotsos.

To further test existing algorithms, the team conducted additional experiments with 17 subjects from 25 to 34 years old. In one of the replicated experiments, participants were shown 2000 colour images. The subjects were not familiar with the images and viewed each image with and without animals, only once. The images were then manipulated in such a way so that only the most central parts of the retina that have the highest resolution would see what was in the image and see nothing in the periphery. Participants were asked to look at the centre of each photograph for 20 seconds before it disappeared. The participants were able to correctly identify if an animal was present in the picture or not.

Tsotsos says this finding has important ramifications for our understanding of human vision and human visual processing especially for diagnosing vision pathologies, such as aspects of autism.

"When you want to diagnose issues in vision, you're basing on it how the healthy visual processing system should work. What we've done with this study is added a piece of the puzzle to how the 'healthy' system works which then would change how you compare an anomaly in order to be able to diagnose it."

Tsotsos adds that this piece of the puzzle could also be useful in building new models and improving current ones for autonomous driving or security applications.
This study, Rapid Visual Categorization is not Guided by Early Salience-Based Selection is published today in PLOS ONE.

York University champions new ways of thinking that drive teaching and research excellence. Our students receive the education they need to create big ideas that make an impact on the world. Meaningful and sometimes unexpected careers result from cross-disciplinary programming, innovative course design and diverse experiential learning opportunities. York students and graduates push limits, achieve goals and find solutions to the world's most pressing social challenges, empowered by a strong community that opens minds. York U is an internationally recognized research university - our 11 faculties and 25 research centres have partnerships with 200+ leading universities worldwide. Located in Toronto, York is the third largest university in Canada, with a strong community of 53,000 students, 7,000 faculty and administrative staff, and more than 300,000 alumni.

York U's fully bilingual Glendon Campus is home to Southern Ontario's Centre of Excellence for French Language and Bilingual Postsecondary Education.

Media Contact: Anjum Nayyar, York University Media Relations, 437-242-1547, anayyar@yorku.ca

York University

Related Vision Articles from Brightsurf:

School-based vision screening programs found 1 in 10 kids had vision problems
A school-based vision screening program in kindergarten, shown to be effective at identifying untreated vision problems in 1 in 10 students, could be useful to implement widely in diverse communities, according to new research in CMAJ (Canadian Medical Association Journal) http://www.cmaj.ca/lookup/doi/10.1503/cmaj.191085.

Restoring vision by gene therapy
Latest scientific findings give hope for people with incurable retinal degeneration.

Vision loss influences perception of sound
People with severe vision loss can less accurately judge the distance of nearby sounds, potentially putting them more at risk of injury.

'Time is vision' after a stroke
University of Rochester researchers studied stroke patients who experienced vision loss and found that the patients retained some visual abilities immediately after the stroke but these abilities diminished gradually and eventually disappeared permanently after approximately six months.

Improving the vision of self-driving vehicles
There may be a better way for autonomous vehicles to learn how to drive themselves: by watching humans.

A new model of vision
MIT researchers have developed a computer model of face processing that could reveal how the brain produces richly detailed visual representations so quickly.

Vision may be the real cause of children's problems
Do you have poor motor skills or struggle to read, write or solve math problems?

Shark and ray vision comes into focus
Until now, little has been known about the evolution of vision in cartilaginous fishes, particularly sharks and their genetic cousins, the rays.

The birth of vision, from the retina to the brain
How do neurons differentiate to become individual components of the visual system?

Tracing the evolution of vision
The function of the visual photopigment rhodopsin and its action in the retina to facilitate vision is well understood.

Read More: Vision News and Vision Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.