Do neural networks dream visual illusions?

November 20, 2020

Aconvolutional neural network is a type of artificial neural network in which the neurons are organized into receptive fields in a very similar way to neurons in the visual cortex of a biological brain. Today, convolutional neural networks (CNNs) are found in a variety of autonomous systems (for example, face detection and recognition, autonomous vehicles, etc.). This type of network is highly effective in many artificial vision tasks, such as in image segmentation and classification, along with many other applications.

Convolutional networks were inspired by the behaviour of the human visual system, particularly its basic structure formed by the concatenation of compound modules comprising a linear operation followed by a non-linear operation. A study published in the advanced online edition of the journal Vision Research examines the phenomenon of visual illusions in convolutional networks compared to their effect on human vision. A study by Alexander Gómez Vila, Adrian Martín, Javier Vázquez-Corral and Marcelo Bertalmío, members of the Department of Information and Communication Technologies (DTIC) with the participation of the researcher Jesús Malo of the University of Valencia.

"Because of this connection of CNNs with our visual system, in this paper we wanted to see if convolutional networks suffer from similar problems to our visual system. Hence, we focused on visual illusions. Visual illusions are images that our brain perceives differently from how they actually are", explains Gómez Vila, first author of the study.

In their study, the authors trained CNNs for simple tasks also performed by human vision, such as denoising and deblurring. What they observed is that these CNNs trained under these experimental conditions are also "deceived" by brightness and colour visual illusions in the same way that visual illusions deceive humans.

Furthermore, as Gómez Villa explains, "for our work we also analyse when such illusions cause responses in the network that are not as physically expected, but neither do they match with human perception", that is to say, cases in which CNNs obtain a different optical illusion than the illusion that humans would perceive.

The results of this study are consistent with the long-standing hypothesis that considers low-level visual illusions as a by-product of the optimization to natural environments (that a human sees in their everyday). Meanwhile, these results highlight the limitations and differences between the human visual system and CNNs artificial neural networks.
-end-
Related work:

Alexander Gómez Vila, Adrian Martín, Javier Vázquez-Corral, Marcelo Bertalmío, Jesús Malo (2020), "Color illusions also deceive CNNs for low-level vision tasks: Analysis and Implications", september, Vision Research, advanced edition online.
https://doi.org/10.1016/j.visres.2020.07.010

Universitat Pompeu Fabra - Barcelona

Related Visual System Articles from Brightsurf:

Visual working memory is hierarchically structured
Researchers from HSE University and the University of California San Diego, Igor Utochkin and Timothy Brady, have found new evidence of hierarchical encoding of images in visual working memory.

The mystery of visual stability
We move our eyes several times per second. These fast eye movements, called saccades, create large image shifts on the retina -- making our visual system work hard to maintain a stable perceptual world.

Why visual perception is a decision process
A popular theory in neuroscience called predictive coding proposes that the brain produces all the time expectations that are compared with incoming information.

Visual impairment among women and dementia risk
Whether visual impairment is a risk factor for dementia was the focus of this observational study that included 1,000 older women who are participants in the Women's Health Initiative studies.

VR is not suited to visual memory?!
Toyohashi university of technology researcher and a research team at Tokyo Denki University have found that virtual reality (VR) may interfere with visual memory.

Dartmouth study finds conscious visual perception occurs outside the visual system
A Dartmouth study finds that the conscious perception of visual location occurs in the frontal lobes of the brain, rather than in the visual system in the back of the brain.

Learning to read boosts the visual brain
How does learning to read change our brain? Does reading take up brain space dedicated to seeing objects such as faces, tools or houses?

How brain rhythms organize our visual perception
Imagine that you are watching a crowded hang-gliding competition, keeping track of a red and orange glider's skillful movements.

Seeing it both ways: Visual perspective in memory
Think of a memory from your childhood. Are you seeing the memory through your own eyes, or can you see yourself, while viewing that child as if you were an observer?

Using visual imagery to find your true passions
You may think you know what you like -- how to spend your time or what profession to pursue.

Read More: Visual System News and Visual System Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.