Smart algorithm cleans up images by searching for clues buried in noise

January 26, 2021

To enter the world of the fantastically small, the main currency is either a ray of light or electrons.

Strong beams, which yield clearer images, are damaging to specimens. On the other hand, weak beams can give noisy, low-resolution images.

In a new study published in , researchers at Texas A&M University describe a machine learning-based algorithm that can reduce graininess in low-resolution images and reveal new details that were otherwise buried within the noise.

"Images taken with low-powered beams can be noisy, which can hide interesting and valuable visual details of biological specimens," said Shuiwang Ji, associate professor in the Department of Computer Science and Engineering. "To solve this problem, we use a pure computational approach to create higher-resolution images, and we have shown in this study that we can improve the resolution up to an extent very similar to what you might obtain using a high beam."

Ji added that unlike other denoising algorithms that can only use information coming from a small patch of pixels within a low-resolution image, their smart algorithm can identify pixel patterns that may be spread across the entire noisy image, increasing its efficacy as a denoising tool.

Instead of solely relying on microscope hardware to improve the images' resolution, a technique known as augmented microscopy uses a combination of software and hardware to enhance the quality of images. Here, a regular image taken on a microscope is superimposed on a computer-generated digital image. This image processing method holds promise to not just cut down costs but also automatize medical image analysis and reveal details that the eye can sometimes miss.

Currently, a type of software based on a machine-learning algorithm called deep learning has been shown to be effective at removing the blurriness or noise in images. These algorithms can be visualized as consisting of many interconnected layers or processing steps that take in a low-resolution input image and generate a high-resolution output image.

In conventional deep-learning-based image processing techniques, the number and network between layers decide how many pixels in the input image contribute to the value of a single pixel in the output image. This value is immutable after the deep-learning algorithm has been trained and is ready to denoise new images. However, Ji said fixing the number for the input pixels, technically called the receptive field, limits the performance of the algorithm.

"Imagine a piece of specimen having a repeating motif, like a honeycomb pattern. Most deep-learning algorithms only use local information to fill in the gaps in the image created by the noise," Ji said. "But this is inefficient because the algorithm is, in essence, blind to the repeating pattern within the image since the receptive field is fixed. Instead, deep-learning algorithms need to have adaptive receptive fields that can capture the information in the overall image structure."

To overcome this hurdle, Ji and his students developed another deep-learning algorithm that can dynamically change the size of the receptive field. In other words, unlike earlier algorithms that can only aggregate information from a small number of pixels, their new algorithm, called global voxel transformer networks (GVTNets), can pool information from a larger area of the image if required.

When they analyzed their algorithm's performance against other deep-learning software, the researchers found that GVTNets required less training data and could denoise images better than other deep-learning algorithms. Furthermore, the high-resolution images obtained were comparable to those obtained using a high-energy light beam.

The researchers noted that their new algorithm can easily be adapted to other applications in addition to denoising, such as label-free fluorescence imaging and 3D to 2D conversions for computer graphics.

"Our research contributes to the emerging area of a smart microscopy, where artificial intelligence is seamlessly integrated into the microscope," Ji said. "Deep-learning algorithms such as ours will allow us to potentially transcend the physical limit posed by light that was not possible before. This can be extremely valuable for a myriad of applications, including clinical ones, like estimating the stage of cancer progression and distinguishing between cell types for disease prognosis."
Equal contributors to this research include Zhengyang Wang and Yaochen Xie from the computer science and engineering department.

This research is funded by the National Science Foundation, the National Institutes of Health and the Defense Advanced Research Projects Agency.

Video link:

Video caption: As the microscope scans a 3D piece of biological tissue from top to bottom, low-resolution images (left, input) are transformed into high-resolution images (middle, prediction) in real time by the deep-learning algorithm GVTNets. For comparison are images (right, ground truth) obtained using a high-energy light beam, which is better for getting more resolved images but is damaging to specimens.

Video credit: Yaochen Xie, Texas A&M Engineering

Texas A&M University

Related Light Articles from Brightsurf:

Light from rare earth: new opportunities for organic light-emitting diodes
Efficient and stable blue OLED is still a challenge due to the lack of emitter simultaneously with high efficiency and short excited-state lifetime.

Guiding light: Skoltech technology puts a light-painting drone at your fingertips
Skoltech researchers have designed and developed an interface that allows a user to direct a small drone to light-paint patterns or letters through hand gestures.

Painting with light: Novel nanopillars precisely control intensity of transmitted light
By shining white light on a glass slide stippled with millions of tiny titanium dioxide pillars, researchers at the National Institute of Standards and Technology (NIST) and their collaborators have reproduced with astonishing fidelity the luminous hues and subtle shadings of 'Girl With a Pearl Earring.'

Seeing the light: Researchers combine technologies for better light control
A new technology that can allow for better light control without requiring large, difficult-to-integrate materials and structures has been developed by Penn State researchers.

A different slant of light
Giant clams manipulate light to assist their symbiotic partner.

New light for plants
Scientists from ITMO in collaboration with their colleagues from Tomsk Polytechnic University came up with an idea to create light sources from ceramics with the addition of chrome: the light from such lamps offers not just red but also infrared (IR) light, which is expected to have a positive effect on plants' growth.

Scientists use light to accelerate supercurrents, access forbidden light, quantum world
Iowa State's Jigang Wang continues to explore using light waves to accelerate supercurrents to access the unique and potentially useful properties of the quantum world.

The power of light
As COVID-19 continues to ravage global populations, the world is singularly focused on finding ways to battle the novel coronavirus.

Seeing the light: MSU research finds new way novae light up the sky
An international team of astronomers from 40 institutes across 17 countries found that shocks cause most the brightness in novae.

Seeing the light: Astronomers find new way novae light up the sky
An international team of researchers, in a paper published today in Nature Astronomy, highlights a new way novae light up the sky: this is shocks from explosions that create the novae that cause most of the their brightness.

Read More: Light News and Light Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to