Nav: Home

Team dramatically reduces image analysis times using deep learning, other approaches

June 29, 2020

WOODS HOLE, Mass. - A picture is worth a thousand words -but only when it's clear what it depicts. And therein lies the rub in making images or videos of microscopic life. While modern microscopes can generate huge amounts of image data from living tissues or cells within a few seconds, extracting meaningful biological information from that data can take hours or even weeks of laborious analysis.

To loosen this major bottleneck, a team led by MBL Fellow Hari Shroff has devised deep-learning and other computational approaches that dramatically reduce image-analysis time by orders of magnitude -- in some cases, matching the speed of data acquisition itself. They report their results this week in Nature Biotechnology.

"It's like drinking from a firehose without being able to digest what you're drinking," says Shroff of the common problem of having too much imaging data and not enough post-processing power. The team's improvements, which stem from an ongoing collaboration at the Marine Biological Laboratory (MBL), speed up image analysis in three major ways.

First, imaging data off the microscope is typically corrupted by blurring. To lessen the blur, an iterative "deconvolution" process is used. The computer goes back and forth between the blurred image and an estimate of the actual object, until it reaches convergence on a best estimate of the real thing.

By tinkering with the classic algorithm for deconvolution, Shroff and co-authors accelerated deconvolution by more than 10-fold. Their improved algorithm is widely applicable "to almost any fluorescence microscope," Shroff says. "It's a strict win, we think. We've released the code and other groups are already using it."

Next, they addressed the problem of 3D registration: aligning and fusing multiple images of an object taken from different angles. "It turns out that it takes much longer to register large datasets, like for light-sheet microscopy, than it does to deconvolve them," Shroff says. They found several ways to accelerate 3D registration, including moving it to the computer's graphics processing unit (GPU). This gave them a 10- to more than 100-fold improvement in processing speed over using the computer's central processing unit (CPU).

"Our improvements in registration and deconvolution mean that for datasets that fit onto a graphics card, image analysis can in principle keep up with the speed of acquisition," Shroff says. "For bigger datasets, we found a way to efficiently carve them up into chunks, pass each chunk to the GPU, do the registration and deconvolution, and then stitch those pieces back together. That's very important if you want to image large pieces of tissue, for example, from a marine animal, or if you are clearing an organ to make it transparent to put on the microscope. Some forms of large microscopy are really enabled and sped up by these two advances."

Lastly, the team used deep learning to accelerate "complex deconvolution" - intractable datasets in which the blur varies significantly in different parts of the image. They trained the computer to recognize the relationship between badly blurred data (the input) and a cleaned, deconvolved image (the output). Then they gave it blurred data it hadn't seen before. "It worked really well; the trained neural network could produce deconvolved results really fast," Shroff says. "That's where we got thousands-fold improvements in deconvolution speed."

While the deep learning algorithms worked surprisingly well, "it's with the caveat that they are brittle," Shroff says. "Meaning, once you've trained the neural network to recognize a type of image, say a cell with mitochondria, it will deconvolve those images very well. But if you give it an image that is a bit different, say the cell's plasma membrane, it produces artifacts. It's easy to fool the neural network." An active area of research is creating neural networks that work in a more generalized way.

"Deep learning augments what is possible," Shroff says. "It's a good tool for analyzing datasets that would be difficult any other way."
-end-
This paper stems from an ongoing collaboration in the MBL Whitman Center between MBL Fellows Hari Shroff, senior investigator at the National Institute of Biomedical Imaging and Bioengineering; Patrick La Rivière, professor at the University of Chicago; and Daniel Colón-Ramos, professor at Yale Medical School. First author Min Guo is postdoctoral fellow in Shroff's lab and a former teaching assistant for Shroff in the MBL's Optical Microscopy and Imaging in the Biological Sciences course.

The MBL is opening an Image Analysis Laboratory to assist scientists with their analysis of large and complex datasets. More information is here.

The Marine Biological Laboratory (MBL) is dedicated to scientific discovery - exploring fundamental biology, understanding marine biodiversity and the environment, and informing the human condition through research and education. Founded in Woods Hole, Massachusetts in 1888, the MBL is a private, nonprofit institution and an affiliate of the University of Chicago.

Marine Biological Laboratory

Related Learning Articles:

Getting kids moving, and learning
Children are set to move more, improve their skills, and come up with their own creative tennis games with the launch of HomeCourtTennis, a new initiative to assist teachers and coaches with keeping kids active while at home.
How expectations influence learning
During learning, the brain is a prediction engine that continually makes theories about our environment and accurately registers whether an assumption is true or not.
Technology in higher education: learning with it instead of from it
Technology has shifted the way that professors teach students in higher education.
Learning is optimized when we fail 15% of the time
If you're always scoring 100%, you're probably not learning anything new.
School spending cuts triggered by great recession linked to sizable learning losses for learning losses for students in hardest hit areas
Substantial school spending cuts triggered by the Great Recession were associated with sizable losses in academic achievement for students living in counties most affected by the economic downturn, according to a new study published today in AERA Open, a peer-reviewed journal of the American Educational Research Association.
Lessons in learning
A new Harvard study shows that, though students felt like they learned more from traditional lectures, they actually learned more when taking part in active learning classrooms.
Learning to look
A team led by JGI scientists has overhauled the perception of inovirus diversity.
Sleep readies synapses for learning
Synapses in the hippocampus are larger and stronger after sleep deprivation, according to new research in mice published in JNeurosci.
Learning from experience is all in the timing
Animals learn the hard way which sights, sounds, and smells are relevant to survival.
Learning language
When it comes to learning a language, the left side of the brain has traditionally been considered the hub of language processing.
More Learning News and Learning Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Meditations on Loneliness
Original broadcast date: April 24, 2020. We're a social species now living in isolation. But loneliness was a problem well before this era of social distancing. This hour, TED speakers explore how we can live and make peace with loneliness. Guests on the show include author and illustrator Jonny Sun, psychologist Susan Pinker, architect Grace Kim, and writer Suleika Jaouad.
Now Playing: Science for the People

#565 The Great Wide Indoors
We're all spending a bit more time indoors this summer than we probably figured. But did you ever stop to think about why the places we live and work as designed the way they are? And how they could be designed better? We're talking with Emily Anthes about her new book "The Great Indoors: The Surprising Science of how Buildings Shape our Behavior, Health and Happiness".
Now Playing: Radiolab

The Third. A TED Talk.
Jad gives a TED talk about his life as a journalist and how Radiolab has evolved over the years. Here's how TED described it:How do you end a story? Host of Radiolab Jad Abumrad tells how his search for an answer led him home to the mountains of Tennessee, where he met an unexpected teacher: Dolly Parton.Jad Nicholas Abumrad is a Lebanese-American radio host, composer and producer. He is the founder of the syndicated public radio program Radiolab, which is broadcast on over 600 radio stations nationwide and is downloaded more than 120 million times a year as a podcast. He also created More Perfect, a podcast that tells the stories behind the Supreme Court's most famous decisions. And most recently, Dolly Parton's America, a nine-episode podcast exploring the life and times of the iconic country music star. Abumrad has received three Peabody Awards and was named a MacArthur Fellow in 2011.