Nav: Home

This AI birdwatcher lets you 'see' through the eyes of a machine

October 31, 2019

DURHAM, N.C. -- It can take years of birdwatching experience to tell one species from the next. But using an artificial intelligence technique called deep learning, Duke University researchers have trained a computer to identify up to 200 species of birds from just a photo.

The real innovation, however, is that the A.I. tool also shows its thinking, in a way that even someone who doesn't know a penguin from a puffin can understand.

The team trained their deep neural network -- algorithms based on the way the brain works -- by feeding it 11,788 photos of 200 bird species to learn from, ranging from swimming ducks to hovering hummingbirds.

The researchers never told the network "this is a beak" or "these are wing feathers." Given a photo of a mystery bird, the network is able to pick out important patterns in the image and hazard a guess by comparing those patterns to typical species traits it has seen before.

Along the way it spits out a series of heat maps that essentially say: "This isn't just any warbler. It's a hooded warbler, and here are the features -- like its masked head and yellow belly -- that give it away."

Duke computer science Ph.D. student Chaofan Chen and undergraduate Oscar Li led the research, along with other team members of the Prediction Analysis Lab directed by Duke professor Cynthia Rudin.

They found their neural network is able to identify the correct species up to 84% of the time -- on par with some of its best-performing counterparts, which don't reveal how they are able to tell, say, one sparrow from the next.

Rudin says their project is about more than naming birds. It's about visualizing what deep neural networks are really seeing when they look at an image.

Similar technology is used to tag people on social networking sites, spot suspected criminals in surveillance cameras, and train self-driving cars to detect things like traffic lights and pedestrians.

The problem, Rudin says, is that most deep learning approaches to computer vision are notoriously opaque. Unlike traditional software, deep learning software learns from the data without being explicitly programmed. As a result, exactly how these algorithms 'think' when they classify an image isn't always clear.

Rudin and her colleagues are trying to show that A.I. doesn't have to be that way. She and her lab are designing deep learning models that explain the reasoning behind their predictions, making it clear exactly why and how they came up with their answers. When such a model makes a mistake, its built-in transparency makes it possible to see why.

For their next project, Rudin and her team are using their algorithm to classify suspicious areas in medical images like mammograms. If it works, their system won't just help doctors detect lumps, calcifications and other symptoms that could be signs of breast cancer. It will also show which parts of the mammogram it's homing in on, revealing which specific features most resemble the cancerous lesions it has seen before in other patients.

In that way, Rudin says, their network is designed to mimic the way doctors make a diagnosis. "It's case-based reasoning," Rudin said. "We're hoping we can better explain to physicians or patients why their image was classified by the network as either malignant or benign."
-end-
The team is presenting a paper on their findings at the Thirty-third Conference on Neural Information Processing Systems (NeurIPS 2019) in Vancouver on December 12.

Other authors of this study include Daniel Tao and Alina Barnett of Duke and Jonathan Su at MIT Lincoln Laboratory.

CITATION: "This Looks Like That: Deep Learning for Interpretable Image Recognition," Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Jonathan Su and Cynthia Rudin. Electronic Proceedings of the Neural Information Processing Systems Conference. December 12, 2019.

Duke University

Related Birds Articles:

How do birds understand 'foreign' calls?
New research from Kyoto University show that the coal tit (Periparus ater) can eavesdrop and react to the predatory warning calls of the Japanese tit (Parus minor) and evokes a visual image of the predator in their mind
How birds evolved big brains
An international team of evolutionary biologists and paleontologists have reconstructed the evolution of the avian brain using a massive dataset of brain volumes from dinosaurs, extinct birds like Archaeopteryx and the great auk, and modern birds.
Microelectronics for birds
Ornithologists and physicists from St Petersburg University have conducted an interdisciplinary study together with colleagues from Sechenov Institute of Evolutionary Physiology and Biochemistry of the Russian Academy of Sciences and the Biological Station Rybachy of the Zoological Institute of the Russian Academy of Sciences.
Birds of a feather better not together
A new study of North American birds from Washington University in St.
Not-so-dirty birds? Not enough evidence to link wild birds to food-borne illness
Despite the perception that wild birds in farm fields can cause food-borne illness, a WSU study has found little evidence linking birds to E. coli, Salmonella and Campylobacter outbreaks.
Birds are shrinking as the climate warms
After 40 years of collecting birds that ran into Chicago buildings, scientists have been able to show that the birds have been shrinking as the climate's warmed up.
Diving birds follow each other when fishing
Diving seabirds watch each other to work out when to dive, new research shows.
Why do birds migrate at night?
Researchers found migratory birds maximize how much light they get from their environment, so they can migrate even at night. 
How can robots land like birds?
Birds can perch on a wide variety of surfaces, thick or thin, rough or slick.
Is wildfire management 'for the birds?'
Spotted owl populations are in decline all along the West Coast, and as climate change increases the risk of large and destructive wildfires in the region, these iconic animals face the real threat of losing even more of their forest habitat.
More Birds News and Birds Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Meditations on Loneliness
Original broadcast date: April 24, 2020. We're a social species now living in isolation. But loneliness was a problem well before this era of social distancing. This hour, TED speakers explore how we can live and make peace with loneliness. Guests on the show include author and illustrator Jonny Sun, psychologist Susan Pinker, architect Grace Kim, and writer Suleika Jaouad.
Now Playing: Science for the People

#565 The Great Wide Indoors
We're all spending a bit more time indoors this summer than we probably figured. But did you ever stop to think about why the places we live and work as designed the way they are? And how they could be designed better? We're talking with Emily Anthes about her new book "The Great Indoors: The Surprising Science of how Buildings Shape our Behavior, Health and Happiness".
Now Playing: Radiolab

The Third. A TED Talk.
Jad gives a TED talk about his life as a journalist and how Radiolab has evolved over the years. Here's how TED described it:How do you end a story? Host of Radiolab Jad Abumrad tells how his search for an answer led him home to the mountains of Tennessee, where he met an unexpected teacher: Dolly Parton.Jad Nicholas Abumrad is a Lebanese-American radio host, composer and producer. He is the founder of the syndicated public radio program Radiolab, which is broadcast on over 600 radio stations nationwide and is downloaded more than 120 million times a year as a podcast. He also created More Perfect, a podcast that tells the stories behind the Supreme Court's most famous decisions. And most recently, Dolly Parton's America, a nine-episode podcast exploring the life and times of the iconic country music star. Abumrad has received three Peabody Awards and was named a MacArthur Fellow in 2011.