Nav: Home

New method improves measurement of animal behaviour using deep learning

October 01, 2019

A new toolkit goes beyond existing machine learning methods by measuring body posture in animals with high speed and accuracy. Developed by researchers from the Centre for the Advanced Study of Collective Behaviour at the University of Konstanz and the Max Planck Institute of Animal Behavior, this deep learning toolkit, called DeepPoseKit, combines previous methods for pose estimation with state-of-the-art developments in computer science. These newly-developed deep learning methods can correctly measure body posture from previously-unseen images after being trained with only 100 examples and can be applied to study wild animals in challenging field settings. Published today in the open access journal eLife, the study is advancing the field of animal behaviour with next-generation tools while at the same time providing an accessible system for non-experts to easily apply machine learning to their behavioural research.

Animals must interact with the physical world in order to survive and reproduce, and studying their behaviour can reveal the solutions that have evolved for achieving these ultimate goals. Yet behaviour is hard to define just by observing it directly: biases and limited processing power of human observers inhibits the quality and resolution of behavioural data that can be collected from animals.

Machine learning has changed that. Various tools have been developed in recent years that allow researchers to automatically track the locations of animals' body parts directly from images or videos - without the need for applying intrusive markers on animals or manually scoring behaviour. These methods, however, have shortcomings that limit performance. "Existing tools for measuring body posture with deep learning were either slower and more accurate or faster and less accurate - but we wanted to achieve the best of both worlds." says lead author Jake Graving, a graduate student in the Max Planck Institute of Animal Behavior.

In the new study, researchers present an approach that overcomes this speed-accuracy trade-off. These new methods use an efficient, state-of-the-art deep learning model to detect body parts in images, and a fast algorithm for calculating the location of these detected body parts with high accuracy. Results from this study also demonstrate that these new methods can be applied across species and experimental conditions - from flies, locusts, and mice in controlled laboratory settings to herds of zebras interacting in the wild. Dr. Blair Costelloe, co-author of the paper, who studies zebras in Kenya says: "The posture data we can now collect for the zebras using DeepPoseKit allows us to know exactly what each individual is doing in the group and how they interact with the surrounding environment. In contrast, existing technologies like GPS will reduce this complexity down to a single point in space, which limits the types of questions you can answer."

Due to its high performance and easy-to-use software interface (the code is publicly available on Github,, the researchers say that DeepPoseKit can immediately benefit scientists across a variety of fields - such as neuroscience, psychology, and ecology - and levels of expertise. Work on this topic can also have applications that affect our daily lives, such as improving similar algorithms for gesture recognition used on smartphones or diagnosing and monitoring movement-related diseases in humans and animals.

"In just a few short years deep learning has gone from being a sort of niche, hard-to-use method to one of the most democratized and widely-used software tools in the world," says Iain Couzin, senior author on the paper who leads the Centre for the Advanced Study of Collective Behaviour at the University of Konstanz and the Department of Collective Behaviour at the Max Planck Institute of Animal Behavior. "Our hope is that we can contribute to behavioural research by developing easy-to-use, high-performance tools that anybody can use." Tools like these are important for studying behaviour because, as Graving puts it: "They allow us to start with first principles, or 'how is the animal moving its body through space?', rather than subjective definitions of what constitutes a behaviour. From there we can begin to apply mathematical models to the data and develop general theories that help us to better understand how individuals and groups of animals adaptively organize their behaviour."

- Jacob M. Graving, Daniel Chae, Hemal Naik, Liang Li, Benjamin Koger, Blair R. Costelloe, Iain D. Couzin (2019) DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. eLife. DOI:


- All authors with the exception of Daniel Chae are part of the new Cluster of Excellence "Centre for the Advanced Study of Collective Behaviour" at the University of Konstanz and the Department of Collective Behaviour, Max Planck Institute of Animal Behavior in Konstanz.

- This project received funding from the European Union's Horizon 2020 research and innovation programme (under the Marie Sklodowska-Curie grant agreement) and the National Science Foundation.

Note to editors:

You can download a photo here:

Caption: A deep learning toolkit, called DeepPoseKit, can automatically detect animal body parts directly from images or video with high speed and accuracy - without attaching physical markers. The method can be used for animals in laboratory settings (e.g. flies and locusts) or in the wild (e.g. zebras).

Credit: Jake Graving

University of Konstanz

Related Behaviour Articles:

Vibes before it bites: 10 types of defensive behaviour for the false coral snake
The False Coral Snake (Oxyrhopus rhombifer) may be capable of recognising various threat levels and demonstrates ten different defensive behaviours, seven of which are registered for the first time for the species.
Unwanted behaviour in dogs is common, with great variance between breeds
All dog breeds have unwanted behaviour, such as noise sensitivity, aggressiveness and separation anxiety, but differences in frequency between breeds are great.
The Lancet Psychiatry: Life-course-persistent antisocial behaviour may be associated with differences in brain structure
Individuals who exhibit life-course-persistent antisocial behaviour - for example, stealing, aggression and violence, bullying, lying, or repeated failure to take care of work or school responsibilities - may have thinner cortex and smaller surface area in regions of the brain previously implicated in studies of antisocial behaviour more broadly, compared to individuals without antisocial behaviour, according to an observational study of 672 participants published in The Lancet Psychiatry journal.
World-first studies reveal occurrence of 'chew and spit' eating behaviour
A landmark study into the prevalence of the disordered eating behaviour known as 'chew and spit' has revealed concerning levels of such episodes among teenagers.
Collective behavior 480 million years ago
Researchers studied fossilized Moroccan Ampyx trilobites, which lived 480 million years ago and showed that the trilobites had probably been buried in their positions -- all oriented in the same direction.
New method improves measurement of animal behaviour using deep learning
Konstanz researchers develop deep learning toolkit for high-speed measurement of body posture in animals.
Impulsive behaviour linked to sleep and screen time, CHEO study finds
A paper published today in Pediatrics suggests that children and youth who do not sleep enough and use screens more than recommended are more likely to act impulsively and make poorer decisions.
Prenatal parental stress linked to behaviour problems in toddlers
Expectant parents' emotional struggles predict emotional and behavioural problems in 2-year-olds, new research shows.
Transformer cells: Shaping cellular 'behaviour'
Scientists from the Sechenov University, conjointly with their fellow Chinese and American researchers, have examined the latest advances in the use of skeletal muscle progenitor cells, specifying the core challenges inherent to the applicability of MPCs in cell therapy, and outlining the most promising breakthrough technologies.
Heritable behavioral differences between cat breeds
Cat breeds differ from each other in behavior with regard to activity, aggressiveness, shyness, sociability and stereotypical behavior.
More Behaviour News and Behaviour Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: The Biology Of Sex
Original broadcast date: May 8, 2020. Many of us were taught biological sex is a question of female or male, XX or XY ... but it's far more complicated. This hour, TED speakers explore what determines our sex. Guests on the show include artist Emily Quinn, journalist Molly Webster, neuroscientist Lisa Mosconi, and structural biologist Karissa Sanbonmatsu.
Now Playing: Science for the People

#569 Facing Fear
What do you fear? I mean really fear? Well, ok, maybe right now that's tough. We're living in a new age and definition of fear. But what do we do about it? Eva Holland has faced her fears, including trauma and phobia. She lived to tell the tale and write a book: "Nerve: Adventures in the Science of Fear".
Now Playing: Radiolab

The Wubi Effect
When we think of China today, we think of a technological superpower. From Huweai and 5G to TikTok and viral social media, China is stride for stride with the United States in the world of computing. However, China's technological renaissance almost didn't happen. And for one very basic reason: The Chinese language, with its 70,000 plus characters, couldn't fit on a keyboard.  Today, we tell the story of Professor Wang Yongmin, a hard headed computer programmer who solved this puzzle and laid the foundation for the China we know today. This episode was reported and produced by Simon Adler with reporting assistance from Yang Yang. Special thanks to Martin Howard. You can view his renowned collection of typewriters at: Support Radiolab today at