Moving cameras talk to each other to identify, track pedestrians

November 12, 2014

It's not uncommon to see cameras mounted on store ceilings, propped up in public places or placed inside subways, buses and even on the dashboards of cars.

Cameras record our world down to the second. This can be a powerful surveillance tool on the roads and in buildings, but it's surprisingly hard to sift through vast amounts of visual data to find pertinent information - namely, making a split-second identification and understanding a person's actions and behaviors as recorded sequentially by cameras in a variety of locations.

Now, University of Washington electrical engineers have developed a way to automatically track people across moving and still cameras by using an algorithm that trains the networked cameras to learn one another's differences. The cameras first identify a person in a video frame, then follow that same person across multiple camera views.

"Tracking humans automatically across cameras in a three-dimensional space is new," said lead researcher Jenq-Neng Hwang, a UW professor of electrical engineering. "As the cameras talk to each other, we are able to describe the real world in a more dynamic sense."

Hwang and his research team presented their results last month in Qingdao, China, at the Intelligent Transportation Systems Conference sponsored by the Institute of Electrical and Electronics Engineers, or IEEE.

Imagine a typical GPS display that maps the streets, buildings and signs in a neighborhood as your car moves forward, then add humans to the picture. With the new technology, a car with a mounted camera could take video of the scene, then identify and track humans and overlay them into the virtual 3-D map on your GPS screen. The UW researchers are developing this to work in real time, which could help pick out people crossing in busy intersections, or track a specific person who is dodging the police.

"Our idea is to enable the dynamic visualization of the realistic situation of humans walking on the road and sidewalks, so eventually people can see the animated version of the real-time dynamics of city streets on a platform like Google Earth," Hwang said.

Hwang's research team in the past decade has developed a way for video cameras - from the most basic models to high-end devices - to talk to each other as they record different places in a common location. The problem with tracking a human across cameras of non-overlapping fields of view is that a person's appearance can vary dramatically in each video because of different perspectives, angles and color hues produced by different cameras.

The researchers overcame this by building a link between the cameras. Cameras first record for a couple of minutes to gather training data, systematically calculating the differences in color, texture and angle between a pair of cameras for a number of people who walk into the frames in a fully unsupervised manner without human intervention.

After this calibration period, an algorithm automatically applies those differences between cameras and can pick out the same people across multiple frames, effectively tracking them without needing to see their faces.

The research team has tested the ability of static and moving cameras to detect and track pedestrians on the UW campus in multiple scenarios. In one experiment, graduate students mounted cameras in their cars to gather data, then applied the algorithms to successfully pick out humans and follow them in a three-dimensional space.

They also installed the tracking system on cameras placed inside a robot and a flying drone, allowing the robot and drone to follow a person, even when the instruments came across obstacles that blocked the person from view.

The linking technology can be used anywhere, as long as the cameras can talk over a wireless network and upload data to the cloud.

This detailed visual record could be useful for security and surveillance, monitoring for unusual behavior or tracking a moving suspect. But it also tells store owners and business proprietors useful information and statistics about consumers' moving patterns. A store owner could, for example, use a tracking system to watch a shopper's movements in the store, taking note of her interests. Then, a coupon or deal for a particular product could be displayed on a nearby screen or pushed to the shopper's phone - in an instant.

Leveraging the visual data produced by our physical actions and movements might, in fact, become the next way in which we receive marketing, advertisements and even helpful tools for our everyday lives.

Inevitably, people will have privacy concerns, Hwang said, and the information extracted from cameras could be encrypted before it's sent to the cloud.

"Cameras and recording won't go away. We might as well take advantage of that fact and extract more useful information for the benefit of the community," he added.
Co-authors are Kuan-Hui Lee, a UW doctoral student in electrical engineering, and Greg Okopal and James Pitton, engineers at the UW Applied Physics Laboratory.

The research was funded by the Electronics and Telecommunications Research Institute of Korea and the Applied Physics Laboratory.

For more information, contact Hwang at or 206-685-1603.

Project website and related paper:


University of Washington

Related Tracking Articles from Brightsurf:

Advancing the accurate tracking of energy poverty
IIASA researchers have developed a novel measurement framework to track energy poverty that better aligns with the services people lack rather than capturing the mere absence of physical connections to a source of electricity.

Using infrared eye tracking to study infant behavior
Researchers have adapted infrared technology to automate the tracking of visual tasks in infants in order to more accurately measure memory and cognitive behavior.

Data Security in Website Tracking
Tracking of our browsing behavior is part of the daily routine of Internet use.

Tracking Australia's gigantic carnivorous dinosaurs
North America had the T. rex, South America had the Giganotosaurus and Africa the Spinosaurus - now evidence shows Australia had gigantic predatory dinosaurs.

A next-generation sensor network for tracking small animals
A newly developed wireless biologging network (WBN) enables high-resolution tracking of small animals, according to a study published April 2 in the open-access journal PLOS Biology by Simon Ripperger of the Leibniz Institute for Evolution and Biodiversity Science, and colleagues.

Tracking adeno-associated virus capsid evolution
Researchers have used high-throughput screening of adeno-associated viral (AAV) vector capsid libraries to maximize the likelihood of obtaining AAV variants with desired properties.

Tracking communication networks and the diffusion of information
Tracking communication and movement in this hyperconnected world can seem overwhelming.

Turtle tracking reveals key feeding grounds
Loggerhead turtles feed in the same places year after year - meaning key locations should be protected, researchers say.

Building a better breast with eye-tracking technology
What makes the female breast attractive? The answer is subjective, of course.

Lead isotopes a new tool for tracking coal ash
Duke University scientists have developed a forensic tracer that uses lead isotopes to detect and measure coal fly ash in dust, soil and sediments.

Read More: Tracking News and Tracking Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to