Could your vacuum be listening to you?

November 17, 2020

A team of researchers demonstrated that popular robotic household vacuum cleaners can be remotely hacked to act as microphones.

The researchers--including Nirupam Roy, an assistant professor in the University of Maryland's Department of Computer Science--collected information from the laser-based navigation system in a popular vacuum robot and applied signal processing and deep learning techniques to recover speech and identify television programs playing in the same room as the device.

The research demonstrates the potential for any device that uses light detection and ranging (Lidar) technology to be manipulated for collecting sound, despite not having a microphone. This work, which is a collaboration with assistant professor Jun Han at the University of Singapore was presented at the Association for Computing Machinery's Conference on Embedded Networked Sensor Systems (SenSys 2020) on November 18, 2020.

"We welcome these devices into our homes, and we don't think anything about it," said Roy, who holds a joint appointment in the University of Maryland Institute for Advanced Computer Studies (UMIACS). "But we have shown that even though these devices don't have microphones, we can repurpose the systems they use for navigation to spy on conversations and potentially reveal private information."

The Lidar navigation systems in household vacuum bots shine a laser beam around a room and sense the reflection of the laser as it bounces off nearby objects. The robot uses the reflected signals to map the room and avoid collisions as it moves through the house.

Privacy experts have suggested that the maps made by vacuum bots, which are often stored in the cloud, pose potential privacy breaches that could give advertisers access to information about such things as home size, which suggests income level, and other lifestyle-related information. Roy and his team wondered if the Lidar in these robots could also pose potential security risks as sound recording devices in users' homes or businesses.

Sound waves cause objects to vibrate, and these vibrations cause slight variations in the light bouncing off an object. Laser microphones, used in espionage since the 1940s, are capable of converting those variations back into sound waves. But laser microphones rely on a targeted laser beam reflecting off very smooth surfaces, such as glass windows.

A vacuum Lidar, on the other hand, scans the environment with a laser and senses the light scattered back by objects that are irregular in shape and density. The scattered signal received by the vacuum's sensor provides only a fraction of the information needed to recover sound waves. The researchers were unsure if a vacuum bot's Lidar system could be manipulated to function as a microphone and if the signal could be interpreted into meaningful sound signals.

First, the researchers hacked a robot vacuum to show they could control the position of the laser beam and send the sensed data to their laptops through Wi-Fi without interfering with the device's navigation.

Next, they conducted experiments with two sound sources. One source was a human voice reciting numbers played over computer speakers and the other was audio from a variety of television shows played through a TV sound bar. Roy and his colleagues then captured the laser signal sensed by the vacuum's navigation system as it bounced off a variety of objects placed near the sound source. Objects included a trash can, cardboard box, takeout container and polypropylene bag--items that might normally be found on a typical floor.

The researchers passed the signals they received through deep learning algorithms that were trained to either match human voices or to identify musical sequences from television shows. Their computer system, which they call LidarPhone, identified and matched spoken numbers with 90% accuracy. It also identified television shows from a minute's worth of recording with more than 90% accuracy.

"This type of threat may be more important now than ever, when you consider that we are all ordering food over the phone and having meetings over the computer, and we are often speaking our credit card or bank information," Roy said. "But what is even more concerning for me is that it can reveal much more personal information. This kind of information can tell you about my living style, how many hours I'm working, other things that I am doing. And what we watch on TV can reveal our political orientations. That is crucial for someone who might want to manipulate the political elections or target very specific messages to me."

The researchers emphasize that vacuum cleaners are just one example of potential vulnerability to Lidar-based spying. Many other devices could be open to similar attacks such as smartphone infrared sensors used for face recognition or passive infrared sensors used for motion detection.

"I believe this is significant work that will make the manufacturers aware of these possibilities and trigger the security and privacy community to come up with solutions to prevent these kinds of attacks," Roy said.
This research was partially supported by a grant from Singapore Ministry of Education Academic Research Fund Tier 1 (Award No. R-252-000-A26-133).

The research paper, Spying with Your Robot Vacuum Cleaner: Eavesdropping via Lidar Sensors, Sriram Sami, Yimin Dai, Sean Rui Xiang Tan, Nirupam Roy and Jun Han, was presented on November 18, 2020, at the Association for Computing Machinery, SenSys 2020.

University of Maryland

Related Laser Articles from Brightsurf:

Laser technology: New trick for infrared laser pulses
For a long time, scientists have been looking for simple methods to produce infrared laser pulses.

Sensors get a laser shape up
Laser writing breathes life into high-performance sensing platforms.

Laser-powered nanomotors chart their own course
The University of Tokyo introduced a system of gold nanorods that acts like a tiny light-driven motor, with its direction of motion is determined by the orientation of the motors.

What laser color do you like?
Researchers at the National Institute of Standards and Technology (NIST) and the University of Maryland have developed a microchip technology that can convert invisible near-infrared laser light into any one of a panoply of visible laser colors, including red, orange, yellow and green.

Laser technology: The Turbulence and the Comb
While the light of an ordinary laser only has one single, well-defined wavelength, a so-called ''frequency comb'' consists of different light frequencies, which are precisely arranged at regular distances, much like the teeth of a comb.

A laser for penetrating waves
The 'Landau-level laser' is an exciting concept for an unusual radiation source.

Laser light detects tumors
A team of researchers from Jena presents a groundbreaking new method for the rapid, gentle and reliable detection of tumors with laser light.

The first laser radio transmitter
For the first time, researchers at Harvard School of Engineering have used a laser as a radio transmitter and receiver, paving the way for towards ultra-high-speed Wi-Fi and new types of hybrid electronic-photonic devices.

The random anti-laser
Scientists at TU Wien have found a way to build the 'opposite' of a laser -- a device that absorbs a specific light wave perfectly.

Laser 'drill' sets a new world record in laser-driven electron acceleration
Combining a first laser pulse to heat up and 'drill' through a plasma, and another to accelerate electrons to incredibly high energies in just tens of centimeters, scientists have nearly doubled the previous record for laser-driven particle acceleration at Berkeley Lab's BELLA Center.

Read More: Laser News and Laser Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to