Repurposed sensor enables smartwatch to detect finger taps and other bio-acoustic signals

October 17, 2016

PITTSBURGH--A smartwatch is capable of detecting and distinguishing a variety of taps, flicks and scratches by the hands and fingers, and all that's required is a software upgrade that repurposes the device's existing accelerometer, Carnegie Mellon University researchers discovered.

This new functionality makes possible new applications that use common gestures to control the smartwatch and, ultimately, other objects connected through the internet of things. By monitoring vibrations that occur when people hold objects or use tools, the smartwatch also would be capable of recognizing objects and activities.

It could even be used to help tune a guitar, with the smartwatch displaying the note transmitted as the guitarist plucks and adjusts each string.

"It's as if you're using your hand as a detection device," said Gierad Laput, a Ph.D. student in the Human-Computer Interaction Institute (HCII), suggesting totally new uses for smartwatches. "The hand is what people use to interact with the world."

This new technology, dubbed ViBand, was developed by Laput and Robert Xiao, both Ph.D. students in the HCII's Future Interfaces Group, along with their adviser Chris Harrison, assistant professor of human-computer interaction. They will present a paper describing the work at the Association for Computing Machinery's User Interface Software and Technology (ACM UIST) Symposium this week in Tokyo.

Normally, a smartwatch accelerometer is used to detect when a person lifts an arm so the screen can activate, or sometimes to count footsteps. To do so, the accelerometer only needs to take measurements about 100 times a second. But when researchers increased the sampling frequency to 4,000 a second, 4 kHz, they found it acted like a vibrational microphone. Rather than detecting sounds transmitted through the air, however, it couples with the body to detect bio-acoustic signals.

"ViBand isn't just a way to control your smartwatch," Harrison said. "It enables you to augment your arm. It's a powerful interface that's always available to you."

A ViBand-enabled watch can tell if someone is tapping on the forearm, the palm of the hand or the back of the hand. It can detect finger flicks, scratches and other motions. It also can sense if a person is holding various mechanical and electrical tools, such as an electric toothbrush, power drill or handsaw. Each body tap, device or activity has distinctive bio-acoustic signals.

To increase the frequency of the accelerometer's sampling rate, the research team developed a custom kernel -- the core of the smartwatch's operating system. That's the only modification required and can be performed as a software update, Laput said.

The team developed several demonstration apps for ViBand, including the use of hand gestures in the area around the watch to control apps on the watch. Similar gestures could be used to control remote devices, such as lights or a TV or other appliances connected via the internet of things.

They also showed it could be used for object-aware apps, such as monitoring meal preparations or providing visual feedback while tuning an acoustic guitar.

Finally, they propose an application using what they call a vibro-tag, which is a small object that emits inaudible, structured vibrations that contain data. A vibro-tag on a person's office door, for instance, might transmit information about office hours or alternative contact information.
-end-
This research was supported by the David and Lucile Packard Foundation, a Google Faculty Research Award and Qualcomm.

About Carnegie Mellon University: Carnegie Mellon is a private, internationally ranked research university with programs in areas ranging from science, technology and business, to public policy, the humanities and the arts. More than 13,000 students in the university's seven schools and colleges benefit from a small student-to-faculty ratio and an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration and innovation.

Carnegie Mellon University

Related Accelerometer Articles from Brightsurf:

High intensity physical activity in early life could lead to stronger bones in adulthood
High intensity physical activity in early life might help maximise peak hip strength and prevent osteoporosis in later life, according to a study published today in JAMA Network Open from researchers at the University of Bristol.

Smartphone accelerometers could help in resistance workouts and rehabilitation protocols
Smartphone accelerometers are effective tools to measure key time-under-tension indicators of muscle training -- and could help in resistance-based workouts and rehabilitation protocols.

Brazilian scientists develop COVID-19 accelerometer
Online application shows in real time whether the disease is spreading faster or slower in over 200 countries and helps evaluate the effectiveness of public policies aimed at containing the pandemic .

Study shows sedentary behavior independently predicts cancer mortality
In the first study to look at objective measures of sedentary behavior and cancer mortality, researchers from The University of Texas MD Anderson Cancer Center found that greater inactivity was independently associated with a higher risk of dying from cancer.

Lung-heart super sensor on a chip tinier than a ladybug
This Lilliputian chip's detection bandwidth is enormous -- from sweeping body motions to faint sounds of the heartbeat, pulse waves traversing body tissues, respiration rate, and lung sounds.

Is step count associated with lower risk of death?
Researchers looked at whether taking more steps and higher intensity stepping were associated with reduced risk of death in this observational study that included almost 4,900 adults (40 and over) who wore a device called an accelerometer to measure their step count and step intensity (steps/minute).

Less active infants had greater fat accumulation, study finds
Less physical activity for infants below one year of age may lead to more fat accumulation which in turn may predispose them to obesity later in life, suggests a study led by researchers at the Johns Hopkins Bloomberg School of Public Health.

Quality over quantity! Interval walking training improves fitness and health in elderly individuals
Interval Walking Training is a method that is effective in increasing overall fitness and decreasing healthcare costs associated with lifestyle-related diseases of the middle-aged and elderly.

Smartphone data can help surgeons understand a patient's recovery
Surgeons report that they can describe the impact of certain postoperative events in their patients by capturing passively collected accelerometer data from a patient's smartphone.

Using Wi-Fi like sonar to measure speed and distance of indoor movement
Researchers have developed a technique for measuring speed and distance in indoor environments, which could be used to improve navigation technologies for robots, drones -- or pedestrians trying to find their way around an airport.

Read More: Accelerometer News and Accelerometer Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.