Nav: Home

Repurposed sensor enables smartwatch to detect finger taps and other bio-acoustic signals

October 17, 2016

PITTSBURGH--A smartwatch is capable of detecting and distinguishing a variety of taps, flicks and scratches by the hands and fingers, and all that's required is a software upgrade that repurposes the device's existing accelerometer, Carnegie Mellon University researchers discovered.

This new functionality makes possible new applications that use common gestures to control the smartwatch and, ultimately, other objects connected through the internet of things. By monitoring vibrations that occur when people hold objects or use tools, the smartwatch also would be capable of recognizing objects and activities.

It could even be used to help tune a guitar, with the smartwatch displaying the note transmitted as the guitarist plucks and adjusts each string.

"It's as if you're using your hand as a detection device," said Gierad Laput, a Ph.D. student in the Human-Computer Interaction Institute (HCII), suggesting totally new uses for smartwatches. "The hand is what people use to interact with the world."

This new technology, dubbed ViBand, was developed by Laput and Robert Xiao, both Ph.D. students in the HCII's Future Interfaces Group, along with their adviser Chris Harrison, assistant professor of human-computer interaction. They will present a paper describing the work at the Association for Computing Machinery's User Interface Software and Technology (ACM UIST) Symposium this week in Tokyo.

Normally, a smartwatch accelerometer is used to detect when a person lifts an arm so the screen can activate, or sometimes to count footsteps. To do so, the accelerometer only needs to take measurements about 100 times a second. But when researchers increased the sampling frequency to 4,000 a second, 4 kHz, they found it acted like a vibrational microphone. Rather than detecting sounds transmitted through the air, however, it couples with the body to detect bio-acoustic signals.

"ViBand isn't just a way to control your smartwatch," Harrison said. "It enables you to augment your arm. It's a powerful interface that's always available to you."

A ViBand-enabled watch can tell if someone is tapping on the forearm, the palm of the hand or the back of the hand. It can detect finger flicks, scratches and other motions. It also can sense if a person is holding various mechanical and electrical tools, such as an electric toothbrush, power drill or handsaw. Each body tap, device or activity has distinctive bio-acoustic signals.

To increase the frequency of the accelerometer's sampling rate, the research team developed a custom kernel -- the core of the smartwatch's operating system. That's the only modification required and can be performed as a software update, Laput said.

The team developed several demonstration apps for ViBand, including the use of hand gestures in the area around the watch to control apps on the watch. Similar gestures could be used to control remote devices, such as lights or a TV or other appliances connected via the internet of things.

They also showed it could be used for object-aware apps, such as monitoring meal preparations or providing visual feedback while tuning an acoustic guitar.

Finally, they propose an application using what they call a vibro-tag, which is a small object that emits inaudible, structured vibrations that contain data. A vibro-tag on a person's office door, for instance, might transmit information about office hours or alternative contact information.
-end-
This research was supported by the David and Lucile Packard Foundation, a Google Faculty Research Award and Qualcomm.

About Carnegie Mellon University: Carnegie Mellon is a private, internationally ranked research university with programs in areas ranging from science, technology and business, to public policy, the humanities and the arts. More than 13,000 students in the university's seven schools and colleges benefit from a small student-to-faculty ratio and an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration and innovation.

Carnegie Mellon University

Related Accelerometer Articles:

Study finds primary school children get less active with age
There is an age-related decline in children's physical activity levels as they progress through primary school, according to a British Heart Foundation-funded study.
Researchers working toward indoor location detection
Rice University computer scientists are mapping a new solution for interior navigational location detection by linking it to existing sensors in mobile devices.
Sonic cyber attack shows security holes in ubiquitous sensors
Sound waves could be used to hack into critical sensors in a broad array of technologies including smartphones, automobiles, medical devices and the Internet of Things, University of Michigan research shows.
Passengers take mobile measure of comfort for railway companies
Passengers could soon be using their mobile phones to help rail companies around the globe improve the ride quality on their trains, thanks to new research.
Too much sitting, too little exercise may accelerate biological aging
Researchers at University of California San Diego School of Medicine report that elderly women who sit for more than 10 hours a day with low physical activity have cells that are biologically older than their chronological age by eight years compared to women who are less sedentary.
Parents' physical activity associated with preschooler activity in underserved populations
Preschool-age children from low-income families are more likely to be physically active if parents increase activity and reduce sedentary behavior while wearing movement monitors (accelerometers), according to a Vanderbilt study published today in the American Journal of Preventive Medicine.
NIH to release findings on impact of parent physical activity, sedentary behavior on their preschool
Young children do follow in their parents' footsteps. Literally. That's the conclusion of NIH-funded researchers who found that in underserved populations, parents' physical activity -- and their sedentary behavior -- directly correlates with the activity level of their preschoolers.
Repurposed sensor enables smartwatch to detect finger taps and other bio-acoustic signals
A smartwatch is capable of detecting and distinguishing a variety of taps, flicks and scratches by the hands and fingers, and all that's required is a software upgrade that repurposes the device's existing accelerometer, Carnegie Mellon University researchers discovered.
Want to optimize those 10,000 (or fewer) steps? Walk faster, sit less
That popular daily target of 10,000 steps is a worthwhile goal, but a new study suggests that if you find that unattainable, don't despair -- a smaller number, especially at moderate or greater intensity, can lead to health benefits too.
How machine learning can help with voice disorders
A team from MIT's Computer Science and Artificial Intelligence Laboratory and Massachusetts General Hospital has developed a wearable device that can tell if a user is misusing their voice and potentially someday be used to help detect vocal disorders.

Related Accelerometer Reading:

Best Science Podcasts 2019

We have hand picked the best science podcasts for 2019. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Jumpstarting Creativity
Our greatest breakthroughs and triumphs have one thing in common: creativity. But how do you ignite it? And how do you rekindle it? This hour, TED speakers explore ideas on jumpstarting creativity. Guests include economist Tim Harford, producer Helen Marriage, artificial intelligence researcher Steve Engels, and behavioral scientist Marily Oppezzo.
Now Playing: Science for the People

#524 The Human Network
What does a network of humans look like and how does it work? How does information spread? How do decisions and opinions spread? What gets distorted as it moves through the network and why? This week we dig into the ins and outs of human networks with Matthew Jackson, Professor of Economics at Stanford University and author of the book "The Human Network: How Your Social Position Determines Your Power, Beliefs, and Behaviours".