Earable computing: A new research area in the making

December 15, 2020

CSL's Systems and Networking Research Group (SyNRG) is defining a new sub-area of mobile technology that they call "earable computing." The team believes that earphones will be the next significant milestone in wearable devices, and that new hardware, software, and apps will all run on this platform.

"The leap from today's earphones to 'earables' would mimic the transformation that we had seen from basic phones to smartphones," said Romit Roy Choudhury, professor in electrical and computer engineering (ECE). "Today's smartphones are hardly a calling device anymore, much like how tomorrow's earables will hardly be a smartphone accessory."

Instead, the group believes tomorrow's earphones will continuously sense human behavior, run acoustic augmented reality, have Alexa and Siri whisper just-in-time information, track user motion and health, and offer seamless security, among many other capabilities.

The research questions that underlie earable computing draw from a wide range of fields, including sensing, signal processing, embedded systems, communications, and machine learning. The SyNRG team is on the forefront of developing new algorithms while also experimenting with them on real earphone platforms with live users.

Computer science PhD student Zhijian Yang and other members of the SyNRG group, including his fellow students Yu-Lin Wei and Liz Li, are leading the way. They have published a series of papers in this area, starting with one on the topic of hollow noise cancellation that was published at ACM SIGCOMM 2018. Recently, the group had three papers published at the 26th Annual International Conference on Mobile Computing and Networking (ACM MobiCom) on three different aspects of earables research: facial motion sensing, acoustic augmented reality, and voice localization for earphones.

"If you want to find a store in a mall," says Zhijian, "the earphone could estimate the relative location of the store and play a 3D voice that simply says 'follow me.' In your ears, the sound would appear to come from the direction in which you should walk, as if it's a voice escort."

The second paper, EarSense: Earphones as a Teeth Activity Sensor, looks at how earphones could sense facial and in-mouth activities such as teeth movements and taps, enabling a hands-free modality of communication to smartphones. Moreover, various medical conditions manifest in teeth chatter, and the proposed technology would make it possible to identify them by wearing earphones during the day. In the future, the team is planning to look into analyzing facial muscle movements and emotions with earphone sensors.

The third publication, Voice Localization Using Nearby Wall Reflections, investigates the use of algorithms to detect the direction of a sound. This means that if Alice and Bob are having a conversation, Bob's earphones would be able to tune into the direction Alice's voice is coming from.

"We've been working on mobile sensing and computing for 10 years," said Wei. "We have a lot of experience to define this emerging landscape of earable computing."
-end-
Haitham Hassanieh, assistant professor in ECE, is also involved in this research. The team has been funded by both NSF and NIH, as well as companies like Nokia and Google. See more at the group's Earable Computing website.

University of Illinois Grainger College of Engineering

Related Smartphones Articles from Brightsurf:

Motorists' smartphones may help highways bosses keep roads safe
Motorists with smartphones could help highway chiefs maintain road quality by sending 'crowdsourced' data from their mobiles that would allow engineers to assess when carriageway repairs are needed, according to a new study.

Next-gen smartphones to keep their cool
Multilayered carbon material could be the perfect fit for heat management in electronic devices.

With digital phenotyping, smartphones may play a role in assessing severe mental illness
Digital phenotyping approaches that collect and analyze Smartphone-user data on locations, activities, and even feelings - combined with machine learning to recognize patterns and make predictions from the data - have emerged as promising tools for monitoring patients with psychosis spectrum illnesses, according to a report in the September/October issue of Harvard Review of Psychiatry.

Smartphones can predict brain function associated with anxiety and depression
Phone data such as social activity, screen time and location can predict connectivity between regions of the brain that are responsible for emotion.

Smartphones can tell when you're drunk by analyzing your walk
Your smartphone can tell when you've had too much to drink by detecting changes in the way you walk, according to a new study published in the Journal of Studies on Alcohol and Drugs.

Smartphones are lowering student's grades, study finds
The ease of finding information on the internet is hurting students' long-term retention and resulting in lower grades on exams, according to a Rutgers University-New Brunswick study.

Smartphones may help detect diabetes
Researchers at UC San Francisco have developed a ''digital biomarker'' that would use a smartphone's built-in camera to detect Type 2 diabetes - one of the world's top causes of disease and death - potentially providing a low-cost, in-home alternative to blood draws and clinic-based screening tools.

Smartphones prove to be time-saving analytical tools
Scientists use a smartphone camera to easily measure soil density -- a key metric for analyzing our soils

Why smartphones are digital truth serum
People are more willing to reveal personal information about themselves online using their smartphones compared to desktop computers.

Which comes first: Smartphone dependency or depression?
New research suggests a person's reliance on his or her smartphone predicts greater loneliness and depressive symptoms, as opposed to the other way around.

Read More: Smartphones News and Smartphones Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.