Object identification and interaction with a smartphone knock

October 02, 2019

A KAIST team has featured a new technology, "Knocker", which identifies objects and executes actions just by knocking on it with the smartphone. Software powered by machine learning of sounds, vibrations, and other reactions will perform the users' directions.

What separates Knocker from existing technology is the sensor fusion of sound and motion. Previously, object identification used either computer vision technology with cameras or hardware such as RFID (Radio Frequency Identification) tags. These solutions all have their limitations. For computer vision technology, users need to take pictures of every item. Even worse, the technology will not work well in poor lighting situations. Using hardware leads to additional costs and labor burdens.

Knocker, on the other hand, can identify objects even in dark environments only with a smartphone, without requiring any specialized hardware or using a camera. Knocker utilizes the smartphone's built-in sensors such as a microphone, an accelerometer, and a gyroscope to capture a unique set of responses generated when a smartphone is knocked against an object. Machine learning is used to analyze these responses and classify and identify objects.

The research team under Professor Sung-Ju Lee from the School of Computing confirmed the applicability of Knocker technology using 23 everyday objects such as books, laptop computers, water bottles, and bicycles. In noisy environments such as a busy café or on the side of a road, it achieved 83% identification accuracy. In a quiet indoor environment, the accuracy rose to 98%.

The team believes Knocker will open a new paradigm of object interaction. For instance, by knocking on an empty water bottle, a smartphone can automatically order new water bottles from a merchant app. When integrated with IoT devices, knocking on a bed's headboard before going to sleep could turn off the lights and set an alarm. The team suggested and implemented 15 application cases in the paper, presented during the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2019) held in London last month.

Professor Sung-Ju Lee said, "This new technology does not require any specialized sensor or hardware. It simply uses the built-in sensors on smartphones and takes advantage of the power of machine learning. It's a software solution that everyday smartphone users could immediately benefit from." He continued, "This technology enables users to conveniently interact with their favorite objects."
-end-
The research was supported in part by the Next-Generation Information Computing Development Program through the National Research Foundation of Korea funded by the Ministry of Science and ICT and an Institute for Information & Communications Technology Promotion (IITP) grant funded by the Ministry of Science and ICT.

The Korea Advanced Institute of Science and Technology (KAIST)

Related Smartphone Articles from Brightsurf:

Mobile smartphone technology is associated with better clinical outcomes for OHCA
Mobile smartphone technology can accelerate first responder dispatch and may be instrumental to improving out?of?hospital cardiac arrest (OCHA) survival.

New tool can diagnose strokes with a smartphone
A new tool created by researchers at Penn State and Houston Methodist Hospital could diagnose a stroke based on abnormalities in a patient's speech ability and facial muscular movements, and with the accuracy of an emergency room physician -- all within minutes from an interaction with a smartphone.

App analyzes coronavirus genome on a smartphone
A team led by Garvan's Dr Ira Deveson developed the app 'Genopo' that can analyse the coronavirus genome on a portable Android device.

Smartphone accelerometers could help in resistance workouts and rehabilitation protocols
Smartphone accelerometers are effective tools to measure key time-under-tension indicators of muscle training -- and could help in resistance-based workouts and rehabilitation protocols.

Parents' smartphone use does not harm parent/child relationships
Contrary to popular views, parental smartphone use is rarely associated with poor parenting, and more often than not, tends to be associated with warm and attached parenting.

The effects of smartphone use on parenting
Parents may worry that spending time on their smartphones has a negative impact on their relationships with their children.

Inexpensive retinal diagnostics via smartphone
Retinal damage due to diabetes is now considered the most common cause of blindness in working-age adults.

Nanosensor can alert a smartphone when plants are stressed
MIT engineers can closely track how plants respond to stresses such as injury, infection, and light damage using sensors made of carbon nanotubes.

Smartphone apps not accurate enough to spot all skin cancers
Smartphone apps that assess the risk of suspicious moles cannot be relied upon to detect all cases of skin cancer, finds a review of the evidence published by The BMJ today.

Detecting mental and physical stress via smartphone
The team led by Professor Enrico Caiani of the Department of Electronics, Information and Bioengineering at Politecnico di Milano, Italy, has shown that it is possible to use our smartphones without any other peripherals or wearables to accurately extract vital parameters, such as heart beat rate and stress level.

Read More: Smartphone News and Smartphone Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.