Nav: Home

Voice-controlled nutrition tracker may aid weight loss

March 24, 2016

For people struggling with obesity, logging calorie counts and other nutritional information at every meal is a proven way to lose weight. The technique does require consistency and accuracy, however, and when it fails, it's usually because people don't have the time to find and record all the information they need.

A few years ago, a team of nutritionists from Tufts University who had been experimenting with mobile-phone apps for recording caloric intake approached members of the Spoken Language Systems Group at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), with the idea of a spoken-language application that would make meal logging even easier.

This week, at the International Conference on Acoustics, Speech, and Signal Processing in Shanghai, the MIT researchers are presenting a Web-based prototype of their speech-controlled nutrition-logging system.

With it, the user verbally describes the contents of a meal, and the system parses the description and automatically retrieves the pertinent nutritional data from an online database maintained by the U.S. Department of Agriculture (USDA).

The data is displayed together with images of the corresponding foods and pull-down menus that allow the user to refine their descriptions -- selecting, for instance, precise quantities of food. But those refinements can also be made verbally. A user who begins by saying, "For breakfast, I had a bowl of oatmeal, bananas, and a glass of orange juice" can then make the amendment, "I had half a banana," and the system will update the data it displays about bananas while leaving the rest unchanged.

"What [the Tufts nutritionists] have experienced is that the apps that were out there to help people try to log meals tended to be a little tedious, and therefore people didn't keep up with them," says James Glass, a senior research scientist at CSAIL, who leads the Spoken Language Systems Group. "So they were looking for ways that were accurate and easy to input information."

The first author on the new paper is Mandy Korpusik, an MIT graduate student in electrical engineering and computer science. She's joined by Glass, who's her thesis advisor; her fellow graduate student Michael Price; and by Calvin Huang, an undergraduate researcher in Glass's group.

Context sensitivity

In the paper, the researchers report the results of experiments with a speech-recognition system that they developed specifically to handle food-related terminology. But that wasn't the main focus of their work; indeed, an online demo of their meal-logging system instead uses Google's free speech-recognition app.

Their research concentrated on two other problems. One is identifying words' functional role: The system needs to recognize that if the user records the phrase "bowl of oatmeal," nutritional information on oatmeal is pertinent, but if the phrase is "oatmeal cookie," it's not.

The other problem is reconciling the user's phrasing with the entries in the USDA database. For instance, the USDA data on oatmeal is recorded under the heading "oats"; the word "oatmeal" shows up nowhere in the entry.

To address the first problem, the researchers used machine learning. Through the Amazon Mechanical Turk crowdsourcing platform, they recruited workers who simply described what they'd eaten at recent meals, then labeled the pertinent words in the description as names of foods, quantities, brand names, or modifiers of the food names. In "bowl of oatmeal," "bowl" is a quantity and "oatmeal" is a food, but in "oatmeal cookie," oatmeal is a modifier.

Once they had roughly 10,000 labeled meal descriptions, the researchers used machine-learning algorithms to find patterns in the syntactic relationships between words that would identify their functional roles.

Semantic matching

To translate between users' descriptions and the labels in the USDA database, the researchers used an open-source database called Freebase, which has entries on more than 8,000 common food items, many of which include synonyms. Where synonyms were lacking, they again recruited Mechanical Turk workers to supply them.

The version of the system presented at the conference is intended chiefly to demonstrate the viability of its approach to natural-language processing; it reports calorie counts but doesn't yet total them automatically. A version that does is in the works, however, and when it's complete, the Tufts researchers plan to conduct a user study to determine whether it indeed makes nutrition logging easier.
-end-
Additional background

ARCHIVE: Learning spoken language

ARCHIVE: Captioning at scale

ARCHIVE: Automatic speaker tracking in audio recordings

ARCHIVE: Text-based video navigation

Massachusetts Institute of Technology

Related Glass Articles:

The nature of glass-forming liquids is more clear
Researchers from The University of Tokyo have found that attractive and repulsive interactions between particles are both essential to form structural order that controls the dynamics of glass-forming liquids.
Experimental study of how 'metallic glass' forms challenges paradigm in glass research
Unlike in a crystal, the atoms in a metallic glass are not ordered when the liquid solidifies.
On-demand glass is right around the corner
A research group coordinated by physicists of the University of Trento was able to probe internal stress in colloidal glasses, a crucial step to control the mechanical properties of glasses.
Glass from a 3D printer
ETH researchers used a 3D printing process to produce complex and highly porous glass objects.
Making glass more clear
Northwestern University researchers have developed an algorithm that makes it possible to design glassy materials with dynamic properties and predict their continually changing behaviors.
Researchers use 3D printer to print glass
For the first time, researchers have successfully 3D printed chalcogenide glass, a unique material used to make optical components that operate at mid-infrared wavelengths.
New family of glass good for lenses
A new composition of germanosilicate glass created by adding zinc oxide has properties good for lens applications, according to Penn State researchers.
In-depth insights into glass corrosion
Silicate glass has many applications, including the use as a nuclear waste form to immobilize radioactive elements from spent fuel.
Laser-fabricated crystals in glass are ferroelectric
For the first time, a team of researchers from Lehigh University, Oak Ridge National Laboratory, Lebanon Valley College and Corning Inc. has demonstrated that laser-generated crystals confined in glass retain controllable ferroelectric properties, key to creating faster, more efficient optical communication systems.
New research questions the 'Glass Cliff' and corroborates the persistent 'Glass Ceiling'
Are women more likely to be appointed to leadership positions in crisis situations when companies are struggling with declining profits?
More Glass News and Glass Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Listen Again: Meditations on Loneliness
Original broadcast date: April 24, 2020. We're a social species now living in isolation. But loneliness was a problem well before this era of social distancing. This hour, TED speakers explore how we can live and make peace with loneliness. Guests on the show include author and illustrator Jonny Sun, psychologist Susan Pinker, architect Grace Kim, and writer Suleika Jaouad.
Now Playing: Science for the People

#565 The Great Wide Indoors
We're all spending a bit more time indoors this summer than we probably figured. But did you ever stop to think about why the places we live and work as designed the way they are? And how they could be designed better? We're talking with Emily Anthes about her new book "The Great Indoors: The Surprising Science of how Buildings Shape our Behavior, Health and Happiness".
Now Playing: Radiolab

The Third. A TED Talk.
Jad gives a TED talk about his life as a journalist and how Radiolab has evolved over the years. Here's how TED described it:How do you end a story? Host of Radiolab Jad Abumrad tells how his search for an answer led him home to the mountains of Tennessee, where he met an unexpected teacher: Dolly Parton.Jad Nicholas Abumrad is a Lebanese-American radio host, composer and producer. He is the founder of the syndicated public radio program Radiolab, which is broadcast on over 600 radio stations nationwide and is downloaded more than 120 million times a year as a podcast. He also created More Perfect, a podcast that tells the stories behind the Supreme Court's most famous decisions. And most recently, Dolly Parton's America, a nine-episode podcast exploring the life and times of the iconic country music star. Abumrad has received three Peabody Awards and was named a MacArthur Fellow in 2011.