Computer scientists unravel 'language of surgery'

December 08, 2006

Borrowing ideas from speech recognition research, Johns Hopkins computer scientists are building mathematical models to represent the safest and most effective ways to perform surgery, including tasks such as suturing, dissecting and joining tissue.

The team's long-term goal is to develop an objective way of evaluating a surgeon's work and to help doctors improve their operating room skills. Ultimately, the research also could enable robotic surgical tools to perform with greater precision.

The project, supported by a three-year National Science Foundation grant, has yielded promising early results in modeling suturing work. The researchers performed the suturing with the help of a robotic surgical device, which recorded the movements and made them available for computer analysis.

"Surgery is a skilled activity, and it has a structure that can be taught and acquired," said Gregory D. Hager, a professor of computer science in the university's Whiting School of Engineering and principal investigator on the project. "We can think of that structure as 'the language of surgery.' To develop mathematical models for this language, we're borrowing techniques from speech recognition technology and applying them to motion recognition and skills assessment."

Complicated surgical tasks, Hager said, unfold in a series of steps that resemble the way that words, sentences and paragraphs are used to convey language. "In speech recognition research, we break these down to their most basic sounds, called phonemes," he said. "Following that example, our team wants to break surgical procedures down to simple gestures that can be represented mathematically by computer software."

With that information in hand, the computer scientists hope to be able to recognize when a surgical task is being performed well and also to identify which movements can lead to operating room problems. Just as a speech recognition program might call attention to poor pronunciation or improper syntax, the system being developed by Hager's team might identify surgical movements that are imprecise or too time-consuming.

But to get to that point, computers first must become fluent in the "language" of surgery. This will require computers to absorb data concerning the best ways to complete surgical tasks. As a first step, the researchers have begun collecting data recorded by Intuitive Surgical's da Vinci Surgical Systems. These systems allow a surgeon, seated at a computer workstation, to guide robotic tools to perform minimally invasive procedures involving the heart, the prostate and other organs. Although only a tiny fraction of hospital operations involve the da Vinci, the device's value to Hager's team is that all of the robot's surgical movements can be digitally recorded and processed.

In a paper presented at the Medical Image Computing and Computer-Assisted Intervention Conference in October 2005, Hager's team announced that it had developed a way to use data from the da Vinci to mathematically model surgical tasks such as suturing, a key first step in deciphering the language of surgery. The lead author, Johns Hopkins graduate student Henry C. Lin, received the conference award for best student paper.

"Now, we're acquiring enough data to go from 'words' to 'sentences,'" said Hager, who is deputy director of the National Science Foundation Engineering Research Center for Computer-Integrated Surgical Systems and Technology, based at Johns Hopkins. "One of our goals for the next few years is to develop a large vocabulary that we can use to represent the motions in surgical tasks."

The team also hopes to incorporate video data from the da Vinci and possibly from minimally invasive procedures performed directly by surgeons. In such operations, surgeons insert instruments and a tiny camera into small incisions to complete a medical procedure. The video data from the camera could contribute data to the team's efforts to identify effective surgical methods.

Hager's Johns Hopkins collaborators include David D. Yuh, a cardiac surgeon from the School of Medicine. "It is fascinating to break down the surgical skills we take for granted into their fundamental components," Yuh said. "Hopefully, a better understanding of how we learn to operate will help more efficiently train future surgeons. With the significantly reduced number of hours surgical residents are permitted to be in the hospital, surgical training programs need to streamline their training methods now more than ever. This research work represents a strong effort toward this."

Hager's other collaborators include Sanjeev Khudanpur, a Johns Hopkins assistant professor of electrical and computer engineering, and Izhak Shafran, who was a postdoctoral fellow affiliated with the university's Center for Language and Speech Processing and who is now an assistant professor at the Oregon Graduate Institute.

Hager cautioned that the project is not intended to produce a "Big Brother" system that would critique a surgeon's every move. "We're trying to find ways to help them become better at what they do," he said. "It's not a new idea. In sports and dance, people are studying the mechanics of movement to see what produces the best possible performance. By understanding the underlying structures, we can become better at what we do. I think surgery's no different."
-end-
Digital photos of the researchers available; contact Phil Sneiderman.

Related Links:
Gregory Hager's Lab Page: http://www.cs.jhu.edu/CIRL/
NSF Center for Computer-Integrated Surgical Systems and Technology: http://www.cisst.org/
Johns Hopkins Department of Computer Science: http://www.cs.jhu.edu/

Johns Hopkins University

Related Language Articles from Brightsurf:

Learning the language of sugars
We're told not to eat too much sugar, but in reality, all of our cells are covered in sugar molecules called glycans.

How effective are language learning apps?
Researchers from Michigan State University recently conducted a study focusing on Babbel, a popular subscription-based language learning app and e-learning platform, to see if it really worked at teaching a new language.

Chinese to rise as a global language
With the continuing rise of China as a global economic and trading power, there is no barrier to prevent Chinese from becoming a global language like English, according to Flinders University academic Dr Jeffrey Gil.

'She' goes missing from presidential language
MIT researchers have found that although a significant percentage of the American public believed the winner of the November 2016 presidential election would be a woman, people rarely used the pronoun 'she' when referring to the next president before the election.

How does language emerge?
How did the almost 6000 languages of the world come into being?

New research quantifies how much speakers' first language affects learning a new language
Linguistic research suggests that accents are strongly shaped by the speaker's first language they learned growing up.

Why the language-ready brain is so complex
In a review article published in Science, Peter Hagoort, professor of Cognitive Neuroscience at Radboud University and director of the Max Planck Institute for Psycholinguistics, argues for a new model of language, involving the interaction of multiple brain networks.

Do as i say: Translating language into movement
Researchers at Carnegie Mellon University have developed a computer model that can translate text describing physical movements directly into simple computer-generated animations, a first step toward someday generating movies directly from scripts.

Learning language
When it comes to learning a language, the left side of the brain has traditionally been considered the hub of language processing.

Learning a second alphabet for a first language
A part of the brain that maps letters to sounds can acquire a second, visually distinct alphabet for the same language, according to a study of English speakers published in eNeuro.

Read More: Language News and Language Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.