Nav: Home

Virtual human hand simulation holds promise for prosthetics

September 26, 2019

Whatever our hands do--reaching, grabbing or manipulating objects--it always appears simple. Yet your hands are one of the most complicated, and important, parts of the body.

Despite this, little is understood about the complexity of the hand's underlying anatomy and, as such, animating human hands has long been considered one of the most challenging problems in computer graphics.

That's because it has been impossible to capture the internal movement of the hand in motion--until now.

Using magnetic resonance imaging (MRI) and a technique inspired by the visual effects industry, a team of USC researchers, comprising two computer scientists and a radiologist, has developed the world's most realistic model of the human hand's musculoskeletal system in motion.

The musculoskeletal system includes muscles, bones, tendons and joints. The breakthrough has implications not only for computer graphics, but also prosthetics, medical education, robotics and virtual reality.

"The hand is very complicated, but prior to this work, nobody had built a precise computational model for how anatomical structures inside the hand actually move as it is articulated," said study co-author Jernej Barbic, an Andrew and Erna Viterbi Early Career Chair and Associate Professor of Computer Science.

Designing better prosthetics

To tackle this problem, Barbic, a computer animation and physically-based simulation expert, and his PhD student, Bohan Wang, the study's lead author, teamed up with George Matcuk, MD, an associate professor of clinical radiology at Keck School of Medicine of USC. The result: the most precise anatomically based model of the hand in motion.

"This is currently the most accurate hand animation model available and the first to combine laser scanning of the hand's surface features and to incorporate an underlying bone rigging model based on MRI," said Matcuk.

In addition to creating more realistic hands for computer games and CGI movies, where hands are often exposed, this system could also be used in prosthetics, to design better fingers and hand prostheses.

"Understanding the motion of internal hand anatomy opens the door for biologically-inspired robotic hands that look and behave like real hands," said Barbic.

"In the not-so-distant future, the work may contribute to the development of anatomically realistic hands and improved hand prosthetics."

The study, titled Hand Modelling and Simulation using Stabilized Magnetic Resonance Imaging, was presented at ACM SIGGRAPH.

A long-standing challenge

To improve realism, virtual hands should be modeled similarly to biological hands, which requires building precise anatomical and kinematic models of real human hands. But we still know surprisingly little about how bones and muscles move inside the hand.

One of the reasons is that, until now, there have been no methods to systematically acquire the motion of internal hand anatomy. Although MRI scanners can provide anatomical details, a previously unaddressed practical challenge exists: the hand must be kept perfectly still in the scanner for around 10 minutes.

"Holding the hand still in a fixed pose for 10 minutes is practically impossible," said Barbic. "A fist is easier to hold steady, but try semi-closing your hand and you'll find you start to shake after about a minute or two. You can't hold it still for 10 minutes."

To overcome this challenge, the researchers developed a manufacturing process using lifecasting materials from the special effects industry to stabilize the hand during the MRI scanning process. Lifecasting involves making a mold of the human form and then reproducing it in various media, including plastic or silicone.

Barbic, who worked on the Oscar-nominated film The Hobbit: the Desolation of Smaug, landed on the idea after seeing an inexpensive hand-cloning product in a visual effects store in Los Angeles while working on a previous project. "That was the eureka moment," said Barbic, who has long pondered a solution for creating more realistic virtual human hands.

First, the team used the life-casting material to create a plastic replica of the model's hand. This replica captures extremely detailed features, down to individual pores and tiny lines on the hand surface, which were then scanned using a laser scanner.

Then, the lifecasting process was used again, this time on the plastic hand, to create a negative 3D mold of the hand out of a rubber-like elastic material. The mold stabilizes the hand in the required pose.  The mold was cut in two parts, and then the subject placed their real hand into the mold for MRI scanning.

"As we refine this work, I think this could be an excellent teaching tool for my students and other doctors who need an understanding of the complex anatomy and biomechanics of the hand." George Matcuk

With assistance from radiology expert Matcuk, a practicing medical doctor at USC, the hand was then scanned by the MRI scanner for 10 minutes. This procedure was repeated 12 times, each time in a different hand pose. Two subjects, one male and one female, were captured in this way. Now, for every pose, the researchers knew exactly where the bones, muscles and tendons were positioned.

After discussing the anatomical features of the MRI scans with Matcuk, Barbic and Wang set to work building a data-driven skeleton kinematic model that captures complex real-world rotations and translations of bones in any pose.

They then added soft tissue simulation, using the finite element method (FEM) to compute the motion of the hand's muscles, tendons and the fat tissue, consistent with the bone motion. This model, combined with surface detail allowed them to create a highly realistic moving hand. The hand can be animated in any motion, even movement which is very different from the captured poses.

Going forward

The team, which recently received a grant from the National Science Foundation to take their work to the next stage, plans to build a public dataset of multi-pose hand MRI scans, for 10 subjects over the next three years. This will be the first dataset of its kind and will enable researchers from around the world to better simulate, model and re-create human hands. The team also plans to integrate the research into education, to train PhD students at USC and for K-12 outreach programs.

"As we refine this work, I think this could be an excellent teaching tool for my students and other doctors who need an understanding of the complex anatomy and biomechanics of the hand," said Matcuk.

The team is currently working on adding better awareness of muscles and tendons into the model and making it real-time. Right now, it takes the computer about an hour to create a minute-long simulation. Barbic and Wang hope to make the system faster, without losing quality.

University of Southern California

Related Magnetic Resonance Imaging Articles:

Detection of very high frequency magnetic resonance could revolutionize electronics
A team of scientists led by a physicist at the University of California, Riverside, has discovered an electrical detection method for terahertz electromagnetic waves, which are extremely difficult to detect.
Artificial intelligence to improve resolution of brain magnetic resonance imaging
Researchers of the ICAI Group -Computational Intelligence and Image Analysis- of the University of Malaga (UMA) have designed an unprecedented method that is capable of improving brain images obtained through magnetic resonance imaging using artificial intelligence.
Simple experiment explains magnetic resonance
Physicists at University of California, Riverside, have designed an experiment to explain the concept of magnetic resonance.
Scientists develop first implantable magnet resonance detector
A new miniature NMR implant measures neuronal activity.
NIH researchers develop MRI with lower magnetic field for cardiac and lung imaging condition
National Institutes of Health researchers, along with researchers at Siemens, have developed a high-performance, low magnetic-field MRI system that vastly improves image quality of the lungs and other internal structures of the human body.
'Resonance' raman spectroscopy with 1-nm resolution
Tip-enhanced Raman spectroscopy resolved 'resonance' Raman scattering with 1-nm resolution in ultrathin zinc oxide films epitaxially grown on a single-crystal silver surface.
CNIC is the coordinator of an international consensus document on the use of magnetic resonance
CNIC has coordinated the first international consensus document providing guidelines on the conduct of magnetic resonance imaging studies after a myocardial infarction in clinical trials or experimental models.
Resonance-enhanced tunneling induces F+H2 reaction in interstellar clouds
Scientists from the Dalian Institute of Chemical Physics (DICP) of the Chinese Academy of Sciences and their collaborators investigated the mechanism of rapid reactivity of the F + H2 reaction at low temperature and found that rapid reactivity was actually induced by resonance-enhanced tunneling.
Diattenuation imaging -- a promising imaging technique for brain research
A new imaging method provides structural information about brain tissue that was previously difficult to access.
What happens to magnetic nanoparticles once in cells?
Although magnetic nanoparticles are being used more and more in cell imaging and tissue bioengineering, what happens to them within stem cells in the long term remained undocumented.
More Magnetic Resonance Imaging News and Magnetic Resonance Imaging Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

There's so much we've yet to explore–from outer space to the deep ocean to our own brains. This hour, Manoush goes on a journey through those uncharted places, led by TED Science Curator David Biello.
Now Playing: Science for the People

#555 Coronavirus
It's everywhere, and it felt disingenuous for us here at Science for the People to avoid it, so here is our episode on Coronavirus. It's ok to give this one a skip if this isn't what you want to listen to right now. Check out the links below for other great podcasts mentioned in the intro. Host Rachelle Saunders gets us up to date on what the Coronavirus is, how it spreads, and what we know and don't know with Dr Jason Kindrachuk, Assistant Professor in the Department of Medical Microbiology and infectious diseases at the University of Manitoba. And...
Now Playing: Radiolab

Dispatch 1: Numbers
In a recent Radiolab group huddle, with coronavirus unraveling around us, the team found themselves grappling with all the numbers connected to COVID-19. Our new found 6 foot bubbles of personal space. Three percent mortality rate (or 1, or 2, or 4). 7,000 cases (now, much much more). So in the wake of that meeting, we reflect on the onslaught of numbers - what they reveal, and what they hide.  Support Radiolab today at