In first demonstrations of 'tele-immersion,' participants many miles apart feel as if they're sitting in the same room

December 13, 2000

PHILADELPHIA -- By marrying telecommunications and technology similar to that used in 3D movies, computer scientists have orchestrated a session where participants sitting in different states feel as if they're chatting in the same room. This first successful demonstration of this technique, known as "tele-immersion," was accomplished by researchers at the University of Pennsylvania, the University of North Carolina at Chapel Hill, Brown University and Advanced Network and Services, a non-profit firm in Armonk, N.Y.

Kostas Daniilidis, Ph.D., Penn's group leader on the National Tele-Immersion Initiative, said that unlike conventional videoconferencing, tele-immersion all but teleports faraway places into the here and now.

"While videoconferencing results in two-dimensional images on a screen, in tele-immersion the screen becomes a window allowing access to a site far away," said Daniilidis, an assistant professor of computer and information science at Penn. "The person with whom you're speaking is projected life-size in three dimensions -- you can even peer behind him or her."

Daniilidis and his NTII colleagues say tele-immersion has the potential to revolutionize the way people communicate, allowing people on opposite ends of the country or world to feel temporarily as if they're in each other's presence. Expert surgeons thousands of miles away could be reassuringly present in an operating room to offer counsel, actors in New York and Los Angeles could rehearse together and distance learning could become as real and engaging as a traditional classroom experience. Executives or researchers on different continents could hold face-to-face meetings without ever boarding a jet.

Key to tele-immersion's realistic feel are a hemispherical bank of digital cameras to capture participants from a variety of angles and tracking gear worn on their heads. Combined with polarized glasses much like those worn at 3D movies, the setup creates subtly different images in each eye -- much as our eyes do in daily life. Working from this sensory input, the brain is able to recreate the three-dimensional "telepresence" of a person actually sitting in a distant studio.

"Tele-immersion essentially takes comprehensive, real-time measures of a person and his surroundings," Daniilidis said, "and conveys that information directly to the senses of a person far away."

The result is that when a participant moves his or her head, the view of the others shifts almost as seamlessly as if the meeting were occurring face-to-face. When participants lean forward, their peers appear larger; when they recline, their virtual friends shrink in size. The Penn group's contribution is the technology for three-dimensional scanning of the environment and moving persons in real time using only conventional videocameras.

In the handful of tele-immersion sessions achieved so far, screens were mounted at right angles to a desk in the corner of a room, with each displaying one of the remote locations. While the screens are solid and flat, once a session begins they appear more like a window. Even the tabletops at the distant locations are perfectly matched to augment the feeling of shared space.

"It's somewhat like the Starship Enterprise's 'Holodeck' on Star Trek," Daniilidis said. "It allows us to interact with flat images as if they were living, breathing people right there in front of us."

As with any brand-new technology, the sessions have been far from seamless: transmission quality has been a tad shaky, and at this point only one participant in any given conference is able to see his distant colleagues. Daniilidis and his colleagues from the NTII plan to address the latter shortcoming soon, with two-way sessions in which all participants can see each other.

Daniilidis said that tele-immersion's potential to revolutionize communications remains a driving force behind the development of Internet2, the collaborative effort to create the high-bandwidth web of the future. While most online applications use only a tiny fraction of Internet2's massive band-width, tele-immersion is one of the few that requires moving far greater quantities of data than today's Internet can handle. A single tele-immersion session occurring on campus has temporarily boosted web traffic at Penn and other participating institutions to four times its normal volume.
The NTII, whose Penn participants include Daniilidis and colleagues Ruzena Bajcsy, Ph.D., and Jane Mulligan, Ph.D., is largely supported by Advanced Network and Services. Funding from the firm, founded by virtual reality pioneer Jaron Lanier, is augmented by support from the National Science Foundation, the Defense Advanced Research Projects Administration, the Department of Energy and the Intel Corporation.

NOTE: Color photos are available to illustrate this story.

University of Pennsylvania

Related Technology Articles from Brightsurf:

December issue SLAS Technology features 'advances in technology to address COVID-19'
The December issue of SLAS Technology is a special collection featuring the cover article, ''Advances in Technology to Address COVID-19'' by editors Edward Kai-Hua Chow, Ph.D., (National University of Singapore), Pak Kin Wong, Ph.D., (The Pennsylvania State University, PA, USA) and Xianting Ding, Ph.D., (Shanghai Jiao Tong University, Shanghai, China).

October issue SLAS Technology now available
The October issue of SLAS Technology features the cover article, 'Role of Digital Microfl-uidics in Enabling Access to Laboratory Automation and Making Biology Programmable' by Varun B.

Robot technology for everyone or only for the average person?
Robot technology is being used more and more in health rehabilitation and in working life.

Novel biomarker technology for cancer diagnostics
A new way of identifying cancer biomarkers has been developed by researchers at Lund University in Sweden.

Technology innovation for neurology
TU Graz researcher Francesco Greco has developed ultra-light tattoo electrodes that are hardly noticeable on the skin and make long-term measurements of brain activity cheaper and easier.

April's SLAS Technology is now available
April's Edition of SLAS Technology Features Cover Article, 'CURATE.AI: Optimizing Personalized Medicine with Artificial Intelligence'.

Technology in higher education: learning with it instead of from it
Technology has shifted the way that professors teach students in higher education.

Post-lithium technology
Next-generation batteries will probably see the replacement of lithium ions by more abundant and environmentally benign alkali metal or multivalent ions.

Rethinking the role of technology in the classroom
Introducing tablets and laptops to the classroom has certain educational virtues, according to Annahita Ball, an assistant professor in the University at Buffalo School of Social Work, but her research suggests that tech has its limitations as well.

The science and technology of FAST
The Five hundred-meter Aperture Spherical radio Telescope (FAST), located in a radio quiet zone, with the targets (e.g., radio pulsars and neutron stars, galactic and extragalactic 21-cm HI emission).

Read More: Technology News and Technology Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to