Readers become part of the action through high-tech mixture of traditional storytelling and virtual reality in University of Washington's 'Magic Book'

September 11, 2000

Imagine reading a biology textbook and coming to a chapter on frogs. The illustrations on the page are helpful, but the flat images can't communicate the sort of information that would come from watching a real frog.

Now imagine donning a pair of specialized glasses. You can still see the book and your surroundings, but suddenly a 3-D image leaps off the page and floats above the book. You can rotate the book to see detail from different angles and the image moves like a real frog. But you want to know more. So you "leap" into a scene in the book. The outside world disappears while you explore a pond that the frog calls home, or even fly into the frog's mouth for an inside look at its gullet.

Sound like magic? That's exactly the term University of Washington researcher Mark Billinghurst and his colleagues are using to describe their latest project, dubbed "Magic Book."

Magic Book was featured in demonstrations of emerging technology at the recent SIGGRAPH 2000 conference, the Association of Computing Machinery's international computer graphics meeting that attracts more than 50,000 participants from academia and industry. The response was enthusiastic, Billinghurst said.

"People from Disney were interested, as were some from Universal Studios, and several training companies and museums," he said. "There are a lot of very innovative and useful things you could do with this."

A Magic Book looks like a traditional book - it has text and colorful pictures. But look at it through a lightweight viewer and moving, three-dimensional images jump off the page. People can read the book together, interacting as they read the story and simultaneously viewing the virtual images from different angles.

But the experience doesn't end there. With the turn of a switch, readers can "fly" into a scene to become part of the action. If more than one person steps into the story, participants can see one another as "avatars," characters that represent people in the virtual world.

"What we've done is create a continuum that people can move along, from the real world to a combination of real and virtual, to an entirely immersive virtual experience," said Billinghurst, a doctoral student in electrical engineering who studies virtual reality in the UW's "HIT Lab" - short for Human Interface Technology Laboratory. "What makes our technology so unusual is the ease with which people can move from one point to the other. It's quite seamless."

The applications for Magic Book range from storybooks to technical manuals and textbooks.

In terms of storybooks, the HIT Lab and partners ATR MIC Labs in Japan and Hirokazu Kato of Hiroshima City University have so far put together about eight books. One features the tale of a Japanese princess, a samurai warrior and an evil ninja who steals the princess' looking glass. As each character is introduced, readers are treated to moving foot-high 3-D images standing above the page. The princess weeps because her mirror is gone. The samurai shows puzzlement because he can't find his horse. And the ninja goes through combat exercises.

There are frequent invitations to join the story. Readers can enter the princess' bedroom and help her look for the missing item. They can help the samurai by searching his farm for his lost horse. And they can hunt down the ninja and find the looking glass in a feudal village. In each case, readers simply look around to take in their surroundings. Pressing a button on the viewer provides forward movement.

The technology also works well for technical material, Billinghurst said.

Imagine opening an anatomy textbook to the chapter covering the human heart. Students would be greeted by a detailed, three-dimensional heart, floating above the page. They could examine it from different angles and study it while it beat.

"Then you could switch to an immersive environment and actually enter the heart and go through it like you were a blood corpuscle," Billinghurst said. "And the whole time you would still be interacting with other students in your study group."

Here's how Magic Book melds the virtual and the physical: Each viewer contains a small camera that provides continuous video feedback to a computer of what's being viewed. Incorporated into the pictures of the book are unique patterns that are recognizable by the computer. When the computer sees a pattern, it projects a corresponding image onto the page. The software continuously calculates the position and orientation of the camera relative to black borders around the pictures so the images appear to stay fixed in relation to the pages of the book. As the book is rotated and tilted, the images move in sync.

When readers enter a virtual scene, tracking switches from visual to a tracker in the viewer. The tracker keeps tabs on head orientation to allow the reader to move smoothly through the virtual surroundings.

Although there's no set timetable, Billinghurst calls commercialization of the technology "the ultimate goal."

"It is very conceivable that the HIT Lab could enter into a consultant relationship with some of the groups interested in this," he said. "A spinoff company is also possible, or an arrangement in which the lab would enter into a licensing agreement with an interested company. We'll see what happens."
For more information, contact Billinghurst at (206) 543-5075 or A Web site about the Magic Book project, which includes digital video clips of the technology in action, can be viewed at

A high-resolution color image showing the technology of Magic Book at work is available on the Web at wsroom/news/images/samuraihr.jpg. Caption information is as follows:

Using a book as a starting point, the Magic Book project allows readers to interact with the text by viewing combinations of virtual reality and the real world. Above, a computer paints onto the real environment the image of a samurai, visible to readers wearing a special viewer. The intent is to enhance reading and learning in storybooks, textbooks and technical publications.

University of Washington

Related Technology Articles from Brightsurf:

December issue SLAS Technology features 'advances in technology to address COVID-19'
The December issue of SLAS Technology is a special collection featuring the cover article, ''Advances in Technology to Address COVID-19'' by editors Edward Kai-Hua Chow, Ph.D., (National University of Singapore), Pak Kin Wong, Ph.D., (The Pennsylvania State University, PA, USA) and Xianting Ding, Ph.D., (Shanghai Jiao Tong University, Shanghai, China).

October issue SLAS Technology now available
The October issue of SLAS Technology features the cover article, 'Role of Digital Microfl-uidics in Enabling Access to Laboratory Automation and Making Biology Programmable' by Varun B.

Robot technology for everyone or only for the average person?
Robot technology is being used more and more in health rehabilitation and in working life.

Novel biomarker technology for cancer diagnostics
A new way of identifying cancer biomarkers has been developed by researchers at Lund University in Sweden.

Technology innovation for neurology
TU Graz researcher Francesco Greco has developed ultra-light tattoo electrodes that are hardly noticeable on the skin and make long-term measurements of brain activity cheaper and easier.

April's SLAS Technology is now available
April's Edition of SLAS Technology Features Cover Article, 'CURATE.AI: Optimizing Personalized Medicine with Artificial Intelligence'.

Technology in higher education: learning with it instead of from it
Technology has shifted the way that professors teach students in higher education.

Post-lithium technology
Next-generation batteries will probably see the replacement of lithium ions by more abundant and environmentally benign alkali metal or multivalent ions.

Rethinking the role of technology in the classroom
Introducing tablets and laptops to the classroom has certain educational virtues, according to Annahita Ball, an assistant professor in the University at Buffalo School of Social Work, but her research suggests that tech has its limitations as well.

The science and technology of FAST
The Five hundred-meter Aperture Spherical radio Telescope (FAST), located in a radio quiet zone, with the targets (e.g., radio pulsars and neutron stars, galactic and extragalactic 21-cm HI emission).

Read More: Technology News and Technology Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to