Nav: Home

Google unveils new virtual reality experience at ACM SIGGRAPH 2018

July 26, 2018

CHICAGO--Google Inc. has unveiled a new virtual reality (VR) immersive experience based on a novel system that captures and renders high-quality, realistic images from the real world using light fields. Created by a team of leading researchers at Google, Welcome to Light Fields is the tech giant's splash into the nascent arena of light fields VR experiences, an exciting corner of VR video technology gaining traction for its promise to deliver extremely high-quality imagery and experiences in the virtual world.

Google released Welcome to Light Fields earlier this year as a free app on Steam VR for HTC Vive, Oculus Rift, and Windows Mixed Reality headsets. The creators will demonstrate the VR experience at SIGGRAPH 2018, in the Immersive Pavilion, a new space for this year's conference. The Pavilion is devoted exclusively to virtual, augmented, and mixed reality and will contain: the Vrcade, a space for VR, AR, and MR games or experiences; the VR Theater, a storytelling extravaganza that is part of the Computer Animation Festival; and the well-known Village, for showcasing large-scale projects. SIGGRAPH 2018, held 12-16 August in Vancouver, British Columbia, is an annual gathering that showcases the world's leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.

Destinations in Welcome to Light Fields include NASA's Space Shuttle Discovery, delivering to viewers an astronaut's view inside the flight deck, which has never been open to the public; the pristine teak and mahogany interiors of the Gamble House, an architectural treasure in Pasadena, CA; and the glorious St. Stephen's Church in Granada Hills, CA, home to a stunning wall of more than 14,000 pieces of glimmering stained glass.

"I love that light fields in VR can teleport you to exotic places in the real world, and truly make you believe you are there," says Ryan Overbeck, software engineer at Google who co-led the project. "To me, this is magic."

To bring this experience to life, Overbeck worked with a team that included Paul Debevec, senior staff engineer at Google, who managed the project and led the hardware piece with engineers Xueming Yu, Jay Busch, and Graham Fyffe. With Overbeck, Daniel Erickson and Daniel Evangelakos focused on the software end. The researchers designed a comprehensive system for capturing and rendering high-quality, spherical light field still images from footage captured in the real world. They developed two easy-to-use light field camera rigs, based on the GoPro Hero4action sports camera, that efficiently capture thousands of images on the surface of a sphere. Those images were then passed through a cloud-based light-field-processing pipeline.

Among other things, explains Overbeck, "The processing pipeline uses computer vision to place the images in 3D and generate depth maps, and we use a modified version of our vp9 video codec [video formatting] to compress the light field data down to a manageable size." To render a light field dataset, he notes, the team used a rendering algorithm that blends between the thousands of light field images in real-time.

The team relied on Google's talented pool of engineers in computer vision, graphics, video compression, and machine learning to overcome the unique challenges posed in light fields technology. They also collaborated closely with the WebM team (who make the vp9 video codec) to develop the high-quality light field compression format incorporated into their system, and leaned heavily on the expertise of the Jump VR team to help pose the images and generate depth maps. (Jump is Google's professional VR system for achieving 3D-360 video production at scale.)

Indeed, with Welcome to Light Fields, the Google team is demonstrating the potential and promise of light field VR technology, showcasing the technology's ability to provide a truly immersive experience with a level of unmatched realism. Though light fields technology has been researched and explored in computer graphics for more than 30 years, practical systems for actually delivering high-quality light field experiences has not yet been possible.

Part of the team's motivation behind creating this VR light field experience is to invigorate the nascent field.

"Welcome to Light Fields proves that it is now possible to make a compelling light field VR viewer that runs on consumer-grade hardware, and we hope that this knowledge will encourage others to get involved with building light field technology and media," says Overbeck. "We understand that in order to eventually make compelling consumer products based on light fields, we need a thriving light field ecosystem. We need open light field codecs, we need artists creating beautiful light field imagery, and we need people using VR in order to engage with light fields."
-end-
About ACM, ACM SIGGRAPH, and SIGGRAPH 2018

ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. SIGGRAPH is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2018, marking the 45th annual conference hosted by ACM SIGGRAPH, will take place from 12-16 August at the Vancouver Convention Centre in Vancouver, B.C.

To register for SIGGRAPH 2018 and experience Welcome to Light Fields first hand, visit s2018.siggraph.org/attend/register.

Association for Computing Machinery

Related Virtual Reality Articles:

New 3-D display takes the eye fatigue out of virtual reality
A new type of 3-D display could solve the long-standing problem eye fatigue when using VR and AR equipment by greatly improving the viewing comfort of these wearable devices.
A glove powered by soft robotics to interact with virtual reality environments
Engineers at UC San Diego are using soft robotics technology to make light, flexible gloves that allow users to feel tactile feedback when they interact with virtual reality environments.
The 'reality' of virtual reality pornography
How the latest digital technology could blur the line between reality and fantasy, pushing the dangers of porn to a whole new level.
Physical keyboards make virtual reality typing easier
What's better than a holographic keyboard? A real one, apparently.
Can virtual reality help us prevent falls in the elderly and others?
Every year, falls lead to hospitalization or death for hundreds of thousands of elderly Americans.
Virtual reality therapy helps decrease pain in hospitalized patients
Virtual reality therapy is effective in significantly reducing pain for hospitalized patients, according to a new Cedars-Sinai study.
Using virtual reality to catch a real ball
Disney Research scientists have found innovative ways to enhance virtual experiences involving interactions with physical objects by showing how a person using a virtual reality system can use it to reliably catch a real ball.
Studying altruism through virtual reality
This new study -- published in the journal Neuropsychologia -- immersed participants in a virtual environment that reproduced a building on fire which they had to evacuate in a hurry, deciding whether to save their lives or help rescue an injured person.
Virtual reality training for 'safety-critical' jobs
New virtual reality training could help prevent accidents in 'safety-critical' industries like the NHS, aviation, the military and nuclear power.
New virtual reality technology may improve motor skills in damaged limbs
New Tel Aviv University research suggests that a combination of traditional physical therapy and technology may improve the motor skills and mobility of an impaired hand by having its healthy partner hand lead by example through virtual reality training.

Related Virtual Reality Reading:

Eye Wonder Colorful Encyclopedia Britannica: Virtual Reality Exploration 6 Book Interactive Library Engages Children In A New Way Of Learning
by Encyclopedia Britannica (Publisher)

Defying Reality: The Inside Story of the Virtual Reality Revolution
by David M. Ewalt (Author)

Unity Virtual Reality Projects: Learn Virtual Reality by developing more than 10 engaging projects with Unity 2018, 2nd Edition
by Jonathan Linowes (Author)

Storytelling for Virtual Reality
by John Bucher (Author)

Virtual & Augmented Reality For Dummies (For Dummies (Computer/Tech))
by Paul Mealy (Author)

The VR Book: Human-Centered Design for Virtual Reality (ACM Books)
by Jason Jerald (Author)

Crafting Stories for Virtual Reality
by Melissa Bosworth (Author), Lakshmi Sarah (Author)

Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do
by Jeremy Bailenson (Author)

Unity Virtual Reality Projects: Explore the world of virtual reality by building immersive and fun VR projects using Unity 3D
by Jonathan Linowes (Author)

Future Presence: How Virtual Reality Is Changing Human Connection, Intimacy, and the Limits of Ordinary Life
by Peter Rubin (Author)

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Circular
We're told if the economy is growing, and if we keep producing, that's a good thing. But at what cost? This hour, TED speakers explore circular systems that regenerate and re-use what we already have. Guests include economist Kate Raworth, environmental activist Tristram Stuart, landscape architect Kate Orff, entrepreneur David Katz, and graphic designer Jessi Arrington.
Now Playing: Science for the People

#504 The Art of Logic
How can mathematics help us have better arguments? This week we spend the hour with "The Art of Logic in an Illogical World" author, mathematician Eugenia Cheng, as she makes her case that the logic of mathematics can combine with emotional resonance to allow us to have better debates and arguments. Along the way we learn a lot about rigorous logic using arguments you're probably having every day, while also learning a lot about our own underlying beliefs and assumptions.