Nav: Home

Google unveils new virtual reality experience at ACM SIGGRAPH 2018

July 26, 2018

CHICAGO--Google Inc. has unveiled a new virtual reality (VR) immersive experience based on a novel system that captures and renders high-quality, realistic images from the real world using light fields. Created by a team of leading researchers at Google, Welcome to Light Fields is the tech giant's splash into the nascent arena of light fields VR experiences, an exciting corner of VR video technology gaining traction for its promise to deliver extremely high-quality imagery and experiences in the virtual world.

Google released Welcome to Light Fields earlier this year as a free app on Steam VR for HTC Vive, Oculus Rift, and Windows Mixed Reality headsets. The creators will demonstrate the VR experience at SIGGRAPH 2018, in the Immersive Pavilion, a new space for this year's conference. The Pavilion is devoted exclusively to virtual, augmented, and mixed reality and will contain: the Vrcade, a space for VR, AR, and MR games or experiences; the VR Theater, a storytelling extravaganza that is part of the Computer Animation Festival; and the well-known Village, for showcasing large-scale projects. SIGGRAPH 2018, held 12-16 August in Vancouver, British Columbia, is an annual gathering that showcases the world's leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.

Destinations in Welcome to Light Fields include NASA's Space Shuttle Discovery, delivering to viewers an astronaut's view inside the flight deck, which has never been open to the public; the pristine teak and mahogany interiors of the Gamble House, an architectural treasure in Pasadena, CA; and the glorious St. Stephen's Church in Granada Hills, CA, home to a stunning wall of more than 14,000 pieces of glimmering stained glass.

"I love that light fields in VR can teleport you to exotic places in the real world, and truly make you believe you are there," says Ryan Overbeck, software engineer at Google who co-led the project. "To me, this is magic."

To bring this experience to life, Overbeck worked with a team that included Paul Debevec, senior staff engineer at Google, who managed the project and led the hardware piece with engineers Xueming Yu, Jay Busch, and Graham Fyffe. With Overbeck, Daniel Erickson and Daniel Evangelakos focused on the software end. The researchers designed a comprehensive system for capturing and rendering high-quality, spherical light field still images from footage captured in the real world. They developed two easy-to-use light field camera rigs, based on the GoPro Hero4action sports camera, that efficiently capture thousands of images on the surface of a sphere. Those images were then passed through a cloud-based light-field-processing pipeline.

Among other things, explains Overbeck, "The processing pipeline uses computer vision to place the images in 3D and generate depth maps, and we use a modified version of our vp9 video codec [video formatting] to compress the light field data down to a manageable size." To render a light field dataset, he notes, the team used a rendering algorithm that blends between the thousands of light field images in real-time.

The team relied on Google's talented pool of engineers in computer vision, graphics, video compression, and machine learning to overcome the unique challenges posed in light fields technology. They also collaborated closely with the WebM team (who make the vp9 video codec) to develop the high-quality light field compression format incorporated into their system, and leaned heavily on the expertise of the Jump VR team to help pose the images and generate depth maps. (Jump is Google's professional VR system for achieving 3D-360 video production at scale.)

Indeed, with Welcome to Light Fields, the Google team is demonstrating the potential and promise of light field VR technology, showcasing the technology's ability to provide a truly immersive experience with a level of unmatched realism. Though light fields technology has been researched and explored in computer graphics for more than 30 years, practical systems for actually delivering high-quality light field experiences has not yet been possible.

Part of the team's motivation behind creating this VR light field experience is to invigorate the nascent field.

"Welcome to Light Fields proves that it is now possible to make a compelling light field VR viewer that runs on consumer-grade hardware, and we hope that this knowledge will encourage others to get involved with building light field technology and media," says Overbeck. "We understand that in order to eventually make compelling consumer products based on light fields, we need a thriving light field ecosystem. We need open light field codecs, we need artists creating beautiful light field imagery, and we need people using VR in order to engage with light fields."
-end-
About ACM, ACM SIGGRAPH, and SIGGRAPH 2018

ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. SIGGRAPH is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2018, marking the 45th annual conference hosted by ACM SIGGRAPH, will take place from 12-16 August at the Vancouver Convention Centre in Vancouver, B.C.

To register for SIGGRAPH 2018 and experience Welcome to Light Fields first hand, visit s2018.siggraph.org/attend/register.

Association for Computing Machinery

Related Virtual Reality Articles:

New 3-D display takes the eye fatigue out of virtual reality
A new type of 3-D display could solve the long-standing problem eye fatigue when using VR and AR equipment by greatly improving the viewing comfort of these wearable devices.
A glove powered by soft robotics to interact with virtual reality environments
Engineers at UC San Diego are using soft robotics technology to make light, flexible gloves that allow users to feel tactile feedback when they interact with virtual reality environments.
The 'reality' of virtual reality pornography
How the latest digital technology could blur the line between reality and fantasy, pushing the dangers of porn to a whole new level.
Physical keyboards make virtual reality typing easier
What's better than a holographic keyboard? A real one, apparently.
Can virtual reality help us prevent falls in the elderly and others?
Every year, falls lead to hospitalization or death for hundreds of thousands of elderly Americans.
Virtual reality therapy helps decrease pain in hospitalized patients
Virtual reality therapy is effective in significantly reducing pain for hospitalized patients, according to a new Cedars-Sinai study.
Using virtual reality to catch a real ball
Disney Research scientists have found innovative ways to enhance virtual experiences involving interactions with physical objects by showing how a person using a virtual reality system can use it to reliably catch a real ball.
Studying altruism through virtual reality
This new study -- published in the journal Neuropsychologia -- immersed participants in a virtual environment that reproduced a building on fire which they had to evacuate in a hurry, deciding whether to save their lives or help rescue an injured person.
Virtual reality training for 'safety-critical' jobs
New virtual reality training could help prevent accidents in 'safety-critical' industries like the NHS, aviation, the military and nuclear power.
New virtual reality technology may improve motor skills in damaged limbs
New Tel Aviv University research suggests that a combination of traditional physical therapy and technology may improve the motor skills and mobility of an impaired hand by having its healthy partner hand lead by example through virtual reality training.

Related Virtual Reality Reading:

Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do
by Jeremy Bailenson (Author)

Virtual Reality
by Jack Challoner (Author)

Virtual & Augmented Reality For Dummies (For Dummies (Computer/Tech))
by Paul Mealy (Author)

Dawn of the New Everything: Encounters with Reality and Virtual Reality
by Jaron Lanier (Author)

Storytelling for Virtual Reality
by John Bucher (Author)

Defying Reality: The Inside Story of the Virtual Reality Revolution
by David M. Ewalt (Author)

Virtual Realities (Adventures in Odyssey)
by AIO Team (Author)

Learning Virtual Reality: Developing Immersive Experiences and Applications for Desktop, Web, and Mobile
by Tony Parisi (Author)

Future Presence: How Virtual Reality Is Changing Human Connection, Intimacy, and the Limits of Ordinary Life
by Peter Rubin (Author)

Practical Augmented Reality: A Guide to the Technologies, Applications, and Human Factors for AR and VR (Usability)
by Steve Aukstakalnis (Author)

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Hacking The Law
We have a vision of justice as blind, impartial, and fair — but in reality, the law often fails those who need it most. This hour, TED speakers explore radical ways to change the legal system. Guests include lawyer and social justice advocate Robin Steinberg, animal rights lawyer Steven Wise, political activist Brett Hennig, and lawyer and social entrepreneur Vivek Maru.
Now Playing: Science for the People

#495 Earth Science in Space
Some worlds are made of sand. Some are made of water. Some are even made of salt. In science fiction and fantasy, planet can be made of whatever you want. But what does that mean for how the planets themselves work? When in doubt, throw an asteroid at it. This is a live show recorded at the 2018 Dragon Con in Atlanta Georgia. Featuring Travor Valle, Mika McKinnon, David Moscato, Scott Harris, and moderated by our own Bethany Brookshire. Note: The sound isn't as good as we'd hoped but we love the guests and the conversation and we wanted to...