Nav: Home

Google unveils new virtual reality experience at ACM SIGGRAPH 2018

July 26, 2018

CHICAGO--Google Inc. has unveiled a new virtual reality (VR) immersive experience based on a novel system that captures and renders high-quality, realistic images from the real world using light fields. Created by a team of leading researchers at Google, Welcome to Light Fields is the tech giant's splash into the nascent arena of light fields VR experiences, an exciting corner of VR video technology gaining traction for its promise to deliver extremely high-quality imagery and experiences in the virtual world.

Google released Welcome to Light Fields earlier this year as a free app on Steam VR for HTC Vive, Oculus Rift, and Windows Mixed Reality headsets. The creators will demonstrate the VR experience at SIGGRAPH 2018, in the Immersive Pavilion, a new space for this year's conference. The Pavilion is devoted exclusively to virtual, augmented, and mixed reality and will contain: the Vrcade, a space for VR, AR, and MR games or experiences; the VR Theater, a storytelling extravaganza that is part of the Computer Animation Festival; and the well-known Village, for showcasing large-scale projects. SIGGRAPH 2018, held 12-16 August in Vancouver, British Columbia, is an annual gathering that showcases the world's leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.

Destinations in Welcome to Light Fields include NASA's Space Shuttle Discovery, delivering to viewers an astronaut's view inside the flight deck, which has never been open to the public; the pristine teak and mahogany interiors of the Gamble House, an architectural treasure in Pasadena, CA; and the glorious St. Stephen's Church in Granada Hills, CA, home to a stunning wall of more than 14,000 pieces of glimmering stained glass.

"I love that light fields in VR can teleport you to exotic places in the real world, and truly make you believe you are there," says Ryan Overbeck, software engineer at Google who co-led the project. "To me, this is magic."

To bring this experience to life, Overbeck worked with a team that included Paul Debevec, senior staff engineer at Google, who managed the project and led the hardware piece with engineers Xueming Yu, Jay Busch, and Graham Fyffe. With Overbeck, Daniel Erickson and Daniel Evangelakos focused on the software end. The researchers designed a comprehensive system for capturing and rendering high-quality, spherical light field still images from footage captured in the real world. They developed two easy-to-use light field camera rigs, based on the GoPro Hero4action sports camera, that efficiently capture thousands of images on the surface of a sphere. Those images were then passed through a cloud-based light-field-processing pipeline.

Among other things, explains Overbeck, "The processing pipeline uses computer vision to place the images in 3D and generate depth maps, and we use a modified version of our vp9 video codec [video formatting] to compress the light field data down to a manageable size." To render a light field dataset, he notes, the team used a rendering algorithm that blends between the thousands of light field images in real-time.

The team relied on Google's talented pool of engineers in computer vision, graphics, video compression, and machine learning to overcome the unique challenges posed in light fields technology. They also collaborated closely with the WebM team (who make the vp9 video codec) to develop the high-quality light field compression format incorporated into their system, and leaned heavily on the expertise of the Jump VR team to help pose the images and generate depth maps. (Jump is Google's professional VR system for achieving 3D-360 video production at scale.)

Indeed, with Welcome to Light Fields, the Google team is demonstrating the potential and promise of light field VR technology, showcasing the technology's ability to provide a truly immersive experience with a level of unmatched realism. Though light fields technology has been researched and explored in computer graphics for more than 30 years, practical systems for actually delivering high-quality light field experiences has not yet been possible.

Part of the team's motivation behind creating this VR light field experience is to invigorate the nascent field.

"Welcome to Light Fields proves that it is now possible to make a compelling light field VR viewer that runs on consumer-grade hardware, and we hope that this knowledge will encourage others to get involved with building light field technology and media," says Overbeck. "We understand that in order to eventually make compelling consumer products based on light fields, we need a thriving light field ecosystem. We need open light field codecs, we need artists creating beautiful light field imagery, and we need people using VR in order to engage with light fields."
-end-
About ACM, ACM SIGGRAPH, and SIGGRAPH 2018

ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. SIGGRAPH is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2018, marking the 45th annual conference hosted by ACM SIGGRAPH, will take place from 12-16 August at the Vancouver Convention Centre in Vancouver, B.C.

To register for SIGGRAPH 2018 and experience Welcome to Light Fields first hand, visit s2018.siggraph.org/attend/register.

Association for Computing Machinery

Related Virtual Reality Articles:

Easing the burden of coronavirus with virtual reality
A new article discusses the psychological stresses imposed by the coronavirus pandemic and suggests that virtual reality can help alleviate the psychological impact of the need for social isolation.
Virtual reality makes empathy easier
Virtual reality activates brain networks that increase your ability to identify with other people, according to new research published in eNeuro.
Physiotherapy could be done at home using virtual reality
Virtual reality could help physiotherapy patients complete their exercises at home successfully thanks to researchers at WMG, University of Warwick, who managed to combine VR technology with 3D motion capture.
Using virtual reality to help individuals with autism spectrum disorder
Novel interventions using virtual reality to aid individuals with autism spectrum disorder (ASD) handle common scenarios may include helping youngsters navigate air travel.
Virtual reality illuminates the power of opioid-associated memories
The brain acts differently when remembering environments associated with drug use.
Virtual reality could help flu vaccination rates
Using a virtual reality simulation to show how flu spreads and its impact on others could be a way to encourage more people to get a flu vaccination, according to a study by researchers at the University of Georgia and the Oak Ridge Associated Universities in Oak Ridge, Tennessee.
Virtual reality becomes more real
Scientists from Skoltech ADASE (Advanced Data Analytics in Science and Engineering) lab have found a way to enhance depth map resolution, which should make virtual reality and computer graphics more realistic.
Is virtual reality the next big thing in art therapy?
Researchers from Drexel University's College of Nursing and Health Professions in the Creative Arts Therapies Department conducted a study to see if virtual reality can be used as an expressive tool in art therapy.
Intuitive in the virtual reality
Through the crafty use of magnetic fields, scientists from HZDR and Johannes Kepler University in Linz have developed the first electronic sensor that can simultaneously process both touchless and tactile stimuli.
An artificial skin that can help rehabilitation and enhance virtual reality
EPFL scientists have developed a soft artificial skin that provides haptic feedback and -- thanks to a sophisticated self-sensing mechanism -- has the potential to instantaneously adapt to a wearer's movements.
More Virtual Reality News and Virtual Reality Current Events

Trending Science News

Current Coronavirus (COVID-19) News

Top Science Podcasts

We have hand picked the top science podcasts of 2020.
Now Playing: TED Radio Hour

Climate Mindset
In the past few months, human beings have come together to fight a global threat. This hour, TED speakers explore how our response can be the catalyst to fight another global crisis: climate change. Guests include political strategist Tom Rivett-Carnac, diplomat Christiana Figueres, climate justice activist Xiye Bastida, and writer, illustrator, and artist Oliver Jeffers.
Now Playing: Science for the People

#562 Superbug to Bedside
By now we're all good and scared about antibiotic resistance, one of the many things coming to get us all. But there's good news, sort of. News antibiotics are coming out! How do they get tested? What does that kind of a trial look like and how does it happen? Host Bethany Brookeshire talks with Matt McCarthy, author of "Superbugs: The Race to Stop an Epidemic", about the ins and outs of testing a new antibiotic in the hospital.
Now Playing: Radiolab

Speedy Beet
There are few musical moments more well-worn than the first four notes of Beethoven's Fifth Symphony. But in this short, we find out that Beethoven might have made a last-ditch effort to keep his music from ever feeling familiar, to keep pushing his listeners to a kind of psychological limit. Big thanks to our Brooklyn Philharmonic musicians: Deborah Buck and Suzy Perelman on violin, Arash Amini on cello, and Ah Ling Neu on viola. And check out The First Four Notes, Matthew Guerrieri's book on Beethoven's Fifth. Support Radiolab today at Radiolab.org/donate.