Smile! Photos converted into 3D from any mobile device

June 25, 2020

CHICAGO--It's not an exaggeration -- many of us have seemingly turned into skilled photographers overnight. With the rapid advances in handheld devices and easy-to-use photo-editing applications, people have been accustomed to snapping their own photos from their phones or tablets for years now. Some of us also are getting savvier and more creative with how photos are shared or posted.

In this new work from Facebook researchers, users are now able to turn the photos they take on their devices into 3D images within seconds. The team will demonstrate their innovative end-to-end system for creating and viewing 3D photos at SIGGRAPH 2020. The conference, which will take place virtually this year starting 17 August, gathers a diverse network of professionals who approach computer graphics and interactive techniques from different perspectives. SIGGRAPH continues to serve as the industry's premier venue for showcasing forward-thinking ideas and research.

The 2D-to-3D photo technique has been available as a "photos feature" on Facebook since late 2018. To take advantage of this feature, originally Facebook users were required to capture photos with a phone equipped with a dual-lens camera. Now, the Facebook team has added an algorithm that automates depth estimation from the 2D input image, and the technique can be utilized directly on any mobile device, expanding the method beyond just the Facebook app and without the requirement of having a dual-lens camera.

"Over the last century, photography has gone through several tech 'upgrades' that increased the level of immersion. Initially, all photos were black and white and grainy, then came color photography, and then digital photography brought us higher quality and better-resolution images," Johannes Kopf, lead author of the work and research scientist at Facebook, says. "Finally, these days we have 3D photography, which makes photos feel a lot more alive and real."

The new framework provides users with a more practical approach to 3D photography, addressing several design objectives. Users can access the new technology via their own mobile device; the real-time conversion from a 2D input image to 3D is seamless, requiring no sophisticated photographic skills by the user and only takes a few seconds to process; and the method is robust enough to work on almost any photo -- new or one previously taken.

To refine the new system, the researchers trained a convolutional neural network (CNN) on millions of pairs of public 3D images and their accompanying depth maps and leveraged mobile-optimization techniques developed by Facebook AI. The framework also incorporates texture inpainting and geometry capture of the 2D input image to convert it into 3D, resulting in images that are more active and lively. Each automated step that converts a user's 2D photo, directly from their mobile device, is optimized to run on a variety of makes and models and is able to work with a device's limited memory and data-transfer capabilities. The best part? Users get instant gratification, as the 3D results are literally generated in a matter of seconds.

Researchers at Facebook have been working toward new and inventive ways to create high quality, immersive 3D experiences, pushing the envelope in computer vision, graphics, and machine learning. In future work, the team is investigating machine-learning methods that enable high-quality depth estimation for videos taken with mobile devices.
-end-
In addition to Kopf, the research team at Facebook who collaborated on "One Shot 3D Photography" includes Kevin Matzen, Suhib Alsisan, Ocean Quigley, Francis Ge, Yangming Chong, Josh Patterson, Jan-Michael Frahm, Shu Wu, Matthew Yu, Peizhao Zhang, Zijian He, Peter Vajda, Ayush Saraf, and Michael Cohen.

About ACM, ACM SIGGRAPH and SIGGRAPH 2020

ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. The SIGGRAPH conference is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2020, the 47th annual conference hosted by ACM SIGGRAPH, will take place virtually.

Association for Computing Machinery

Related Facebook Articles from Brightsurf:

Facebook political ads more partisan, less negative than TV
More political candidates may be shifting primarily to social media to advertise rather than TV, according to a study of advertising trends from the 2018 campaign season.

Facebook anniversaries inspire reflection, nostalgia
Posted on Facebook, milestones such as birthdays and anniversaries prompt users to reflect on the passage of time and the patterns of their lives -- and help the social media giant recycle content in order to boost engagement, according to new Cornell research.

Healthier and happier without Facebook
Two weeks of 20 minutes less time per day on Facebook: a team of psychologists from Ruhr-Universit├Ąt Bochum (RUB) invited 140 test persons to participate in this experiment.

Facebook language changes before an emergency hospital visit
A new study published in Nature Scientific Reports reveals that the language people use on Facebook subtly changes before they make a visit to the emergency department (ED).

When college students post about depression on Facebook
When college students post about feelings of depression on Facebook, their friends are unlikely to encourage them to seek help, a small study suggests.

Quitting Facebook could boost exam results
In research that validates what many parents and educators suspect, students whose grades are below average could boost their exam results if they devoted less time to Facebook and other social networking sites.

Depressed by Facebook and the like
Great holiday, fantastic party, adorable children, incredible food: everyone shows their life in the best light on social networks.

Can Facebook improve your mental health?
Contrary to popular belief, using social media and the internet regularly could improve mental health among adults and help fend off serious psychological distress, such as depression and anxiety, finds a new Michigan State University study.

How stress leads to Facebook addiction
Friends on social media such as Facebook can be a great source of comfort during periods of stress.

The dead may outnumber the living on Facebook within 50 years
New analysis by academics from the Oxford Internet Institute (OII), part of the University of Oxford, predicts the dead may outnumber the living on Facebook within 50 years, a trend that will have grave implications for how we treat our digital heritage in the future.

Read More: Facebook News and Facebook Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.