Computer graphics research team to present new tool for sketching faces

July 02, 2018

CHICAGO--Across popular social media platforms, users are posting countless images every day. On Instagram alone, there are more than 40 billion uploaded photos--a figure that's skyrocketing by 95 million daily. This presents a clear need for intuitive yet robust photo-editing tools that allow the average user to perform advanced editing functions.

And while there is a definite need for interactive image editing with respect to social media, improved editing tools and systems also remain an important aspect of computer graphics and computer vision. However, there is a lack of tools that feature more complex editing functions for inexperienced users, such as changing the facial expression in a photo.

A research team, led by computer scientists from the University of Bern-Switzerland and University of Maryland-College Park, have devised a sketch-based editing framework that enables a user to edit their photos by "sketching" a few strokes on top of them. Their system, called FaceShop, also offers a copy-paste function, which allows users to edit any part of a photo by copying-and-pasting the portion to be edited from another (better) photo, eliminating the need to hand-draw or sketch anything at all.

The team's approach is built on machine-learning techniques, which, in the end, give users more control over their desired edits in real-time and produce more realistic results.

"Most other approaches rely on more traditional, handcrafted techniques, which impose some limitations. For instance, these systems are either [by design] restricted to limited sets of predefined editing operations, or they are very flexible but hard to use and require experienced users to spend a considerable amount of time to perform rather basic edits," says Tiziano Protenier, lead author of the work and PhD candidate at the University of Bern. "In contrast, our system is very flexible and allows untrained users to perform complex edits within minutes using an intuitive interface."

Protenier developed the new system with collaborators Qiyang Hu, Attila Szabó, Siavash Arjomand Bigdeli and Paolo Favaro from University of Bern, and Matthias Zwicker from University of Maryland. The team will present their work at SIGGRAPH 2018, held 12-16 August in Vancouver, British Columbia. This annual gathering showcases the world's leading professionals, academics, and creative minds at the forefront of computer graphics and interactive techniques.

In their paper, "FaceShop: Deep Sketch-based Face Image Editing," the researchers demonstrate their new system via several examples. In one, a woman's nose is slightly manipulated, and a strand of hair is edited so that it is pushed away from her face, removing a dark shadow that had appeared on just one side of her face in the original photo. Another photo shows how a user is able to enhance a woman's eye make-up and bring out the color of her eyes. The examples highlighted in the paper showcase how the novel system works, intuitively, and produces high-quality, realistic results.

The team's method is based on generative adversarial neural networks (GANs), a form of artificial intelligence (AI) that, in recent years, has attracted a lot of research interest for its ability to generate realistic looking images. These GANs consist of two AIs that fight against each other. The first component tries to distinguish the generated images from genuine images, whereas the second component tries to produce images that fool the other AI. During training, the two components learn from each other, eventually resulting in a system that autonomously learned to produce realistic looking images, without any human judgement in the loop.

Unlike other AI-driven image editing methods, once the system is trained, users are more in control of the edits to their photos. "Our system features an optimum degree of control, which makes it very interesting from an application point of view," notes Portenier. "Other techniques that try to incorporate user control are either limited in the scope of possible edits or have been shown to work well on the data that was used to train the systems, but fail to produce convincing results with real user input. We are presenting a technique that mitigates this issue, resulting in a system that works surprisingly well in practice."

In future work, the researchers intend to explore additional user interaction tools to add to their framework, and consider how to leverage AI for sketch-based editing of videos.
For the full paper and video, visit the team's project page.


ACM, the Association for Computing Machinery, is the world's largest educational and scientific computing society, uniting educators, researchers, and professionals to inspire dialogue, share resources, and address the field's challenges. ACM SIGGRAPH is a special interest group within ACM that serves as an interdisciplinary community for members in research, technology, and applications in computer graphics and interactive techniques. SIGGRAPH is the world's leading annual interdisciplinary educational experience showcasing the latest in computer graphics and interactive techniques. SIGGRAPH 2018, marking the 45th annual conference hosted by ACM SIGGRAPH, will take place from 12-16 August at the Vancouver Convention Centre in Vancouver, B.C.

To register for SIGGRAPH 2018 and hear from the authors themselves, visit

Association for Computing Machinery

Related Social Media Articles from Brightsurf:

it's not if, but how people use social media that impacts their well-being
New research from UBC Okanagan indicates what's most important for overall happiness is how a person uses social media.

Social media postings linked to hate crimes
A new paper in the Journal of the European Economic Association, published by Oxford University Press, explores the connection between social media and hate crimes.

How Steak-umm became a social media phenomenon during the pandemic
A new study outlines how a brand of frozen meat products took social media by storm - and what other brands can learn from the phenomenon.

COVID-19: Social media users more likely to believe false information
A new study led by researchers at McGill University finds that people who get their news from social media are more likely to have misperceptions about COVID-19.

Stemming the spread of misinformation on social media
New research reported in the journal Psychological Science finds that priming people to think about accuracy could make them more discerning in what they subsequently share on social media.

Looking for better customer engagement value? Be more strategic on social media
According to a new study from the University of Vaasa and University of Cyprus, the mere use of social media alone does not generate customer value, but rather, the connections and interactions between the firm and its customers -- as well as among customers themselves -- can be used strategically for resource transformation and exchanges between the interacting parties.

Exploring the use of 'stretchable' words in social media
An investigation of Twitter messages reveals new insights and tools for studying how people use stretched words, such as 'duuuuude,' 'heyyyyy,' or 'noooooooo.' Tyler Gray and colleagues at the University of Vermont in Burlington present these findings in the open-access journal PLOS ONE on May 27, 2020.

How social media platforms can contribute to dehumanizing people
A recent analysis of discourse on Facebook highlights how social media can be used to dehumanize entire groups of people.

Social media influencers could encourage adolescents to follow social distancing guidelines
Public health bodies should consider incentivizing social media influencers to encourage adolescents to follow social distancing guidelines, say researchers.

Social grooming factors influencing social media civility on COVID-19
A new study analyzing tweets about COVID-19 found that users with larger social networks tend to use fewer uncivil remarks when they have more positive responses from others.

Read More: Social Media News and Social Media Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to