Nav: Home

Hate speech-detecting AIs are fools for 'love'

September 14, 2018

State-of-the-art detectors that screen out online hate speech can be easily duped by humans, shows new study.

Hateful text and comments are an ever-increasing problem in online environments, yet addressing the rampant issue relies on being able to identify toxic content. A new study by the Aalto University Secure Systems research group https://ssg.aalto.fi has discovered weaknesses in many machine learning detectors currently used to recognize and keep hate speech at bay.

Many popular social media and online platforms use hate speech detectors that a team of researchers led by Professor N. Asokan have now shown to be brittle and easy to deceive. Bad grammar and awkward spelling--intentional or not--might make toxic social media comments harder for AI detectors to spot.

The team put seven state-of-the-art hate speech detectors to the test. All of them failed.

Modern natural language processing techniques (NLP) can classify text based on individual characters, words or sentences. When faced with textual data that differs from that used in their training, they begin to fumble.

'We inserted typos, changed word boundaries or added neutral words to the original hate speech. Removing spaces between words was the most powerful attack, and a combination of these methods was effective even against Google's comment-ranking system Perspective,' says Tommi Gröndahl, doctoral student at Aalto University.

Google Perspective ranks the 'toxicity' of comments using text analysis methods. In 2017, researchers from the University of Washington showed that Google Perspective can be fooled by introducing simple typos. Gröndahl and his colleagues have now found that Perspective has since become resilient to simple typos yet can still be fooled by other modifications such as removing spaces or adding innocuous words like 'love'.

A sentence like 'I hate you' slipped through the sieve and became non-hateful when modified into 'Ihateyou love'.

The researchers note that in different contexts the same utterance can be regarded either as hateful or merely offensive. Hate speech is subjective and context-specific, which renders text analysis techniques insufficient as stand-alone solutions.

The researchers recommend that more attention be paid to the quality of data sets used to train machine learning models--rather than refining the model design. The results indicate that character-based detection could be a viable way to improve current applications.
-end-
The study was carried out in collaboration with researchers from University of Padua in Italy. The results will be presented at the ACM AISec workshop in October.

The study is part of an ongoing project called Deception Detection via Text Analysis https://ssg.aalto.fi/research/projects/deception-detection-via-text-analysis in the Secure Systems group https://ssg.aalto.fi> at Aalto University.

Research article:

Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, N.Asokan:
All You Need is "Love": Evading Hate-speech Detection.
https://arxiv.org/abs/1808.09115

More information:

Tommi Gröndahl, Doctoral Candidate
Aalto University
Secure Systems group
tommi.grondahl@aalto.fi
tel. +358 400 426 523

N. Asokan, Professor
Aalto University
Secure Systems group
n.asokan@aalto.fi
tel. +358 50 483 6465

Aalto University

Related Perspective Articles:

'Own-point-of-view' video method leverages power of perception to improve emergency care
The 'own-point-of-view' perspective video technique, coupled with a subjective re situ interview, provides a better understanding of how physicians make clinical decisions in an authentic treatment setting, compared with the conventional external perspective.
An integrated perspective on diabetic, alcoholic, and drug-induced neuropathy
Neuropathic pain (NeuP) is a persistent, debilitating form of chronic pain that results from damaged nerves.
New perspective: Vegetation phenology variability based on tibetan plateau tree-ring data
Recently, a research group headed by Prof. YANG Bao from the Key Laboratory of Desert and Desertification, Northwest Institute of Eco-Environment and Resources of the Chinese Academy of Sciences, together with coauthors from Russia, Germany, Canada and Sweden, has reconciled these conflicting results based on a 55-year series of vegetation phenology for the TP derived from well-validated process-based Vaganov-Shashkin model (V-S) simulations of tree-ring growth data.
You spy with your little eye -- dogs can adopt the perspective of humans
Humans are able to interpret the behaviour of others by attributing mental states to them.
A new perspective on the European colonization of Asia
Although James Cook's 18th century expeditions into the South Pacific Ocean are considered historical feats, Spanish voyages of discovery in this region preceded them.
A systems biology perspective on molecular cytogenetics
Professor Henry Heng's team, from the medical school at Wayne State University, has published a perspective article titled A Systems Biology Perspective on Molecular Cytogenetics to address the issue.
Overcoming egocentricity increases self-control
Neurobiological models of self-control usually focus on brain mechanisms involved in impulse control and emotion regulation.
Is Earthly life premature from a cosmic perspective?
The universe is 13.8 billion years old, while our planet formed just 4.5 billion years ago.
JBMR perspective: A crisis in the treatment of osteoporosis
The remarkable progress made over the past 30 years to reduce fractures and dramatically improve the quality of life for millions of osteoporosis patients is rapidly being reversed, say two bone health experts in a Journal of Bone and Mineral Research article published online today.
Religious beliefs don't always lead to violence
From the Christian Crusades to the Paris attacks, countless conflicts and acts of violence have been claimed to be the result of differing religious beliefs.

Related Perspective Reading:

Framed Perspective Vol. 1: Technical Perspective and Visual Storytelling
by Marcos Mateu-Mestre (Author)

Perspective Made Easy (Dover Art Instruction)
by Ernest R. Norling (Author)

Perspectives on the World Christian Movement: A Reader (Perspectives)
by Ralph D. Winter (Editor), Steven C. Hawthorne (Editor)

Basic Perspective Drawing: A Visual Approach
by John Montague (Author)

Framed Perspective Vol. 2: Technical Drawing for Shadows, Volume, and Characters
by Marcos Mateu-Mestre (Author)

Perspective (Artist's Library series #13)
by William F Powell (Author)

Signs and Symptoms Analysis from a Functional Perspective
by Dr. Dicken Weatherby (Author)

The Art of Perspective: The Ultimate Guide for Artists in Every Medium
by Phil Metzger (Author)

Perspective: A Journey of Demonstrated Thought Change
by Dr. Benjamin Andrew (Author)

Perspective! for Comic Book Artists: How to Achieve a Professional Look in your Artwork
by David Chelsea (Author)

Best Science Podcasts 2018

We have hand picked the best science podcasts for 2018. Sit back and enjoy new science podcasts updated daily from your favorite science news services and scientists.
Now Playing: TED Radio Hour

Dying Well
Is there a way to talk about death candidly, without fear ... and even with humor? How can we best prepare for it with those we love? This hour, TED speakers explore the beauty of life ... and death. Guests include lawyer Jason Rosenthal, humorist Emily Levine, banker and travel blogger Michelle Knox, mortician Caitlin Doughty, and entrepreneur Lux Narayan.
Now Playing: Science for the People

#492 Flint Water Crisis
This week we dig into the Flint water crisis: what happened, how it got so bad, what turned the tide, what's still left to do, and the mix of science, politics, and activism that are still needed to finish pulling Flint out of the crisis. We spend the hour with Dr Mona Hanna-Attisha, a physician, scientist, activist, the founder and director of the Pediatric Public Health Initiative, and author of the book "What the Eyes Don't See: A Story of Crisis, Resistance, and Hope in an American City".