Averting toxic chats: Computer model predicts when online conversations turn sour

July 24, 2018

ITHACA, N.Y. - The internet offers the potential for constructive dialogue and cooperation, but online conversations too often degenerate into personal attacks. In hopes that those attacks can be averted, Cornell University researchers have created a model to predict which civil conversations might take a turn and derail.

After analyzing hundreds of exchanges between Wikipedia editors, the researchers developed a computer program that scans for warning signs in the language used by participants at the start of a conversation - such as repeated, direct questioning or use of the word "you" - to predict which initially civil conversations would go awry.

Early exchanges that included greetings, expressions of gratitude, hedges such as "it seems," and the words "I" and "we" were more likely to remain civil, the study found.

"There are millions of such discussions taking place every day, and you can't possibly monitor all of them live. A system based on this finding might help human moderators better direct their attention," said Cristian Danescu-Niculescu-Mizil, assistant professor of information science and co-author of the paper "Conversations Gone Awry: Detecting Early Signs of Conversational Failure."

"We, as humans, have an intuition of whether a conversation is about to go awry, but it's often just a suspicion. We can't do it 100 percent of the time. We wonder if we can build systems to replicate or even go beyond this intuition," Danescu-Niculescu-Mizil said.

The computer model, which also considered Google's Perspective, a machine-learning tool for evaluating "toxicity," was correct around 65 percent of the time. Humans guessed correctly 72 percent of the time.

People can test their own ability to guess which conversations will derail at an online quiz.

The study analyzed 1,270 conversations that began civilly but degenerated into personal attacks, culled from 50 million conversations across 16 million Wikipedia "talk" pages, where editors discuss articles or other issues. They examined exchanges in pairs, comparing each conversation that ended badly with one that succeeded on the same topic, so the results weren't skewed by sensitive subject matter such as politics.

The paper is co-written with Cornell Ph.D. information science student Justine Zhang; Ph.D. computer science students Jonathan P. Chang, and Yiqing Hua; Lucas Dixon and Nithum Thain of Jigsaw; and Dario Taraborelli of the Wikimedia Foundation.

The researchers hope this model can be used to rescue at-risk conversations and improve online dialogue, rather than for banning specific users or censoring certain topics. Some online posters, such as nonnative English speakers, may not realize they could be perceived as aggressive, and nudges from such a system could help them self-adjust.

"If I have tools that find personal attacks, it's already too late, because the attack has already happened and people have already seen it," Chang said. "But if you understand this conversation is going in a bad direction and take action then, that might make the place a little more welcoming."
-end-
Cornell University has television, ISDN and dedicated Skype/Google+ Hangout studios available for media interviews.

Cornell University

Related Computer Model Articles from Brightsurf:

Computer model explains altered decision making in schizophrenia
Scientists have built a computer 'brain circuit', or artificial neural network, that mirrors human decision-making processes and sheds light on how circuits might be altered in psychiatric diseases.

Computer model shows how COVID-19 could lead to runaway inflammation
New study addresses a mystery first raised in March: Why do some people with COVID-19 develop severe inflammation?

Computer model predicts how drugs affect heart rhythm
UC Davis Health researchers have developed a computer model to screen drugs for unintended cardiac side effects, especially arrhythmia risk.

Computer model described the dynamic instability of microtubules
Researchers of Sechenov University together with their colleagues from several Russian institutes studied the dynamics of microtubules that form the basis of the cytoskeleton and take part in the transfer of particles within a cell and its division.

Computer model helps make sense of human memory
Researchers at the Okinawa Institute of Science and Technology Graduate University (OIST) and the RIKEN Center for Brain Science have created an artificial network to simulate the brain, demonstrating that tinkering with inhibitory circuits leads to extended memory.

Computer model could help test new sickle cell drugs
A new computer model that captures the dynamics of the red blood cell sickling process could help in evaluating drugs for treating sickle cell disease.

Novel computer model supports cancer therapy
Researchers from the Life Sciences Research Unit (LSRU) of the University of Luxembourg have developed a computer model that simulates the metabolism of cancer cells.

Reverse-engineered computer model provides new insights into larval behavior
Scientists have developed a new approach to describe the behaviors of microscopic marine larvae, which will improve future predictions of how they disperse and distribute.

New computer-aided model may help predict sepsis
Can a computer-aided model predict life-threatening sepsis? A model developed in the UK that uses routinely collected data to identify early symptoms of sepsis, published in CMAJ, shows promise.

'NarcoLogic' computer model shows unintended consequences of cocaine interdiction
Efforts to curtail the flow of cocaine into the United States from South America have made drug trafficking operations more widespread and harder to eradicate.

Read More: Computer Model News and Computer Model Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.