When prompts to draft work emails and job applications contain language commonly used by women, AI chatbots including ChatGPT offer less sophisticated responses compared to office correspondence requested with language associated with men, new Johns Hopkins University research finds.
Every popular AI chatbot tested picked up on subtle, gender-oriented language patterns that writers likely aren’t aware of. The findings suggest that as people increasingly rely on AI for professional communication, the tools might disadvantage women—or anyone whose prompts include those linguistic features.
“If you prompt a model to write an email you’re going to send to someone else at your company, and you’re using language features that women more commonly use, you’ll get back a response that’s less complex, at a lower grade level, and less formal,” said senior author Anjalie Field , a Johns Hopkins computer scientist who studies ethics and discrimination in AI. “That’s going to reflect on how the recipient of that document perceives you.”
The work will be presented at the Oct. 6-9 Conference on Language Modeling in San Francisco.
Researchers have previously demonstrated that AI large language models like ChatGPT can display bias and play into gender stereotypes. But less studied is whether they perform differently for different types of people.
“In American English, men and women just talk differently, and there’s a number of features that are well documented as being good discriminators between male speech and female speech,” said lead author Katherine Van Koevering , an inaugural postdoctoral fellow with the university’s Data Science and AI Institute . “Our idea was, if we feed these features into a model, will it pick up on them and will it respond differently?”
The team took real chatbot prompts for workplace correspondence—emails, job applications and resignation letters—and added language associated with women. These gendered signatures included hedging (“maybe,” “I think”), collective phrasing (“we,” “our team”) and expressive adjectives (“lovely,” “wonderful”).
The team fed the prompts into four popular AI systems—GPT-4, Llama, Gemma, and Mistral. No matter the model, prompts with woman-associated language consistently returned work emails and other professional correspondence that were less sophisticated and less formal.
Language associated with men produced longer, more complex and more formal responses, the team found.
The model wasn’t just mimicking the writer’s tone — the gap persisted even after the researchers accounted for that.
In one example, two prompts request a response to a thank-you email.
Response to the male-coded prompt : I am writing to acknowledge your recent email expressing your gratitude. I sincerely appreciate your kind words and the time you took to write to me. It was indeed a pleasure being of assistance to you, and I am glad to know that you were satisfied with the service you received.
Response to the female-coded prompt : We were absolutely delighted to receive your wonderfully appreciative email earlier. Your words of praise and acknowledgment have indeed warmed our hearts and brought immense satisfaction to our team.
“I was just so surprised by how different the responses were,” Van Koevering said. “Some responses were so bad I couldn’t believe the model would suggest it. And sometimes I just thought, “Wow I should be more careful in my emails.’”
In a particularly striking finding, the team tested whether including a traditionally male or female name changed the AI’s response. It had virtually no effect.
“We thought if you ask the AI for an email with a women-associated linguistic prompt, but sign it ‘John,’ the model would pick up on the ‘John’ more strongly than the ‘would you kindly write me an email?’” Van Koevering said. “But no, you get the same response and it just says John at the end.”
The language patterns in question are largely unconscious and extremely difficult to change, say the researchers, who predict that as people begin to use voice systems more to interact with AI, the biases could become even more pronounced.
“Language is hard for people to control,” Van Koevering said. “The companies need to fix the models, rather than putting all of the burden on the user.”
Next the team hopes to investigate whether similar effects appear across other demographics like age, race and ethnicity. They would also like to study whether AI users, over time, adapt their communication style to match what AI rewards.
It’s How You Ask: Gender-Associated Linguistic Bias in LLMs