New computer system beats humans at guessing gender from face and voice cues

December 12, 2001

University Park, Pa. --- A new computer classification system developed by Penn State computer scientists can correctly identify a person's gender - based only on eyes, nose, mouth and voice cues - better than human beings can. Dr. Rajeev Sharma, associate professor of computer science and engineering, says the new system is right nearly 100 percent of the time. Human beings consistently score in the low 90 percents.

The new system, which is the first computer system to combine both face and voice cues, has potential for use in security systems, market research and human/computer interaction systems as well as other applications. For example, the system could be used to signal when unauthorized individuals tried to enter a restroom, fitting room or dormitory. Law enforcement agents or private citizens could also use it as part of a system designed to scan videotape to find images of a particular individual. The new system could also be used to collect market research data automatically, for example, on how many women and how many men sat behind the wheel of a specific car at a car showroom or selected white versus red chrysanthemums at a garden center.

Sharma adds, "The new system could even be used to provide a richer communication environment for people using human and computer interaction systems." For example, a computer system that helps people locate stores in a mall might be able to provide better service if it could recognize the gender of the person inquiring. The new system is based on powerful pattern recognition software technique, called support vector machines (SVM) that can learn. SVMs have previously been used to scan cell samples for abnormalities or in other applications where patterns are very similar and difficult to separate. Sharma and his research group adapted SVMs separately for face and voice recognition. The Penn State researchers trained the software dedicated to faces on 1755 thumbnail images of human faces from a standard database. The thumbnails showed only the section of the face that includes eyes, nose and mouth - no hair, ears or neck. Another SVM was trained on voice samples. The voice samples also came from a standard database and included just fractions of a second of voice data.

When the face software and voice software each had been trained separately to the level of human proficiency at classifying gender, Sharma and his group added a SVM "manager" to fuse the results, make the final gender classification and improve the system's accuracy. For example, if the face software and voice software disagree on a particular gender classification, the SVM manager reviews both decision processes - with a view to the specific weaknesses of the face and voice software - and makes the final decision. The result is a system that classifies gender correctly more often than human beings supplied with the same data.

Besides Sharma, the inventors also include his former student, Leena A. Walavalkar, and visiting researcher Dr. Mohammed Yeasin. Penn State has submitted a provisional application for a patent for the invention that has been licensed to Sharma's company, Advanced Interface Technologies. Sharma thinks the new system will eventually find application as part of a wide variety of "intelligent" systems. He imagines, for example, a passive security check point system for a 'smart building,' where people would not have to stop to provide identification or even slow down. They could simply walk past the checkpoint, say hi and keep on going - unless they were intruders.

Sharma also notes that the system could be used to help ensure gender equity and equal access to public facilities by alerting officials if one gender predominated.
The research was supported, in part, by a grant from the National Science Foundation.

Besides Sharma, the inventors also include his former student, Leena A. Walavalkar, and visiting researcher Dr. Mohammed Yeasin. Penn State has submitted a provisional application for a patent for the invention that has been licensed to Sharma's company, Advanced Interface Technologies


Penn State

Related Gender Articles from Brightsurf:

Transgender and gender-diverse individuals more likely to be autistic
Transgender and gender-diverse adults are three to six times more likely as cisgender adults (individuals whose gender identity corresponds to their sex assigned at birth) to be diagnosed as autistic, according to a new study by scientists at the University of Cambridge's Autism Research Centre.

Gender bias kept alive by people who think it's dead
Workplace gender bias is being kept alive by people who think it's no longer an issue, new research suggests.

Earliest age gender dysphoria experienced by transgender adults seeking gender-affirming surgery
Data collected from 155 adult transgender women and 55 transgender men were used to identify the earliest age at which gender dysphoria was experienced in this patient population seeking genital gender-affirming surgery at a California hospital.

How many gender subtypes exist in the brain?
In a new paper, published recently in the journal Cerebral Cortex, researchers argue that there are at least nine directions of brain-gender variation.

The neurobiological basis of gender dysphoria
A new theory of gender dysphoria argues the symptoms of the condition are due to changes in network activity, rather than incorrect brain sex, according to work recently published in eNeuro.

Gender quotas in business -- how do Europeans feel?
Despite years trying to bring more women to the top boards of business, the proportion of women on the committees of companies is tiny.

Scientists should have sex and gender on the brain
Thinking about sex and gender would help scientists improve their research, a new article published today argues.

Facial recognition software has a gender problem
A new study of popular facial analysis services found they misidentified trans men as much as 38% of the time, mischaracterized non-binary individuals 100% of the time and appeared to be based on outdated gender stereotypes.

The Wikipedia gender gap
In a recent University of Washington study, researchers interviewed women 'Wikipedians' to examine the lack of female and non-binary editors in Wikipedia.

The Neolithic precedents of gender inequality
Inequality between men and women was not generally consolidated in Iberia during the Neolithic.

Read More: Gender News and Gender Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to