Performance of gender detection tools: a comparative study of name-to-gender inference services
Journal of the Medical Library Association · 05.10.2021 · 109 (3)
DOI: 10.5195/jmla.2021.1185 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the Medical Library Association, online erschienen am 05.10.2021. Verfasst von Paul Sebo. Beteiligte Einrichtung: paulsebo@hotmail.com, Primary Care Unit, Faculty of Medicine, University of Geneva, Geneva, Switzerland. Hauptthemen laut MEDLINE: Gender Identity, Information Storage and Retrieval. Die Arbeit wurde bisher 177-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Objective
To evaluate the performance of gender detection tools that allow the uploading of files (e.g., Excel or CSV files) containing first names, are usable by researchers without advanced computer skills, and are at least partially free of charge.
Methods
The study was conducted using four physician datasets (total number of physicians: 6,131; 50.3% female) from Switzerland, a multilingual country. Four gender detection tools met the inclusion criteria: three partially free (Gender API, NamSor, and genderize.io) and one completely free (Wiki-Gendersort). For each tool, we recorded the number of correct classifications (i.e., correct gender assigned to a name), misclassifications (i.e., wrong gender assigned to a name), and nonclassifications (i.e., no gender assigned). We computed three metrics: the proportion of misclassifications excluding nonclassifications (errorCodedWithoutNA), the proportion of nonclassifications (naCoded), and the proportion of misclassifications and nonclassifications (errorCoded).
Results
The proportion of misclassifications was low for all four gender detection tools (errorCodedWithoutNA between 1.5 and 2.2%). By contrast, the proportion of unrecognized names (naCoded) varied: 0% for NamSor, 0.3% for Gender API, 4.5% for Wiki-Gendersort, and 16.4% for genderize.io. Using errorCoded, which penalizes both types of error equally, we obtained the following results: Gender API 1.8%, NamSor 2.0%, Wiki-Gendersort 6.6%, and genderize.io 17.7%.
Conclusions
Gender API and NamSor were the most accurate tools. Genderize.io led to a high number of nonclassifications. Wiki-Gendersort may be a good compromise for researchers wishing to use a completely free tool. Other studies would be useful to evaluate the performance of these tools in other populations (e.g., Asian).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- paulsebo@hotmail.com, Primary Care Unit, Faculty of Medicine, University of Geneva, Geneva, Switzerland.
Zitieren
APA
Sebo, P. (2021). Performance of gender detection tools: a comparative study of name-to-gender inference services. Journal of the Medical Library Association, 109(3). https://doi.org/10.5195/jmla.2021.1185
Vancouver
Sebo P. Performance of gender detection tools: a comparative study of name-to-gender inference services. Journal of the Medical Library Association. 2021;109(3). doi:10.5195/jmla.2021.1185
BibTeX
@article{Sebo2021,
author = {Sebo, Paul},
title = {Performance of gender detection tools: a comparative study of name-to-gender inference services},
journal = {Journal of the Medical Library Association},
year = {2021},
volume = {109},
number = {3},
doi = {10.5195/jmla.2021.1185}
}
EUVIMED