Inferring gender from first names: Comparing the accuracy of Genderize, Gender API, and the gender R package on authors of diverse nationality
PLOS Digital Health · 29.10.2024 · 3 (10) · e0000456
DOI: 10.1371/journal.pdig.0000456 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Digital Health, online erschienen am 29.10.2024. Verfasst von Alexander D VanHelene und 5 weiteren Autor:innen. Beteiligte Einrichtung: Lifespan Cancer Institute, Rhode Island Hospital, Providence, Rhode Island, United States of America. Die Arbeit wurde bisher 62-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCI NIH HHS, National Cancer Institute.
Abstract (Originalsprache)
Meta-researchers commonly leverage tools that infer gender from first names, especially when studying gender disparities. However, tools vary in their accuracy, ease of use, and cost. The objective of this study was to compare the accuracy and cost of the commercial software Genderize and Gender API, and the open-source gender R package. Differences in binary gender prediction accuracy between the three services were evaluated. Gender prediction accuracy was tested on a multi-national dataset of 32,968 gender-labeled clinical trial authors. Additionally, two datasets from previous studies with 5779 and 6131 names, respectively, were re-evaluated with modern implementations of Genderize and Gender API. The gender inference accuracy of Genderize and Gender API were compared, both with and without supplying trialists’ country of origin in the API call. The accuracy of the gender R package was only evaluated without supplying countries of origin. The accuracy of Genderize, Gender API, and the gender R package were defined as the percentage of correct gender predictions. Accuracy differences between methods were evaluated using McNemar’s test. Genderize and Gender API demonstrated 96.6% and 96.1% accuracy, respectively, when countries of origin were not supplied in the API calls. Genderize and Gender API achieved the highest accuracy when predicting the gender of German authors with accuracies greater than 98%. Genderize and Gender API were least accurate with South Korean, Chinese, Singaporean, and Taiwanese authors, demonstrating below 82% accuracy. Genderize can provide similar accuracy to Gender API while being 4.85x less expensive. The gender R package achieved below 86% accuracy on the full dataset. In the replication studies, Genderize and gender API demonstrated better performance than in the original publications. Our results indicate that Genderize and Gender API achieve similar accuracy on a multinational dataset. The gender R package is uniformly less accurate than Genderize and Gender API.
Autor:innen und Institutionen
- Lifespan Cancer Institute, Rhode Island Hospital, Providence, Rhode Island, United States of America.
- Center for Clinical Cancer Informatics and Data Science, Legorreta Cancer Center, Brown University, Providence, Rhode Island.
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America.
- Warren Alpert Medical School, Brown University, Providence, Rhode Island, United States of America.
- Department of Internal Medicine, Vanderbilt University, Nashville, Tennessee, United States of America.
- Center for Computational Molecular Biology, Brown University, Providence, Rhode Island, United States of America.
ORCID-Profile: Alexander D VanHelene, C Beau Hilton, Sanjay Mishra, Jeremy L Warner
Förderung
- NCI NIH HHS
- National Cancer Institute
Zitieren
APA
VanHelene, A. D., Khatri, I., Hilton, C. B., Mishra, S., Gamsiz Uzun, E. D., & Warner, J. L. (2024). Inferring gender from first names: Comparing the accuracy of Genderize, Gender API, and the gender R package on authors of diverse nationality. PLOS Digital Health, 3(10), e0000456. https://doi.org/10.1371/journal.pdig.0000456
Vancouver
VanHelene AD, Khatri I, Hilton CB, Mishra S, Gamsiz Uzun ED, Warner JL. Inferring gender from first names: Comparing the accuracy of Genderize, Gender API, and the gender R package on authors of diverse nationality. PLOS Digital Health. 2024;3(10):e0000456. doi:10.1371/journal.pdig.0000456
BibTeX
@article{VanHelene2024,
author = {VanHelene, Alexander D and Khatri, Ishaani and Hilton, C Beau and Mishra, Sanjay and Gamsiz Uzun, Ece D and Warner, Jeremy L},
title = {Inferring gender from first names: Comparing the accuracy of Genderize, Gender API, and the gender R package on authors of diverse nationality},
journal = {PLOS Digital Health},
year = {2024},
volume = {3},
number = {10},
pages = {e0000456},
doi = {10.1371/journal.pdig.0000456}
}
EUVIMED