EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance of gender detection tools: a comparative study of name-to-gender inference services

Paul Sebo

Journal of the Medical Library Association · 05.10.2021 · 109 (3)

DOI: 10.5195/jmla.2021.1185 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the Medical Library Association, online erschienen am 05.10.2021. Verfasst von Paul Sebo. Beteiligte Einrichtung: paulsebo@hotmail.com, Primary Care Unit, Faculty of Medicine, University of Geneva, Geneva, Switzerland. Hauptthemen laut MEDLINE: Gender Identity, Information Storage and Retrieval. Die Arbeit wurde bisher 177-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To evaluate the performance of gender detection tools that allow the uploading of files (e.g., Excel or CSV files) containing first names, are usable by researchers without advanced computer skills, and are at least partially free of charge.

Methods

The study was conducted using four physician datasets (total number of physicians: 6,131; 50.3% female) from Switzerland, a multilingual country. Four gender detection tools met the inclusion criteria: three partially free (Gender API, NamSor, and genderize.io) and one completely free (Wiki-Gendersort). For each tool, we recorded the number of correct classifications (i.e., correct gender assigned to a name), misclassifications (i.e., wrong gender assigned to a name), and nonclassifications (i.e., no gender assigned). We computed three metrics: the proportion of misclassifications excluding nonclassifications (errorCodedWithoutNA), the proportion of nonclassifications (naCoded), and the proportion of misclassifications and nonclassifications (errorCoded).

Results

The proportion of misclassifications was low for all four gender detection tools (errorCodedWithoutNA between 1.5 and 2.2%). By contrast, the proportion of unrecognized names (naCoded) varied: 0% for NamSor, 0.3% for Gender API, 4.5% for Wiki-Gendersort, and 16.4% for genderize.io. Using errorCoded, which penalizes both types of error equally, we obtained the following results: Gender API 1.8%, NamSor 2.0%, Wiki-Gendersort 6.6%, and genderize.io 17.7%.

Conclusions

Gender API and NamSor were the most accurate tools. Genderize.io led to a high number of nonclassifications. Wiki-Gendersort may be a good compromise for researchers wishing to use a completely free tool. Other studies would be useful to evaluate the performance of these tools in other populations (e.g., Asian).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Gender IdentityInformation Storage and RetrievalDatabases, BibliographicFemaleHumansMale

Stichwörter der Autor:innen

accuracygender detectionmisclassificationnamename-to-genderperformance

Autor:innen und Institutionen

  • paulsebo@hotmail.com, Primary Care Unit, Faculty of Medicine, University of Geneva, Geneva, Switzerland.

Zitieren

APA

Sebo, P. (2021). Performance of gender detection tools: a comparative study of name-to-gender inference services. Journal of the Medical Library Association, 109(3). https://doi.org/10.5195/jmla.2021.1185

Vancouver

Sebo P. Performance of gender detection tools: a comparative study of name-to-gender inference services. Journal of the Medical Library Association. 2021;109(3). doi:10.5195/jmla.2021.1185

BibTeX
@article{Sebo2021,
  author = {Sebo, Paul},
  title = {Performance of gender detection tools: a comparative study of name-to-gender inference services},
  journal = {Journal of the Medical Library Association},
  year = {2021},
  volume = {109},
  number = {3},
  doi = {10.5195/jmla.2021.1185}
}
Kostenlos starten →