Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis
The Laryngoscope · 2025-01 · 135 (1) · 87-93
DOI: 10.1002/lary.31705 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Laryngoscope, online erschienen am 19.08.2024. Verfasst von Akash S Halagur und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, California, U.S.A. Hauptthemen laut MEDLINE: Internship and Residency, Otolaryngology. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 31 Literaturangaben.
Abstract (Originalsprache)
Objectives
To investigate potential demographic bias in artificial intelligence (AI)‐based simulations of otolaryngology, residency selection committee (RSC) members tasked with selecting one applicant among candidates with varied racial, gender, and sexual orientations.
Methods
This study employed random sampling of simulated RSC member decisions using a novel Application Programming Interface (API) to virtually connect to OpenAI's Generative Pre‐Trained Transformers (GPT‐4 and GPT‐4o). Simulated RSC members with diverse demographics were tasked with ranking to match 1 applicant among 10 with varied racial, gender, and sexual orientations. All applicants had identical qualifications; only demographics of the applicants and RSC members were varied for each simulation. Each RSC simulation ran 1000 times. Chi‐square tests analyzed differences across categorical variables. GPT‐4o simulations additionally requested a rationale for each decision.
Results
Simulated RSCs consistently showed racial, gender, and sexual orientation bias. Most applicant pairwise comparisons showed statistical significance ( p < 0.05). White and Black RSCs exhibited greatest preference for applicants sharing their own demographic characteristics, favoring White and Black female applicants, respectively, over others (all pairwise p < 0.001). Asian male applicants consistently received lowest selection rates. Male RSCs favored White male and female applicants, while female RSCs preferred LGBTQIA+, White and Black female applicants (all p < 0.05). High socioeconomic status (SES) RSCs favored White female and LGBTQIA+ applicants, while low SES RSCs favored Black female and LGBTQIA+ applicants over others (all p < 0.001). Results from the newest iteration of the LLM, ChatGPT‐4o, indicated evolved selection preferences favoring Black female and LGBTQIA+ applicants across all RSCs, with the rationale of prioritizing inclusivity given in >95% of such decisions.
Conclusion
Utilizing publicly available LLMs to aid in otolaryngology residency selection may introduce significant racial, gender, and sexual orientation bias. Potential for significant and evolving LLM bias should be appreciated and minimized to promote a diverse and representative field of future otolaryngologists in alignment with current workforce data. Level of Evidence NA Laryngoscope , 135:87–93, 2025
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, California, U.S.A.
- Department of Otolaryngology-Head & Neck Surgery, Geisel School of Medicine at Dartmouth, Hanover, New Hampshire, U.S.A.
- Division of Pediatric Otolaryngology, Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Palo Alto, California, U.S.A.
- Division of Rhinology and Skull Base Surgery, Department of Otolaryngology-Head & Neck Surgery, Mass Eye and Ear, Boston, Massachusetts, U.S.A.
ORCID-Profile: Karthik Balakrishnan, Noel Ayoub
Zitieren
APA
Halagur, A. S., Balakrishnan, K., & Ayoub, N. (2025). Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis. The Laryngoscope, 135(1), 87-93. https://doi.org/10.1002/lary.31705
Vancouver
Halagur AS, Balakrishnan K, Ayoub N. Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis. The Laryngoscope. 2025;135(1):87-93. doi:10.1002/lary.31705
BibTeX
@article{Halagur2025,
author = {Halagur, Akash S and Balakrishnan, Karthik and Ayoub, Noel},
title = {Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis},
journal = {The Laryngoscope},
year = {2025},
volume = {135},
number = {1},
pages = {87--93},
doi = {10.1002/lary.31705}
}
EUVIMED