EUVIMEDEUVIMED entdecken
Crossref

Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis

Akash S Halagur, Karthik Balakrishnan iD, Noel Ayoub iD

The Laryngoscope · 2025-01 · 135 (1) · 87-93

DOI: 10.1002/lary.31705 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Laryngoscope, online erschienen am 19.08.2024. Verfasst von Akash S Halagur und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, California, U.S.A. Hauptthemen laut MEDLINE: Internship and Residency, Otolaryngology. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 31 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To investigate potential demographic bias in artificial intelligence (AI)‐based simulations of otolaryngology, residency selection committee (RSC) members tasked with selecting one applicant among candidates with varied racial, gender, and sexual orientations.

Methods

This study employed random sampling of simulated RSC member decisions using a novel Application Programming Interface (API) to virtually connect to OpenAI's Generative Pre‐Trained Transformers (GPT‐4 and GPT‐4o). Simulated RSC members with diverse demographics were tasked with ranking to match 1 applicant among 10 with varied racial, gender, and sexual orientations. All applicants had identical qualifications; only demographics of the applicants and RSC members were varied for each simulation. Each RSC simulation ran 1000 times. Chi‐square tests analyzed differences across categorical variables. GPT‐4o simulations additionally requested a rationale for each decision.

Results

Simulated RSCs consistently showed racial, gender, and sexual orientation bias. Most applicant pairwise comparisons showed statistical significance ( p < 0.05). White and Black RSCs exhibited greatest preference for applicants sharing their own demographic characteristics, favoring White and Black female applicants, respectively, over others (all pairwise p < 0.001). Asian male applicants consistently received lowest selection rates. Male RSCs favored White male and female applicants, while female RSCs preferred LGBTQIA+, White and Black female applicants (all p < 0.05). High socioeconomic status (SES) RSCs favored White female and LGBTQIA+ applicants, while low SES RSCs favored Black female and LGBTQIA+ applicants over others (all p < 0.001). Results from the newest iteration of the LLM, ChatGPT‐4o, indicated evolved selection preferences favoring Black female and LGBTQIA+ applicants across all RSCs, with the rationale of prioritizing inclusivity given in >95% of such decisions.

Conclusion

Utilizing publicly available LLMs to aid in otolaryngology residency selection may introduce significant racial, gender, and sexual orientation bias. Potential for significant and evolving LLM bias should be appreciated and minimized to promote a diverse and representative field of future otolaryngologists in alignment with current workforce data. Level of Evidence NA Laryngoscope , 135:87–93, 2025

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Internship and ResidencyOtolaryngologyHumansMaleFemaleSchool Admission CriteriaLanguagePersonnel SelectionAdult

Stichwörter der Autor:innen

Artificial Intelligence in Healthcarebiasdiversity, equity, and inclusion (DEI)large language modelsotolaryngology residency admissions

Autor:innen und Institutionen

  • Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Stanford, California, U.S.A.
  • Department of Otolaryngology-Head & Neck Surgery, Geisel School of Medicine at Dartmouth, Hanover, New Hampshire, U.S.A.
  • Division of Pediatric Otolaryngology, Department of Otolaryngology-Head & Neck Surgery, Stanford University School of Medicine, Palo Alto, California, U.S.A.
  • Division of Rhinology and Skull Base Surgery, Department of Otolaryngology-Head & Neck Surgery, Mass Eye and Ear, Boston, Massachusetts, U.S.A.

ORCID-Profile: Karthik Balakrishnan, Noel Ayoub

Zitieren

APA

Halagur, A. S., Balakrishnan, K., & Ayoub, N. (2025). Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis. The Laryngoscope, 135(1), 87-93. https://doi.org/10.1002/lary.31705

Vancouver

Halagur AS, Balakrishnan K, Ayoub N. Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis. The Laryngoscope. 2025;135(1):87-93. doi:10.1002/lary.31705

BibTeX
@article{Halagur2025,
  author = {Halagur, Akash S and Balakrishnan, Karthik and Ayoub, Noel},
  title = {Large Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis},
  journal = {The Laryngoscope},
  year = {2025},
  volume = {135},
  number = {1},
  pages = {87--93},
  doi = {10.1002/lary.31705}
}
Kostenlos starten →