EUVIMEDEUVIMED entdecken
Crossref

Evaluating large language models for specialist referral triage in primary care: a quantitative study using otolaryngology scenarios

Sholem Hack iD, Rebecca Attal, David Yogev, Shibli Alsleibi, Ben Gvili, David Shahal, Eran Glikson iD

Family Practice · 2025

DOI: 10.1093/fampra/cmaf087 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Family Practice, 2025. Verfasst von Sholem Hack und 6 weiteren Autor:innen. Beteiligte Einrichtung: City St. George's University London School of Medicine, Program Delivered by University of Nicosia at the Chaim Sheba Medical Center, Ramat Gan 52621…. Hauptthemen laut MEDLINE: Triage, Referral and Consultation, Primary Health Care, Otolaryngology. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 33 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Effective and timely referral to specialist care is a fundamental responsibility of primary care providers, including family physicians, general practitioners, and community health workers. However, challenges in triage can lead to delays, unnecessary referrals, and increased strain on healthcare systems. Advances in artificial intelligence (AI) now offer new opportunities to support referral decision-making at the primary care level.

Objective

This study evaluates the performance and reliability of two AI models as referral decision support tools, using simulated scenarios commonly encountered in primary care settings involving ear, nose, and throat conditions.

Methods

Sixteen clinical vignettes representing common or high-stakes primary care presentations requiring specialist input were presented to each model in both structured clinical and informal patient-language formats. Responses were independently assessed by five otolaryngologists and 10 lay reviewers using a standardized rubric focused on appropriateness, clarity, safety, and usefulness. Quantitative analysis included comparisons of model performance, reviewer agreement, and the impact of prompt structure on output quality.

Results

Both AI models generated safe and clinically appropriate referral recommendations when provided with structured clinical input. No statistically significant differences were observed between the two models across the evaluated domains. Performance declined for one model when prompts were presented in informal language, underscoring the importance of clear input structure. Reviewer agreement was high, confirming the reliability of findings.

Conclusion

AI decision support tools show potential to assist specialist referral triage in primary care. Clear, structured input is essential to maximize safety and reliability.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

TriageReferral and ConsultationPrimary Health CareOtolaryngologyArtificial IntelligenceHumansReproducibility of ResultsLanguageFemaleMaleLarge Language Models

Stichwörter der Autor:innen

artificial intelligenceclinical decision supportfamily medicinelarge language modelsprimary carereferral triage

Autor:innen und Institutionen

  • City St. George's University London School of Medicine, Program Delivered by University of Nicosia at the Chaim Sheba Medical Center, Ramat Gan 52621, Israel.
  • Department of Otolaryngology, Sheba Medical Center, Ramat Gan 52621, Israel.
  • Faculty of Medicine, Tel-Aviv University, Tel Aviv 69978, Israel.

ORCID-Profile: Sholem Hack, Eran Glikson

Zitieren

APA

Hack, S., Attal, R., Yogev, D., Alsleibi, S., Gvili, B., Shahal, D., & Glikson, E. (2025). Evaluating large language models for specialist referral triage in primary care: a quantitative study using otolaryngology scenarios. Family Practice. https://doi.org/10.1093/fampra/cmaf087

Vancouver

Hack S, Attal R, Yogev D, Alsleibi S, Gvili B, Shahal D, et al. Evaluating large language models for specialist referral triage in primary care: a quantitative study using otolaryngology scenarios. Family Practice. 2025. doi:10.1093/fampra/cmaf087

BibTeX
@article{Hack2025,
  author = {Hack, Sholem and Attal, Rebecca and Yogev, David and Alsleibi, Shibli and Gvili, Ben and Shahal, David and Glikson, Eran},
  title = {Evaluating large language models for specialist referral triage in primary care: a quantitative study using otolaryngology scenarios},
  journal = {Family Practice},
  year = {2025},
  doi = {10.1093/fampra/cmaf087}
}
Kostenlos starten →