EUVIMEDEUVIMED entdecken
Systematische ÜbersichtStudienprotokollOpen Access

Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review

Shweta Rao iD, Carol Boutrous iD, Andrey Kormilitzin, Xin You Tai iD, Lewis Hotchkiss iD, Cen Cong iD, Edward Meinert iD, Huizhi Liang iD, Hang Dong iD, Judith R Harrison iD

JMIR Research Protocols · 10.09.2026 · 15 · e91348-e91348

DOI: 10.2196/91348 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift JMIR Research Protocols, online erschienen am 10.09.2026. Verfasst von Shweta Rao und 9 weiteren Autor:innen. Beteiligte Einrichtung: School of Medicine, Newcastle University, Newcastle, AK, United Kingdom. Hauptthemen laut MEDLINE: Large Language Models. Die Arbeit nennt 47 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly applied in health care for clinical decision-making, education, and patient communication. However, bias in LLM outputs may exacerbate health care disparities and compromise trust. Despite rapid adoption, there is limited synthesis of how bias is identified, measured, and mitigated in medical applications of LLMs.

Objective

This systematic review aims to evaluate how bias is detected, measured, and mitigated in health care applications of LLMs and to identify methodological trends and gaps in the current literature.

Methods

We will conduct a systematic review of studies evaluating bias, fairness, or subgroup performance in medical applications of LLMs. Searches will be conducted across Embase, MEDLINE, PsycINFO, PubMed, ACL Anthology, ACM Digital Library, arXiv, medRxiv, and bioRxiv for studies published from 2017 onward. The review uses a staged design. In phase 1, records identified in the original June 2025 search were screened and assessed manually using predefined eligibility criteria. This fully manually reviewed dataset will serve as a reference standard for validating the review workflow. In phase 2, an updated June 2026 search will be screened using a prespecified LLM-assisted workflow. Two independent LLMs will apply the same eligibility criteria used in manual screening, with studies marked as potentially relevant or uncertain by either model progressing to further assessment. Final inclusion decisions will be made by at least 2 human reviewers. The performance of the LLM-assisted workflow will be evaluated against the manually reviewed reference dataset using sensitivity, specificity, precision, negative predictive value, inclusion agreement, and Cohen κ. Data will be extracted using a standardized framework covering study characteristics, model details, health care use case, bias type, bias assessment methods, mitigation strategies, transparency, generalizability, and ethical or regulatory framing. Findings will be synthesized narratively.

Results

The phase 1 search, conducted on June 11, 2025, yielded 15,976 records after deduplication and has undergone complete manual screening. The phase 2 search, conducted on June 8, 2026, increased the total number of records to 41,143. Screening of the updated dataset using the LLM-assisted workflow is ongoing. The completed review will report included study characteristics, approaches to bias detection and mitigation, and validation metrics for the LLM-assisted workflow. The review is expected to be submitted for publication in late November 2026, with publication anticipated thereafter subject to the journal’s peer-review and editorial process.

Conclusions

This review aims to provide a comprehensive synthesis of current approaches to bias detection and mitigation in medical LLMs, highlighting methodological strengths, limitations, and areas for future research. The findings aim to inform the development of more equitable and transparent AI systems in health care.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsHumansSystematic Reviews as TopicBiasResearch Design

Stichwörter der Autor:innen

LLMbiashealth carelarge language modelmedicine

Autor:innen und Institutionen

  • School of Medicine, Newcastle University, Newcastle, AK, United Kingdom.
  • Department of Psychiatry, University of Oxford, Headington, Oxford, England, United Kingdom.
  • Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, England, United Kingdom.
  • Dementias Platform UK (DPUK), Swansea University, Swansea, Wales, United Kingdom.
  • Translational and Clinical Research Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, England, United Kingdom.
  • Translational and Clinical Research Institute 3rd Floor, Biomedical Research Building Campus for Ageing and Vitality, Westgate Road, Newcastle upon Tyne NE4 5PL, Newcastle, England, United Kingdom, 44 7756339941.

ORCID-Profile: Shweta Rao, Carol Boutrous, Xin You Tai, Lewis Hotchkiss, Cen Cong, Edward Meinert, Huizhi Liang, Hang Dong, Judith R Harrison

Zitieren

APA

Rao, S., Boutrous, C., Kormilitzin, A., Tai, X. Y., Hotchkiss, L., Cong, C., Meinert, E., Liang, H., Dong, H., & Harrison, J. R. (2026). Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review. JMIR Research Protocols, 15, e91348-e91348. https://doi.org/10.2196/91348

Vancouver

Rao S, Boutrous C, Kormilitzin A, Tai XY, Hotchkiss L, Cong C, et al. Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review. JMIR Research Protocols. 2026;15:e91348-e91348. doi:10.2196/91348

BibTeX
@article{Rao2026,
  author = {Rao, Shweta and Boutrous, Carol and Kormilitzin, Andrey and Tai, Xin You and Hotchkiss, Lewis and Cong, Cen and Meinert, Edward and Liang, Huizhi and Dong, Hang and Harrison, Judith R},
  title = {Exploring Bias in Medical Applications of Large Language Models: Protocol for a Systematic Review},
  journal = {JMIR Research Protocols},
  year = {2026},
  volume = {15},
  pages = {e91348--e91348},
  doi = {10.2196/91348}
}
Kostenlos starten →