Structural insights into clinical large language models and their barriers to translational readiness
Journal of the American Medical Informatics Association · 01.03.2026 · 33 (3) · 732-742
DOI: 10.1093/jamia/ocaf230 ↗Kurz gefasst
Übersichtsarbeit in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 11.01.2026. Verfasst von Jiwon You und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Informatics and Statistics, Asan Medical Center, Brain Korea 21 Project, University of Ulsan College of Medicine, Seoul 05505, …. Hauptthemen laut MEDLINE: Large Language Models, Translational Research, Biomedical, Clinical Decision-Making. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: National Research Foundation of Korea, Ministry of Science and ICT of Korea Government, Asan Institute for Life Sciences, Asan Medical Center, Republic of Korea.
Abstract (Originalsprache)
Background
Despite rapid integration into clinical decision-making, clinical large language models (LLMs) face substantial translational barriers due to insufficient structural characterization and limited external validation.
Objective
We systematically map the clinical LLM research landscape to identify key structural patterns influencing their readiness for real-world clinical deployment.
Methods
We identified 73 clinical LLM studies published between January 2020 and March 2025 using a structured evidence-mapping approach. To ensure transparency and reproducibility in study selection, we followed key principles from the PRISMA 2020 framework. Each study was categorized by clinical task, base architecture, alignment strategy, data type, language, study design, validation methods, and evaluation metrics.
Results
Studies often addressed multiple early stage clinical tasks—question answering (56.2%), knowledge structuring (31.5%), and disease prediction (43.8%)—primarily using text data (52.1%) and English-language resources (80.8%). GPT models favored retrieval-augmented generation (43.8%), and LLaMA models consistently adopted multistage pretraining and fine-tuning strategies. Only 6.9% of studies included external validation, and prospective designs were observed in just 4.1% of cases, reflecting significant gaps in translational reliability. Evaluations were predominantly quantitative only (79.5%), though qualitative and mixed-method approaches are increasingly recognized for assessing clinical usability and trustworthiness.
Conclusion
Clinical LLM research remains exploratory, marked by limited generalizability across languages, data types, and clinical environments. To bridge this gap, future studies must prioritize multilingual and multimodal training, prospective study designs with rigorous external validation, and hybrid evaluation frameworks combining quantitative performance with qualitative clinical usability metrics.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medical Informatics and Statistics, Asan Medical Center, Brain Korea 21 Project, University of Ulsan College of Medicine, Seoul 05505, Republic of Korea.
ORCID-Profile: Hangsik Shin
Förderung
- National Research Foundation of Korea
- Ministry of Science and ICT of Korea Government
- Asan Institute for Life Sciences, Asan Medical Center, Republic of Korea
Zitieren
APA
You, J., & Shin, H. (2026). Structural insights into clinical large language models and their barriers to translational readiness. Journal of the American Medical Informatics Association, 33(3), 732-742. https://doi.org/10.1093/jamia/ocaf230
Vancouver
You J, Shin H. Structural insights into clinical large language models and their barriers to translational readiness. Journal of the American Medical Informatics Association. 2026;33(3):732-742. doi:10.1093/jamia/ocaf230
BibTeX
@article{You2026,
author = {You, Jiwon and Shin, Hangsik},
title = {Structural insights into clinical large language models and their barriers to translational readiness},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
volume = {33},
number = {3},
pages = {732--742},
doi = {10.1093/jamia/ocaf230}
}
EUVIMED