Beyond metrics to methods: a scoping review of transformers and large language models for detection of social drivers of health in clinical notes
Journal of the American Medical Informatics Association · 2026
DOI: 10.1093/jamia/ocaf201 ↗Kurz gefasst
Scoping Review in der Zeitschrift Journal of the American Medical Informatics Association, 2026. Verfasst von Ahmed Farrag und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL 32611, United States. Hauptthemen laut MEDLINE: Large Language Models, Social Determinants of Health, Electronic Health Records, Natural Language Processing. Die Arbeit nennt 130 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Objective
This scoping review aimed to (1) map current applications of transformers and large language models (LLMs) for extracting social drivers of health (SDOH) from clinical text, (2) benchmark model performance across SDOH domains, and (3) evaluate methodological rigor to identify research gaps and inform clinical deployment. Materials and Methods We searched PubMed, Web of Science, Embase, Scopus, and IEEE Xplore for studies applying transformers or LLMs to detect SDOH in clinical narratives. We developed a novel methodological framework integrating (1) hierarchical classification of SDOH domains and transformer/LLM architectures, (2) systematic synthesis of performance metrics, and (3) a 7-domain instrument assessing internal validity, external validity, and reporting transparency.
Results
Forty-two studies met inclusion criteria. Performance varied substantially across SDOH domains. Behavioral Factors achieved the highest median F1-score (0.87), while Health Care Access and Quality showed the lowest performance and greatest variability (median F1 = 0.59). Research concentrated in the United States (85.7%), relied predominantly on private institutional datasets (69%), and focused primarily on critical care populations (45.2%). Methodological assessment revealed critical gaps; only 29% of studies provided annotation guidelines, 24% assessed fairness across demographic groups, and 21% performed external validation.
Discussion
Smaller open-source transformer models show promise for democratizing SDOH detection by achieving competitive performance at lower costs while enabling secure local deployment in resource-limited settings. Advancing clinical readiness requires standardized reporting practices, diverse benchmark datasets across care settings, and systematic equity evaluation to prevent perpetuating health disparities.
Conclusion
Transformer and LLM performance for SDOH detection varied substantially across domains, with encoder-based models excelling at structured tasks and decoder-only models at linguistically complex tasks. Critical gaps in fairness assessment, external validation, and dataset diversity restrict generalizability and readiness for widespread clinical deployment.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL 32611, United States.
- Department of Health Outcomes & Biomedical Informatics, University of Florida, Gainesville, FL 32611, United States.
- Intelligent Clinical Care Center (IC3), University of Florida, Gainesville, FL 32610, United States.
- Division of General Internal Medicine, Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD 21287, United States.
- Center for Population Health Information Technology, Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205, United States.
- Center for Drug Evaluation and Safety (CoDES), University of Florida, Gainesville, FL 32611, United States.
ORCID-Profile: Ahmed Farrag, Ahmed Soliman, Elham Hatef, Amie Goodin, Masoud Rouhizadeh
Zitieren
APA
Farrag, A., Soliman, A., Hatef, E., Goodin, A., & Rouhizadeh, M. (2026). Beyond metrics to methods: a scoping review of transformers and large language models for detection of social drivers of health in clinical notes. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaf201
Vancouver
Farrag A, Soliman A, Hatef E, Goodin A, Rouhizadeh M. Beyond metrics to methods: a scoping review of transformers and large language models for detection of social drivers of health in clinical notes. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocaf201
BibTeX
@article{Farrag2026,
author = {Farrag, Ahmed and Soliman, Ahmed and Hatef, Elham and Goodin, Amie and Rouhizadeh, Masoud},
title = {Beyond metrics to methods: a scoping review of transformers and large language models for detection of social drivers of health in clinical notes},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
doi = {10.1093/jamia/ocaf201}
}
EUVIMED