The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis
JMIR Medical Informatics · 02.01.2025 · 13 · e58457
DOI: 10.2196/58457 ↗Kurz gefasst
Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 02.01.2025. Verfasst von Amadeo Jesus Wals Zurita und 6 weiteren Autor:innen. Beteiligte Einrichtung: Servicio Oncologia Radioterápica, Hospital Universitario Virgen Macarena, Andalusian Health Service, Seville, Spain. Hauptthemen laut MEDLINE: Data Mining, Electronic Health Records. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
In this study, we evaluate the accuracy, efficiency, and cost-effectiveness of large language models in extracting and structuring information from free-text clinical reports, particularly in identifying and classifying patient comorbidities within oncology electronic health records. We specifically compare the performance of gpt-3.5-turbo-1106 and gpt-4-1106-preview models against that of specialized human evaluators.
Objective
We specifically compare the performance of gpt-3.5-turbo-1106 and gpt-4-1106-preview models against that of specialized human evaluators.
Methods
We implemented a script using the OpenAI application programming interface to extract structured information in JavaScript object notation format from comorbidities reported in 250 personal history reports. These reports were manually reviewed in batches of 50 by 5 specialists in radiation oncology. We compared the results using metrics such as sensitivity, specificity, precision, accuracy, F-value, κ index, and the McNemar test, in addition to examining the common causes of errors in both humans and generative pretrained transformer (GPT) models.
Results
The GPT-3.5 model exhibited slightly lower performance compared to physicians across all metrics, though the differences were not statistically significant (McNemar test, P=.79). GPT-4 demonstrated clear superiority in several key metrics (McNemar test, P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Servicio Oncologia Radioterápica, Hospital Universitario Virgen Macarena, Andalusian Health Service, Seville, Spain.
ORCID-Profile: Amadeo Jesus Wals Zurita, Hector Miras Del Rio, Nerea Ugarte Ruiz de Aguirre, Cristina Nebrera Navarro, Maria Rubio Jimenez, David Muñoz Carmona, Carlos Miguez Sanchez
Zitieren
APA
Wals Zurita, A. J., Miras Del Rio, H., Ugarte Ruiz de Aguirre, N., Nebrera Navarro, C., Rubio Jimenez, M., Muñoz Carmona, D., & Miguez Sanchez, C. (2025). The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis. JMIR Medical Informatics, 13, e58457. https://doi.org/10.2196/58457
Vancouver
Wals Zurita AJ, Miras Del Rio H, Ugarte Ruiz de Aguirre N, Nebrera Navarro C, Rubio Jimenez M, Muñoz Carmona D, et al. The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis. JMIR Medical Informatics. 2025;13:e58457. doi:10.2196/58457
BibTeX
@article{WalsZurita2025,
author = {Wals Zurita, Amadeo Jesus and Miras Del Rio, Hector and Ugarte Ruiz de Aguirre, Nerea and Nebrera Navarro, Cristina and Rubio Jimenez, Maria and Muñoz Carmona, David and Miguez Sanchez, Carlos},
title = {The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis},
journal = {JMIR Medical Informatics},
year = {2025},
volume = {13},
pages = {e58457},
doi = {10.2196/58457}
}
EUVIMED