EUVIMEDEUVIMED entdecken
Open Access

The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis

Amadeo Jesus Wals Zurita iD, Hector Miras Del Rio iD, Nerea Ugarte Ruiz de Aguirre iD, Cristina Nebrera Navarro iD, Maria Rubio Jimenez iD, David Muñoz Carmona iD, Carlos Miguez Sanchez iD

JMIR Medical Informatics · 02.01.2025 · 13 · e58457

DOI: 10.2196/58457 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 02.01.2025. Verfasst von Amadeo Jesus Wals Zurita und 6 weiteren Autor:innen. Beteiligte Einrichtung: Servicio Oncologia Radioterápica, Hospital Universitario Virgen Macarena, Andalusian Health Service, Seville, Spain. Hauptthemen laut MEDLINE: Data Mining, Electronic Health Records. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 24 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

In this study, we evaluate the accuracy, efficiency, and cost-effectiveness of large language models in extracting and structuring information from free-text clinical reports, particularly in identifying and classifying patient comorbidities within oncology electronic health records. We specifically compare the performance of gpt-3.5-turbo-1106 and gpt-4-1106-preview models against that of specialized human evaluators.

Objective

We specifically compare the performance of gpt-3.5-turbo-1106 and gpt-4-1106-preview models against that of specialized human evaluators.

Methods

We implemented a script using the OpenAI application programming interface to extract structured information in JavaScript object notation format from comorbidities reported in 250 personal history reports. These reports were manually reviewed in batches of 50 by 5 specialists in radiation oncology. We compared the results using metrics such as sensitivity, specificity, precision, accuracy, F-value, κ index, and the McNemar test, in addition to examining the common causes of errors in both humans and generative pretrained transformer (GPT) models.

Results

The GPT-3.5 model exhibited slightly lower performance compared to physicians across all metrics, though the differences were not statistically significant (McNemar test, P=.79). GPT-4 demonstrated clear superiority in several key metrics (McNemar test, P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Data MiningElectronic Health RecordsHumansNatural Language ProcessingComorbidity

Stichwörter der Autor:innen

ChatGPTEHRLLMsdata miningelectronic health recordlarge language modelsoncologyradiotherapy

Autor:innen und Institutionen

  • Servicio Oncologia Radioterápica, Hospital Universitario Virgen Macarena, Andalusian Health Service, Seville, Spain.

ORCID-Profile: Amadeo Jesus Wals Zurita, Hector Miras Del Rio, Nerea Ugarte Ruiz de Aguirre, Cristina Nebrera Navarro, Maria Rubio Jimenez, David Muñoz Carmona, Carlos Miguez Sanchez

Zitieren

APA

Wals Zurita, A. J., Miras Del Rio, H., Ugarte Ruiz de Aguirre, N., Nebrera Navarro, C., Rubio Jimenez, M., Muñoz Carmona, D., & Miguez Sanchez, C. (2025). The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis. JMIR Medical Informatics, 13, e58457. https://doi.org/10.2196/58457

Vancouver

Wals Zurita AJ, Miras Del Rio H, Ugarte Ruiz de Aguirre N, Nebrera Navarro C, Rubio Jimenez M, Muñoz Carmona D, et al. The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis. JMIR Medical Informatics. 2025;13:e58457. doi:10.2196/58457

BibTeX
@article{WalsZurita2025,
  author = {Wals Zurita, Amadeo Jesus and Miras Del Rio, Hector and Ugarte Ruiz de Aguirre, Nerea and Nebrera Navarro, Cristina and Rubio Jimenez, Maria and Muñoz Carmona, David and Miguez Sanchez, Carlos},
  title = {The Transformative Potential of Large Language Models in Mining Electronic Health Records Data: Content Analysis},
  journal = {JMIR Medical Informatics},
  year = {2025},
  volume = {13},
  pages = {e58457},
  doi = {10.2196/58457}
}
Kostenlos starten →