Automated structured data extraction from intraoperative echocardiography reports using large language models
British Journal of Anaesthesia · 2025-05 · 134 (5) · 1308-1317
DOI: 10.1016/j.bja.2025.01.028 ↗Kurz gefasst
Fachartikel in der Zeitschrift British Journal of Anaesthesia, 2025-05. Verfasst von Emily J MacKay und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Anaesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Penn Center for Peri…. Hauptthemen laut MEDLINE: Echocardiography, Transesophageal, Echocardiography, Natural Language Processing, Information Storage and Retrieval. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NHLBI NIH HHS, National Heart, Lung, and Blood Institute, National Center for Advancing Translational Sciences.
Abstract (Originalsprache)
BACKGROUND: Consensus-based large language model (LLM) ensembles might provide an automated solution for extracting structured data from unstructured text in echocardiography reports. METHODS: This cross-sectional study utilised 600 intraoperative transoesophageal reports (100 for prompt engineering; 500 for testing) randomly sampled from 7106 adult patients undergoing cardiac surgery at two hospitals within the University of Pennsylvania Healthcare System. Three echocardiographic parameters (left ventricular ejection fraction, right ventricular systolic function, and tricuspid regurgitation) were extracted from both the presurgical and postsurgical sections of the reports. LLM ensembles were generated using five open-source LLMs and four voting strategies: (1) unanimous (five out of five in agreement); (2) supermajority (four or more of five in agreement); (3) majority (three or more of five in agreement); and (4) plurality (two or more of five in agreement). Returned LLM ensemble responses were compared with the reference standard dataset to calculate raw accuracy, consensus accuracy, error rate, and yield. RESULTS: Of the four LLM ensembles, the unanimous LLM ensemble achieved the highest consensus accuracies (99.4% presurgical; 97.9% postsurgical) and the lowest error rates (0.6% presurgical; 2.1% postsurgical) but had the lowest data extraction yields (81.7% presurgical; 80.5% postsurgical) and the lowest raw accuracies (81.2% presurgical; 78.9% postsurgical). In contrast, the plurality LLM ensemble achieved the highest raw accuracies (96.1% presurgical; 93.7% postsurgical) and the highest data extraction yields (99.4% presurgical; 98.9% postsurgical) but had the lowest consensus accuracies (96.7% presurgical; 94.7% postsurgical) and highest error rates (3.3% presurgical; 5.3% postsurgical). CONCLUSIONS: A consensus-based LLM ensemble successfully generated structured data from unstructured text contained in intraoperative transoesophageal reports.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Anaesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Penn Center for Perioperative Outcomes Research and Transformation (CPORT), Perelman Schoo…
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; School of Arts and Sciences, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA.
- School of Medicine and Health Sciences, Des Moines University, Des Moines, IA, USA.
- Department of Anaesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
- Harvard Medical School, Harvard University, Boston, MA, USA.
Förderung
- NHLBI NIH HHS
- National Heart, Lung, and Blood Institute
- National Center for Advancing Translational Sciences
- University of Pennsylvania
- Foundation for the National Institutes of Health
- National Institutes of Health
Zitieren
APA
MacKay, E. J., Goldfinger, S., Chan, T. J., Grasfield, R. H., Eswar, V. J., Li, K., Cao, Q., & Pouch, A. M. (2025). Automated structured data extraction from intraoperative echocardiography reports using large language models. British Journal of Anaesthesia, 134(5), 1308-1317. https://doi.org/10.1016/j.bja.2025.01.028
Vancouver
MacKay EJ, Goldfinger S, Chan TJ, Grasfield RH, Eswar VJ, Li K, et al. Automated structured data extraction from intraoperative echocardiography reports using large language models. British Journal of Anaesthesia. 2025;134(5):1308-1317. doi:10.1016/j.bja.2025.01.028
BibTeX
@article{MacKay2025,
author = {MacKay, Emily J and Goldfinger, Shir and Chan, Trevor J and Grasfield, Rachel H and Eswar, Vikram J and Li, Kelly and Cao, Quy and Pouch, Alison M},
title = {Automated structured data extraction from intraoperative echocardiography reports using large language models},
journal = {British Journal of Anaesthesia},
year = {2025},
volume = {134},
number = {5},
pages = {1308--1317},
doi = {10.1016/j.bja.2025.01.028}
}
EUVIMED