EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Automated structured data extraction from intraoperative echocardiography reports using large language models

Emily J MacKay, Shir Goldfinger, Trevor J Chan, Rachel H Grasfield, Vikram J Eswar, Kelly Li, Quy Cao, Alison M Pouch

British Journal of Anaesthesia · 2025-05 · 134 (5) · 1308-1317

DOI: 10.1016/j.bja.2025.01.028 ↗

Kurz gefasst

Fachartikel in der Zeitschrift British Journal of Anaesthesia, 2025-05. Verfasst von Emily J MacKay und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Anaesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Penn Center for Peri…. Hauptthemen laut MEDLINE: Echocardiography, Transesophageal, Echocardiography, Natural Language Processing, Information Storage and Retrieval. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NHLBI NIH HHS, National Heart, Lung, and Blood Institute, National Center for Advancing Translational Sciences.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Consensus-based large language model (LLM) ensembles might provide an automated solution for extracting structured data from unstructured text in echocardiography reports. METHODS: This cross-sectional study utilised 600 intraoperative transoesophageal reports (100 for prompt engineering; 500 for testing) randomly sampled from 7106 adult patients undergoing cardiac surgery at two hospitals within the University of Pennsylvania Healthcare System. Three echocardiographic parameters (left ventricular ejection fraction, right ventricular systolic function, and tricuspid regurgitation) were extracted from both the presurgical and postsurgical sections of the reports. LLM ensembles were generated using five open-source LLMs and four voting strategies: (1) unanimous (five out of five in agreement); (2) supermajority (four or more of five in agreement); (3) majority (three or more of five in agreement); and (4) plurality (two or more of five in agreement). Returned LLM ensemble responses were compared with the reference standard dataset to calculate raw accuracy, consensus accuracy, error rate, and yield. RESULTS: Of the four LLM ensembles, the unanimous LLM ensemble achieved the highest consensus accuracies (99.4% presurgical; 97.9% postsurgical) and the lowest error rates (0.6% presurgical; 2.1% postsurgical) but had the lowest data extraction yields (81.7% presurgical; 80.5% postsurgical) and the lowest raw accuracies (81.2% presurgical; 78.9% postsurgical). In contrast, the plurality LLM ensemble achieved the highest raw accuracies (96.1% presurgical; 93.7% postsurgical) and the highest data extraction yields (99.4% presurgical; 98.9% postsurgical) but had the lowest consensus accuracies (96.7% presurgical; 94.7% postsurgical) and highest error rates (3.3% presurgical; 5.3% postsurgical). CONCLUSIONS: A consensus-based LLM ensemble successfully generated structured data from unstructured text contained in intraoperative transoesophageal reports.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Echocardiography, TransesophagealEchocardiographyNatural Language ProcessingInformation Storage and RetrievalMonitoring, IntraoperativeHumansCross-Sectional StudiesMaleFemaleMiddle AgedAgedCardiac Surgical ProceduresAdultLarge Language Models

Stichwörter der Autor:innen

artificial intelligence (AI)cardiac surgeryechocardiographylarge language modelsperioperative medicine

Autor:innen und Institutionen

  • Department of Anaesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Penn Center for Perioperative Outcomes Research and Transformation (CPORT), Perelman Schoo…
  • Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; School of Arts and Sciences, University of Pennsylvania, Philadelphia, PA, USA.
  • Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA.
  • School of Medicine and Health Sciences, Des Moines University, Des Moines, IA, USA.
  • Department of Anaesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
  • Harvard Medical School, Harvard University, Boston, MA, USA.

Förderung

  • NHLBI NIH HHS
  • National Heart, Lung, and Blood Institute
  • National Center for Advancing Translational Sciences
  • University of Pennsylvania
  • Foundation for the National Institutes of Health
  • National Institutes of Health

Zitieren

APA

MacKay, E. J., Goldfinger, S., Chan, T. J., Grasfield, R. H., Eswar, V. J., Li, K., Cao, Q., & Pouch, A. M. (2025). Automated structured data extraction from intraoperative echocardiography reports using large language models. British Journal of Anaesthesia, 134(5), 1308-1317. https://doi.org/10.1016/j.bja.2025.01.028

Vancouver

MacKay EJ, Goldfinger S, Chan TJ, Grasfield RH, Eswar VJ, Li K, et al. Automated structured data extraction from intraoperative echocardiography reports using large language models. British Journal of Anaesthesia. 2025;134(5):1308-1317. doi:10.1016/j.bja.2025.01.028

BibTeX
@article{MacKay2025,
  author = {MacKay, Emily J and Goldfinger, Shir and Chan, Trevor J and Grasfield, Rachel H and Eswar, Vikram J and Li, Kelly and Cao, Quy and Pouch, Alison M},
  title = {Automated structured data extraction from intraoperative echocardiography reports using large language models},
  journal = {British Journal of Anaesthesia},
  year = {2025},
  volume = {134},
  number = {5},
  pages = {1308--1317},
  doi = {10.1016/j.bja.2025.01.028}
}
Kostenlos starten →