EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Generative artificial intelligence for automated data extraction from unstructured medical text

Nam Dao, Luisa Quesada, Syed Moin Hassan iD, Monica Iturrioz Campo, Shelsey Johnson, Suchandra Ghose, Raúl San José Estépar, Aaron Waxman, George Washko, Farbod N Rahaghi

JAMIA Open · 02.09.2025 · 8 (5)

DOI: 10.1093/jamiaopen/ooaf097 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 04.09.2025. Verfasst von Nam Dao und 9 weiteren Autor:innen. Beteiligte Einrichtung: Division of Pulmonary and Critical Care, Brigham and Women's Hospital, Boston, MA, United States. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Institute of Health T32 program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Unstructured data, such as procedure notes, contain valuable medical information that is frequently underutilized due to the labor-intensive nature of data extraction. This study aims to develop a generative artificial intelligence (GenAI) pipeline using an open-source Large Language Model (LLM) with built-in guardrails and a retry mechanism to extract data from unstructured right heart catheterization (RHC) notes while minimizing errors, including hallucinations. Materials and Methods A total of 220 RHC notes were randomly selected for pipeline development and 200 for validation from the Pulmonary Vascular Disease Registry. The pipeline comprised three main components: the Engineered Preload Framework (EPF), which integrated schemas and instructions; the LLM module, enhanced by reasoning capabilities; and the validation and retry mechanism, which ensured data accuracy through iterative self-correction. A clinical expert manually extracted data from the validation cohort to establish the ground truth. Pipeline performance was evaluated using precision, recall, and F1 score. Additionally, the dataset was stratified into quartiles to assess the pipeline’s ability to handle varying levels of data availability.

Results

The pipeline achieved 99.0% precision, 85.0% recall, and a 91.5% F1 score, with an overall accuracy of 90% when evaluated at the note level. The most common error was missed values (5.2%), while hallucinations were the least frequent (<0.01%). Discussion and Conclusion This study demonstrates the feasibility of a robust GenAI pipeline for automating structured data extraction from unstructured RHC procedure notes. The approach highlights the potential of LLMs in medical data mining, improving research efficiency and clinical applications.

Schlagwörter

Stichwörter der Autor:innen

cardiac catheterizationdata mininggenerative artificial intelligencelarge language modelspulmonary hypertension

Autor:innen und Institutionen

  • Division of Pulmonary and Critical Care, Brigham and Women's Hospital, Boston, MA, United States.
  • Division of Sleep Medicine, Brigham and Women's Hospital, Boston, MA, United States.
  • Department of Radiology, Brigham and Women's Hospital, Boston, MA, United States.

ORCID-Profile: Syed Moin Hassan

Förderung

  • National Institute of Health T32 program

Zitieren

APA

Dao, N., Quesada, L., Hassan, S. M., Campo, M. I., Johnson, S., Ghose, S., San José Estépar, R., Waxman, A., Washko, G., & Rahaghi, F. N. (2025). Generative artificial intelligence for automated data extraction from unstructured medical text. JAMIA Open, 8(5). https://doi.org/10.1093/jamiaopen/ooaf097

Vancouver

Dao N, Quesada L, Hassan SM, Campo MI, Johnson S, Ghose S, et al. Generative artificial intelligence for automated data extraction from unstructured medical text. JAMIA Open. 2025;8(5). doi:10.1093/jamiaopen/ooaf097

BibTeX
@article{Dao2025,
  author = {Dao, Nam and Quesada, Luisa and Hassan, Syed Moin and Campo, Monica Iturrioz and Johnson, Shelsey and Ghose, Suchandra and San José Estépar, Raúl and Waxman, Aaron and Washko, George and Rahaghi, Farbod N},
  title = {Generative artificial intelligence for automated data extraction from unstructured medical text},
  journal = {JAMIA Open},
  year = {2025},
  volume = {8},
  number = {5},
  doi = {10.1093/jamiaopen/ooaf097}
}
Kostenlos starten →