Generative artificial intelligence for automated data extraction from unstructured medical text
JAMIA Open · 02.09.2025 · 8 (5)
DOI: 10.1093/jamiaopen/ooaf097 ↗Kurz gefasst
Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 04.09.2025. Verfasst von Nam Dao und 9 weiteren Autor:innen. Beteiligte Einrichtung: Division of Pulmonary and Critical Care, Brigham and Women's Hospital, Boston, MA, United States. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: National Institute of Health T32 program.
Abstract (Originalsprache)
Objectives
Unstructured data, such as procedure notes, contain valuable medical information that is frequently underutilized due to the labor-intensive nature of data extraction. This study aims to develop a generative artificial intelligence (GenAI) pipeline using an open-source Large Language Model (LLM) with built-in guardrails and a retry mechanism to extract data from unstructured right heart catheterization (RHC) notes while minimizing errors, including hallucinations. Materials and Methods A total of 220 RHC notes were randomly selected for pipeline development and 200 for validation from the Pulmonary Vascular Disease Registry. The pipeline comprised three main components: the Engineered Preload Framework (EPF), which integrated schemas and instructions; the LLM module, enhanced by reasoning capabilities; and the validation and retry mechanism, which ensured data accuracy through iterative self-correction. A clinical expert manually extracted data from the validation cohort to establish the ground truth. Pipeline performance was evaluated using precision, recall, and F1 score. Additionally, the dataset was stratified into quartiles to assess the pipeline’s ability to handle varying levels of data availability.
Results
The pipeline achieved 99.0% precision, 85.0% recall, and a 91.5% F1 score, with an overall accuracy of 90% when evaluated at the note level. The most common error was missed values (5.2%), while hallucinations were the least frequent (<0.01%). Discussion and Conclusion This study demonstrates the feasibility of a robust GenAI pipeline for automating structured data extraction from unstructured RHC procedure notes. The approach highlights the potential of LLMs in medical data mining, improving research efficiency and clinical applications.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Pulmonary and Critical Care, Brigham and Women's Hospital, Boston, MA, United States.
- Division of Sleep Medicine, Brigham and Women's Hospital, Boston, MA, United States.
- Department of Radiology, Brigham and Women's Hospital, Boston, MA, United States.
ORCID-Profile: Syed Moin Hassan
Förderung
- National Institute of Health T32 program
Zitieren
APA
Dao, N., Quesada, L., Hassan, S. M., Campo, M. I., Johnson, S., Ghose, S., San José Estépar, R., Waxman, A., Washko, G., & Rahaghi, F. N. (2025). Generative artificial intelligence for automated data extraction from unstructured medical text. JAMIA Open, 8(5). https://doi.org/10.1093/jamiaopen/ooaf097
Vancouver
Dao N, Quesada L, Hassan SM, Campo MI, Johnson S, Ghose S, et al. Generative artificial intelligence for automated data extraction from unstructured medical text. JAMIA Open. 2025;8(5). doi:10.1093/jamiaopen/ooaf097
BibTeX
@article{Dao2025,
author = {Dao, Nam and Quesada, Luisa and Hassan, Syed Moin and Campo, Monica Iturrioz and Johnson, Shelsey and Ghose, Suchandra and San José Estépar, Raúl and Waxman, Aaron and Washko, George and Rahaghi, Farbod N},
title = {Generative artificial intelligence for automated data extraction from unstructured medical text},
journal = {JAMIA Open},
year = {2025},
volume = {8},
number = {5},
doi = {10.1093/jamiaopen/ooaf097}
}
EUVIMED