Using a Large Language Model–generated Prompt to Extract Features from Synthetic MRI Brain Scan Reports: A Cross-sectional Study
Methods of Information in Medicine · 01.05.2025 · 64 (01/02) · 054-062
DOI: 10.1055/a-2797-4295 ↗Kurz gefasst
Querschnittstudie in der Zeitschrift Methods of Information in Medicine, erschienen am 01.05.2025. Verfasst von John J Hanna und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Internal Medicine, ECU Brody School of Medicine, Greenville, North Carolina, United States. Hauptthemen laut MEDLINE: Large Language Models, Magnetic Resonance Imaging, Brain. Die Arbeit nennt 20 Literaturangaben.
Abstract (Originalsprache)
Feature extraction from free text medical reports is a frequently required clinical, operational, or research procedure. Large language models (LLMs) hold a promise for automating feature extraction, which can also enable category assignment tasks. To compare the groundedness of extracted features by five LLMs from magnetic resonance imaging (MRI) brain scan reports using a clinician-engineered versus an LLM-generated prompt. Five OpenAI LLMs were evaluated for their ability to extract nine binary features from synthetic MRI brain reports. Two types of prompts, a clinician-engineered and an LLM-generated, were used. Metrics including recall, precision, accuracy, and F1 score were calculated to assess model performance. For all extracted features by all studied models from both tested prompts, the overall average recall was 0.956, the average precision was 0.9347, the average accuracy was 0.982, and the average F1 score was 0.9431. Using GPT-3.5-turbo, the LLM-generated prompt had better numerical performance than the clinician-engineered prompt. For the other four GPT-4 models examined, overall recall, precision, and accuracy were higher regardless of the prompt source. This study highlights the potential of LLMs to generate prompts and accurately extract features, with newer models like GPT-4 performing consistently well. The efficacy of feature extraction by LLMs depends on the engineered prompt and model used. Our experimentation demonstrates the potential of LLMs to engineer prompts and extract features from MRI brain scan reports.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Internal Medicine, ECU Brody School of Medicine, Greenville, North Carolina, United States.
- Information Services, ECU Health, Greenville, North Carolina, United States.
- Clinical Informatics Center, University of Texas Southwestern, Dallas, Texas, United States.
- Department of Emergency Medicine, ECU Brody School of Medicine, Greenville, North Carolina, United States.
- ECU Health Neurosurgery and Spine, ECU Health, Greenville, North Carolina, United States.
- Department of Pediatrics, University of Texas Southwestern, Dallas, Texas, United States.
Zitieren
APA
Hanna, J. J., Evans, C. S., Dennis, C. R., Lee, K. S., Lehmann, C. U., & Medford, R. J. (2025). Using a Large Language Model–generated Prompt to Extract Features from Synthetic MRI Brain Scan Reports: A Cross-sectional Study. Methods of Information in Medicine, 64(01/02), 054-062. https://doi.org/10.1055/a-2797-4295
Vancouver
Hanna JJ, Evans CS, Dennis CR, Lee KS, Lehmann CU, Medford RJ. Using a Large Language Model–generated Prompt to Extract Features from Synthetic MRI Brain Scan Reports: A Cross-sectional Study. Methods of Information in Medicine. 2025;64(01/02):054-062. doi:10.1055/a-2797-4295
BibTeX
@article{Hanna2025,
author = {Hanna, John J and Evans, Christopher S and Dennis, Christopher R and Lee, K Stuart and Lehmann, Christoph U and Medford, Richard J},
title = {Using a Large Language Model–generated Prompt to Extract Features from Synthetic MRI Brain Scan Reports: A Cross-sectional Study},
journal = {Methods of Information in Medicine},
year = {2025},
volume = {64},
number = {01/02},
pages = {054--062},
doi = {10.1055/a-2797-4295}
}
EUVIMED