EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Leveraging Large Language Models for Automated Extraction of Abdominal Aortic Aneurysm Features from Radiology Reports

Praneel Mukherjee, Ryan C Lee iD, Roham Hadidchi iD, Sonya Henry iD, Michael Coard, Matthew Davis, Yossef Rubinov, Ha Nguyen-Luong iD, Leah Katz, Tim Q Duong iD

Diagnostics · 03.04.2026 · 16 (7) · 1083

DOI: 10.3390/diagnostics16071083 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Diagnostics, online erschienen am 03.04.2026. Verfasst von Praneel Mukherjee und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Montefiore Medical Center and Albert Einstein College of Medicine, Bronx, NY 10461, USA. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives. Abdominal computed tomography (CT) radiology reports contain critical information for abdominal aortic aneurysm (AAA) management, including aneurysm presence, size, rupture status, and prior repair. However, this information is often embedded within lengthy, heterogeneous reports, making manual extraction inefficient. We evaluated the performance of multiple large language models (LLMs) for automated extraction of AAA-related findings from radiology reports.

Methods

We retrospectively analyzed 500 abdominal CT reports mentioning AAA from an urban academic health system (2020–2024). Ground truth labels were established by manual review. Four open-source LLMs (Qwen2.5-7B-Instruct, Llama3-Med42-8B, GPT-OSS-20B, and MedGemma-27B-text-it) were evaluated for extraction of aneurysm presence, size, morphology, rupture status, impending rupture, and prior aortic repair. Model outputs were compared with ground truth using exact-match accuracy, and inter-model agreement was assessed using Fleiss’ kappa. Reasoning traces were examined to characterize correct and incorrect model behavior.

Results

Accuracy for identifying AAA presence ranged from 0.90 to 0.95 (κ = 0.851), and prior aortic repair from 0.90 to 0.97 (κ = 0.793). Accuracy for aneurysm size ranged from 0.67 to 0.88 (κ = 0.340), with low κ’s due to class imbalance or dimension misselection. Rupture and impending rupture were identified with accuracies exceeding 0.90 across models, though agreement was lower (κ = 0.485 and 0.589), reflecting low event prevalence. Larger models (GPT-OSS-20B, MedGemma-27B) generally outperformed smaller models. Reasoning analysis revealed strengths in measurement prioritization but recurrent errors, including dimension misselection, over-inference of prior repair, and conservative classification of rupture-related findings.

Conclusions

LLMs can accurately extract clinically relevant AAA information from radiology reports with interpretable reasoning, with larger and medically trained models outperforming smaller or general-purpose models. Performance varies by task and model, underscoring the need for careful validation and human-in-the-loop deployment in clinical settings.

Schlagwörter

Stichwörter der Autor:innen

abdominal aortic aneurysmcomputed tomographyinformation extractionlarge language modelradiology report

Autor:innen und Institutionen

  • Department of Radiology, Montefiore Medical Center and Albert Einstein College of Medicine, Bronx, NY 10461, USA.
  • Renaissance School of Medicine, Stony Brook University, Stony Brook, NY 11794, USA.

ORCID-Profile: Ryan C Lee, Roham Hadidchi, Sonya Henry, Ha Nguyen-Luong, Tim Q Duong

Zitieren

APA

Mukherjee, P., Lee, R. C., Hadidchi, R., Henry, S., Coard, M., Davis, M., Rubinov, Y., Nguyen-Luong, H., Katz, L., & Duong, T. Q. (2026). Leveraging Large Language Models for Automated Extraction of Abdominal Aortic Aneurysm Features from Radiology Reports. Diagnostics, 16(7), 1083. https://doi.org/10.3390/diagnostics16071083

Vancouver

Mukherjee P, Lee RC, Hadidchi R, Henry S, Coard M, Davis M, et al. Leveraging Large Language Models for Automated Extraction of Abdominal Aortic Aneurysm Features from Radiology Reports. Diagnostics. 2026;16(7):1083. doi:10.3390/diagnostics16071083

BibTeX
@article{Mukherjee2026,
  author = {Mukherjee, Praneel and Lee, Ryan C and Hadidchi, Roham and Henry, Sonya and Coard, Michael and Davis, Matthew and Rubinov, Yossef and Nguyen-Luong, Ha and Katz, Leah and Duong, Tim Q},
  title = {Leveraging Large Language Models for Automated Extraction of Abdominal Aortic Aneurysm Features from Radiology Reports},
  journal = {Diagnostics},
  year = {2026},
  volume = {16},
  number = {7},
  pages = {1083},
  doi = {10.3390/diagnostics16071083}
}
Kostenlos starten →