Using GPT ‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports
Liver International · 2024-07 · 44 (7) · 1578-1587
DOI: 10.1111/liv.15891 ↗Kurz gefasst
Fachartikel in der Zeitschrift Liver International, online erschienen am 23.04.2024. Verfasst von Kyowon Gu und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea. Hauptthemen laut MEDLINE: Magnetic Resonance Imaging, Liver Neoplasms. Die Arbeit wurde bisher 45-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Korea Health Industry Development Institute.
Abstract (Originalsprache)
Background and Aims The Liver Imaging Reporting and Data System (LI‐RADS) offers a standardized approach for imaging hepatocellular carcinoma. However, the diverse styles and structures of radiology reports complicate automatic data extraction. Large language models hold the potential for structured data extraction from free‐text reports. Our objective was to evaluate the performance of Generative Pre‐trained Transformer (GPT)‐4 in extracting LI‐RADS features and categories from free‐text liver magnetic resonance imaging (MRI) reports.
Methods
Three radiologists generated 160 fictitious free‐text liver MRI reports written in Korean and English, simulating real‐world practice. Of these, 20 were used for prompt engineering, and 140 formed the internal test cohort. Seventy‐two genuine reports, authored by 17 radiologists were collected and de‐identified for the external test cohort. LI‐RADS features were extracted using GPT‐4, with a Python script calculating categories. Accuracies in each test cohort were compared.
Results
On the external test, the accuracy for the extraction of major LI‐RADS features, which encompass size, nonrim arterial phase hyperenhancement, nonperipheral ‘washout’, enhancing ‘capsule’ and threshold growth, ranged from .92 to .99. For the rest of the LI‐RADS features, the accuracy ranged from .86 to .97. For the LI‐RADS category, the model showed an accuracy of .85 (95% CI: .76, .93).
Conclusions
GPT‐4 shows promise in extracting LI‐RADS features, yet further refinement of its prompting strategy and advancements in its neural network architecture are crucial for reliable use in processing complex real‐world MRI reports.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea.
- Department of Radiology, Soonchunhyang University College of Medicine, Seoul Hospital, Seoul, Republic of Korea.
ORCID-Profile: Kyowon Gu, Jeong Hyun Lee, Jeong Ah Hwang, Min Woo Lee
Förderung
- Korea Health Industry Development Institute
Zitieren
APA
Gu, K., Lee, J. H., Shin, J., Hwang, J. A., Min, J. H., Jeong, W. K., Lee, M. W., Song, K. D., & Bae, S. H. (2024). Using GPT ‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports. Liver International, 44(7), 1578-1587. https://doi.org/10.1111/liv.15891
Vancouver
Gu K, Lee JH, Shin J, Hwang JA, Min JH, Jeong WK, et al. Using GPT ‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports. Liver International. 2024;44(7):1578-1587. doi:10.1111/liv.15891
BibTeX
@article{Gu2024,
author = {Gu, Kyowon and Lee, Jeong Hyun and Shin, Jaeseung and Hwang, Jeong Ah and Min, Ji Hye and Jeong, Woo Kyoung and Lee, Min Woo and Song, Kyoung Doo and Bae, Sung Hwan},
title = {Using GPT ‐4 for LI‐RADS feature extraction and categorization with multilingual free‐text reports},
journal = {Liver International},
year = {2024},
volume = {44},
number = {7},
pages = {1578--1587},
doi = {10.1111/liv.15891}
}
EUVIMED