A Large Language Model to Detect Negated Expressions in Radiology Reports
Journal of Imaging Informatics in Medicine · 25.09.2024 · 38 (3) · 1297-1303
DOI: 10.1007/s10278-024-01274-9 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Imaging Informatics in Medicine, online erschienen am 25.09.2024. Verfasst von Yvonne Su und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Perelman School of Medicine, University of Pennsylvania, 3400 Spruce Street, Philadelphia, 19104, PA, USA. Hauptthemen laut MEDLINE: Natural Language Processing, Radiology Information Systems. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Natural language processing (NLP) is crucial to extract information accurately from unstructured text to provide insights for clinical decision-making, quality improvement, and medical research. This study compared the performance of a rule-based NLP system and a medical-domain transformer-based model to detect negated concepts in radiology reports. Using a corpus of 984 de-identified radiology reports from a large U.S.-based academic health system (1000 consecutive reports, excluding 16 duplicates), the investigators compared the rule-based medspaCy system and the Clinical Assertion and Negation Classification Bidirectional Encoder Representations from Transformers (CAN-BERT) system to detect negated expressions of terms from RadLex, the Unified Medical Language System Metathesaurus, and the Radiology Gamuts Ontology. Power analysis determined a sample size of 382 terms to achieve α = 0.05 and β = 0.8 for McNemar’s test; based on an estimate of 15% negated terms, 2800 randomly selected terms were annotated manually as negated or not negated. Precision, recall, and F1 of the two models were compared using McNemar’s test. Of the 2800 terms, 387 (13.8%) were negated. For negation detection, medspaCy attained a recall of 0.795, precision of 0.356, and F1 of 0.492. CAN-BERT achieved a recall of 0.785, precision of 0.768, and F1 of 0.777. Although recall was not significantly different, CAN-BERT had significantly better precision ( χ 2 = 304.64; p < 0.001). The transformer-based CAN-BERT model detected negated terms in radiology reports with high precision and recall; its precision significantly exceeded that of the rule-based medspaCy system. Use of this system will improve data extraction from textual reports to support information retrieval, AI model training, and discovery of causal relationships.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, 3400 Spruce Street, Philadelphia, 19104, PA, USA.
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, 3400 Spruce Street, Philadelphia, 19104, PA, USA. ckahn@upenn.edu.
- Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA. ckahn@upenn.edu.
Zitieren
APA
Su, Y., Babore, Y. B., & Kahn, C. E. (2024). A Large Language Model to Detect Negated Expressions in Radiology Reports. Journal of Imaging Informatics in Medicine, 38(3), 1297-1303. https://doi.org/10.1007/s10278-024-01274-9
Vancouver
Su Y, Babore YB, Kahn CE. A Large Language Model to Detect Negated Expressions in Radiology Reports. Journal of Imaging Informatics in Medicine. 2024;38(3):1297-1303. doi:10.1007/s10278-024-01274-9
BibTeX
@article{Su2024,
author = {Su, Yvonne and Babore, Yonatan B and Kahn, Charles E},
title = {A Large Language Model to Detect Negated Expressions in Radiology Reports},
journal = {Journal of Imaging Informatics in Medicine},
year = {2024},
volume = {38},
number = {3},
pages = {1297--1303},
doi = {10.1007/s10278-024-01274-9}
}
EUVIMED