EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Large Language Model to Detect Negated Expressions in Radiology Reports

Yvonne Su, Yonatan B Babore, Charles E Kahn

Journal of Imaging Informatics in Medicine · 25.09.2024 · 38 (3) · 1297-1303

DOI: 10.1007/s10278-024-01274-9 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging Informatics in Medicine, online erschienen am 25.09.2024. Verfasst von Yvonne Su und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Perelman School of Medicine, University of Pennsylvania, 3400 Spruce Street, Philadelphia, 19104, PA, USA. Hauptthemen laut MEDLINE: Natural Language Processing, Radiology Information Systems. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Natural language processing (NLP) is crucial to extract information accurately from unstructured text to provide insights for clinical decision-making, quality improvement, and medical research. This study compared the performance of a rule-based NLP system and a medical-domain transformer-based model to detect negated concepts in radiology reports. Using a corpus of 984 de-identified radiology reports from a large U.S.-based academic health system (1000 consecutive reports, excluding 16 duplicates), the investigators compared the rule-based medspaCy system and the Clinical Assertion and Negation Classification Bidirectional Encoder Representations from Transformers (CAN-BERT) system to detect negated expressions of terms from RadLex, the Unified Medical Language System Metathesaurus, and the Radiology Gamuts Ontology. Power analysis determined a sample size of 382 terms to achieve α = 0.05 and β = 0.8 for McNemar’s test; based on an estimate of 15% negated terms, 2800 randomly selected terms were annotated manually as negated or not negated. Precision, recall, and F1 of the two models were compared using McNemar’s test. Of the 2800 terms, 387 (13.8%) were negated. For negation detection, medspaCy attained a recall of 0.795, precision of 0.356, and F1 of 0.492. CAN-BERT achieved a recall of 0.785, precision of 0.768, and F1 of 0.777. Although recall was not significantly different, CAN-BERT had significantly better precision ( χ 2 = 304.64; p < 0.001). The transformer-based CAN-BERT model detected negated terms in radiology reports with high precision and recall; its precision significantly exceeded that of the rule-based medspaCy system. Use of this system will improve data extraction from textual reports to support information retrieval, AI model training, and discovery of causal relationships.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingRadiology Information SystemsHumansLarge Language Models

Stichwörter der Autor:innen

Large language modelsNamed entity recognitionNatural language processingNegated expression (negex) detectionRadiology reports

Autor:innen und Institutionen

  • Department of Radiology, Perelman School of Medicine, University of Pennsylvania, 3400 Spruce Street, Philadelphia, 19104, PA, USA.
  • Department of Radiology, Perelman School of Medicine, University of Pennsylvania, 3400 Spruce Street, Philadelphia, 19104, PA, USA. ckahn@upenn.edu.
  • Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA. ckahn@upenn.edu.

Zitieren

APA

Su, Y., Babore, Y. B., & Kahn, C. E. (2024). A Large Language Model to Detect Negated Expressions in Radiology Reports. Journal of Imaging Informatics in Medicine, 38(3), 1297-1303. https://doi.org/10.1007/s10278-024-01274-9

Vancouver

Su Y, Babore YB, Kahn CE. A Large Language Model to Detect Negated Expressions in Radiology Reports. Journal of Imaging Informatics in Medicine. 2024;38(3):1297-1303. doi:10.1007/s10278-024-01274-9

BibTeX
@article{Su2024,
  author = {Su, Yvonne and Babore, Yonatan B and Kahn, Charles E},
  title = {A Large Language Model to Detect Negated Expressions in Radiology Reports},
  journal = {Journal of Imaging Informatics in Medicine},
  year = {2024},
  volume = {38},
  number = {3},
  pages = {1297--1303},
  doi = {10.1007/s10278-024-01274-9}
}
Kostenlos starten →