EUVIMEDEUVIMED entdecken
Open Access

CACER: Clinical concept Annotations for Cancer Events and Relations

Yujuan Velvin Fu, Giridhar Kaushik Ramachandran iD, Ahmad Halwani, Bridget T McInnes, Fei Xia, Kevin Lybarger iD, Meliha Yetisgen, Özlem Uzuner iD

Journal of the American Medical Informatics Association · 2024

DOI: 10.1093/jamia/ocae231 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2024. Verfasst von Yujuan Velvin Fu und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, WA 98195, United States. Hauptthemen laut MEDLINE: Neoplasms, Natural Language Processing, Electronic Health Records. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NLM NIH HHS, NIH HHS, NIH.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. Materials and Methods We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48 000 medical problems and drug events and 10 000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL).

Results

In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks.

Discussion

The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models.

Conclusions

We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

NeoplasmsNatural Language ProcessingElectronic Health RecordsHumansAntineoplastic AgentsSemanticsData Mining

Stichwörter der Autor:innen

cancer patientsdata miningelectronic health recordsinformation extractionmachine learningnatural language processing

Substanzen

Antineoplastic Agents

Autor:innen und Institutionen

  • Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, WA 98195, United States.
  • Department of Information Sciences and Technology, George Mason University, Fairfax, VA 22030, United States.
  • Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, United States.
  • Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, United States.
  • Department of Linguistics, University of Washington, Seattle, WA 98195, United States.

ORCID-Profile: Giridhar Kaushik Ramachandran, Kevin Lybarger, Özlem Uzuner

Förderung

  • NLM NIH HHS
  • NIH HHS
  • NIH
  • NCI NIH HHS
  • National Institutes of Health
  • National Cancer Institute
  • National Library of Medicine

Zitieren

APA

Fu, Y. V., Ramachandran, G. K., Halwani, A., McInnes, B. T., Xia, F., Lybarger, K., Yetisgen, M., & Uzuner, Ö. (2024). CACER: Clinical concept Annotations for Cancer Events and Relations. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocae231

Vancouver

Fu YV, Ramachandran GK, Halwani A, McInnes BT, Xia F, Lybarger K, et al. CACER: Clinical concept Annotations for Cancer Events and Relations. Journal of the American Medical Informatics Association. 2024. doi:10.1093/jamia/ocae231

BibTeX
@article{Fu2024,
  author = {Fu, Yujuan Velvin and Ramachandran, Giridhar Kaushik and Halwani, Ahmad and McInnes, Bridget T and Xia, Fei and Lybarger, Kevin and Yetisgen, Meliha and Uzuner, Özlem},
  title = {CACER: Clinical concept Annotations for Cancer Events and Relations},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  doi = {10.1093/jamia/ocae231}
}
Kostenlos starten →