Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes
JCO Clinical Cancer Informatics · 2025
DOI: 10.1200/cci-25-00002 ↗Kurz gefasst
Fachartikel in der Zeitschrift JCO Clinical Cancer Informatics, 2025. Verfasst von Amara Tariq und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Mayo Clinic, Phoenix, AZ. Hauptthemen laut MEDLINE: Breast Neoplasms, Electronic Health Records, Data Mining, Unified Medical Language System. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCI NIH HHS.
Abstract (Originalsprache)
PURPOSE Automated curation of breast cancer treatment data with minimal human involvement could accelerate the collection of statewide and nationwide evidence for patient management and assessing the effectiveness of treatment pathways. The primary challenges are the complexity and inconsistency of structured clinical data streams and accurate extraction of this information from free-text clinical narratives. MATERIALS AND METHODS We proposed a hybrid two-phase information extraction framework that combined a Unified Medical Language System parser (phase-1) with a fine-tuned large language model (LLM; phase-2) to extract longitudinal treatment timelines from time-stamped clinical notes. Our framework was developed through end-to-end joint learning as a question-answering model, where the model was trained to simultaneously answer five questions, each corresponding to a specific treatment. RESULTS We fine-tuned and internally validated the model on 26,692 patients with breast cancer (diagnosed between 2013 and 2020) receiving treatment at Mayo Clinic and externally validated the model on 162 randomly selected patients from Stanford Healthcare. Zero-shot LLM (out-of-the-box) had high specificity but low sensitivity, indicating that although these frameworks are useful for generic language understanding, they are lacking in terms of targeted clinical tasks. The proposed model achieved 0.942 average AUROC on the internal and 0.924 on the external data, demonstrating only marginal drop in performance when evaluated on external. The proposed model also achieved better trade-off between sensitivity (average: 79.2%) and specificity (average: 76.2%) compared with rule-based (average sensitivity: 70.5%, average specificity: 68.1%) and structured codes (average sensitivity: 64.1%, average specificity: 83.5%). CONCLUSION The proposed framework can extract temporal information about cancer treatments from various time-stamped clinic notes, regardless of the setting of treatment administration (inpatient or outpatient) or time frame. To support the cancer research community for such data curation and longitudinal analysis, we have packaged the code as a docker image, which needs minimal system reconfiguration and shared with an open-source academic license.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Radiology, Mayo Clinic, Phoenix, AZ.
- Departments of Medicine and of Epidemiology & Population Health, Stanford University School of Medicine, Palo Alto, CA.
- Rollins School of Public Health, Emory University, Atlanta, GA.
- Department of Internal Medicine, UC Davis School of Medicine, Sacramento, CA.
- Department of Biomedical Data Science, Radiology, and Medicine, Stanford University School of Medicine, Palo Alto, CA.
- School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ.
ORCID-Profile: Amara Tariq, Madhu Sikha, Allison W Kurian, Kevin Ward, Theresa H M Keegan, Daniel L Rubin, Imon Banerjee
Förderung
- NCI NIH HHS
Zitieren
APA
Tariq, A., Sikha, M., Kurian, A. W., Ward, K., Keegan, T. H. M., Rubin, D. L., & Banerjee, I. (2025). Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes. JCO Clinical Cancer Informatics. https://doi.org/10.1200/cci-25-00002
Vancouver
Tariq A, Sikha M, Kurian AW, Ward K, Keegan THM, Rubin DL, et al. Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes. JCO Clinical Cancer Informatics. 2025. doi:10.1200/cci-25-00002
BibTeX
@article{Tariq2025,
author = {Tariq, Amara and Sikha, Madhu and Kurian, Allison W and Ward, Kevin and Keegan, Theresa H M and Rubin, Daniel L and Banerjee, Imon},
title = {Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes},
journal = {JCO Clinical Cancer Informatics},
year = {2025},
doi = {10.1200/cci-25-00002}
}
EUVIMED