EUVIMEDEUVIMED entdecken
Open Access

Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes

Amara Tariq iD, Madhu Sikha iD, Allison W Kurian iD, Kevin Ward iD, Theresa H M Keegan iD, Daniel L Rubin iD, Imon Banerjee iD

JCO Clinical Cancer Informatics · 2025

DOI: 10.1200/cci-25-00002 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JCO Clinical Cancer Informatics, 2025. Verfasst von Amara Tariq und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology, Mayo Clinic, Phoenix, AZ. Hauptthemen laut MEDLINE: Breast Neoplasms, Electronic Health Records, Data Mining, Unified Medical Language System. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

PURPOSE Automated curation of breast cancer treatment data with minimal human involvement could accelerate the collection of statewide and nationwide evidence for patient management and assessing the effectiveness of treatment pathways. The primary challenges are the complexity and inconsistency of structured clinical data streams and accurate extraction of this information from free-text clinical narratives. MATERIALS AND METHODS We proposed a hybrid two-phase information extraction framework that combined a Unified Medical Language System parser (phase-1) with a fine-tuned large language model (LLM; phase-2) to extract longitudinal treatment timelines from time-stamped clinical notes. Our framework was developed through end-to-end joint learning as a question-answering model, where the model was trained to simultaneously answer five questions, each corresponding to a specific treatment. RESULTS We fine-tuned and internally validated the model on 26,692 patients with breast cancer (diagnosed between 2013 and 2020) receiving treatment at Mayo Clinic and externally validated the model on 162 randomly selected patients from Stanford Healthcare. Zero-shot LLM (out-of-the-box) had high specificity but low sensitivity, indicating that although these frameworks are useful for generic language understanding, they are lacking in terms of targeted clinical tasks. The proposed model achieved 0.942 average AUROC on the internal and 0.924 on the external data, demonstrating only marginal drop in performance when evaluated on external. The proposed model also achieved better trade-off between sensitivity (average: 79.2%) and specificity (average: 76.2%) compared with rule-based (average sensitivity: 70.5%, average specificity: 68.1%) and structured codes (average sensitivity: 64.1%, average specificity: 83.5%). CONCLUSION The proposed framework can extract temporal information about cancer treatments from various time-stamped clinic notes, regardless of the setting of treatment administration (inpatient or outpatient) or time frame. To support the cancer research community for such data curation and longitudinal analysis, we have packaged the code as a docker image, which needs minimal system reconfiguration and shared with an open-source academic license.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Breast NeoplasmsElectronic Health RecordsData MiningUnified Medical Language SystemHumansFemaleNatural Language ProcessingLarge Language Models

Autor:innen und Institutionen

  • Department of Radiology, Mayo Clinic, Phoenix, AZ.
  • Departments of Medicine and of Epidemiology & Population Health, Stanford University School of Medicine, Palo Alto, CA.
  • Rollins School of Public Health, Emory University, Atlanta, GA.
  • Department of Internal Medicine, UC Davis School of Medicine, Sacramento, CA.
  • Department of Biomedical Data Science, Radiology, and Medicine, Stanford University School of Medicine, Palo Alto, CA.
  • School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ.

ORCID-Profile: Amara Tariq, Madhu Sikha, Allison W Kurian, Kevin Ward, Theresa H M Keegan, Daniel L Rubin, Imon Banerjee

Förderung

  • NCI NIH HHS

Zitieren

APA

Tariq, A., Sikha, M., Kurian, A. W., Ward, K., Keegan, T. H. M., Rubin, D. L., & Banerjee, I. (2025). Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes. JCO Clinical Cancer Informatics. https://doi.org/10.1200/cci-25-00002

Vancouver

Tariq A, Sikha M, Kurian AW, Ward K, Keegan THM, Rubin DL, et al. Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes. JCO Clinical Cancer Informatics. 2025. doi:10.1200/cci-25-00002

BibTeX
@article{Tariq2025,
  author = {Tariq, Amara and Sikha, Madhu and Kurian, Allison W and Ward, Kevin and Keegan, Theresa H M and Rubin, Daniel L and Banerjee, Imon},
  title = {Open-Source Hybrid Large Language Model Integrated System for Extraction of Breast Cancer Treatment Pathway From Free-Text Clinical Notes},
  journal = {JCO Clinical Cancer Informatics},
  year = {2025},
  doi = {10.1200/cci-25-00002}
}
Kostenlos starten →