Identifying Distant Breast Cancer Recurrence: A Proof-of-Concept Comparison of Discrete-Field Algorithm and Large Language Model Performance and Cost
JCO Clinical Cancer Informatics · 2026-10 · 10 (4)
DOI: 10.1200/cci-26-00103 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JCO Clinical Cancer Informatics, 2026-10. Verfasst von Megan A Mullins und 14 weiteren Autor:innen. Beteiligte Einrichtung: Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, TX. Hauptthemen laut MEDLINE: Breast Neoplasms, Neoplasm Recurrence, Local, Algorithms. Die Arbeit nennt 29 Literaturangaben.
Abstract (Originalsprache)
Purpose
Reliably identifying distant cancer recurrence and determining its timing remain longstanding challenges that limit metastatic cancer research. We characterized the performance of a large language model (LLM)–based approach versus gold standard chart review and a validated discrete-field algorithm for identifying distant metastatic breast cancer (mBC) recurrence.
Methods
Using electronic health record data and notes from January 2010 to June 2023, we applied the Ritzwoller et al discrete-field algorithm and an LLM using GPT-5-mini for identifying distant mBC recurrence. Both were assessed against manual chart review. To optimize computation, the LLM processed clinical notes containing metasta by year in ascending temporal order. Once a patient was classified as having a distant recurrence, subsequent notes were excluded from further processing. We assessed the performance of case identification and timing of recurrence.
Results
Our cohort included 429 women initially diagnosed with stage I to III breast cancer. Compared with manual chart review, GPT-5-mini achieved a sensitivity of 1.00 (95% CI, 0.96 to 1.00), specificity of 0.94 (95% CI, 0.91 to 0.96), PPV of 0.80 (95% CI, 0.72 to 0.87), and NPV of 1.00 (95% CI, 0.99 to 1.00). The algorithm had a sensitivity of 0.82 (95% CI, 0.73 to 0.89), specificity of 0.98 (95% CI, 0.96 to 0.99), PPV of 0.91 (95% CI, 0.83 to 0.96), and NPV of 0.95 (95% CI, 0.93 to 0.97). For recurrence timing, 69.0% of GPT-5-mini recurrence dates were within 1 month of the gold-standard date compared with only 10% of algorithm estimates.
Conclusion
In this single-institution proof-of-concept study, LLM performance is well-suited for more inclusive case capture than manual review or complex algorithm application with minimal instruction and could be further improved with disease-specific knowledge.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, TX.
- Department of Health Economics, Systems, and Policy, Peter O'Donnell Jr. School of Public Health, University of Texas Southwestern Medical Center, Dallas, TX.
- Harold C. Simmons Comprehensive Cancer Center, UT Southwestern Medical Center, Dallas, TX.
- Department of Health Data Science & Biostatistics, Peter O'Donnell Jr. School of Public Health University of Texas Southwestern Medical Center, Dallas, TX.
- Magnet Program and Office of Nursing Research, University of Texas Southwestern Medical Center, Dallas, TX.
- Department of Surgical Oncology, University of Texas Southwestern Medical Center, Dallas, TX.
ORCID-Profile: Megan A Mullins, Lesi He, Omer Abdrabbuh, Abril Carrillo, Malcolm Su, Pranathi Pilla, Yingzi Zhang, Francisco Villalobos, Shifa Kanjwal, Wenqi Shi, Arthur S Hong, Jason B Fleming, Timothy P Hogan, Wendy Chapman
Zitieren
APA
Mullins, M. A., He, L., Rana, C., Abdrabbuh, O., Carrillo, A., Su, M., Pilla, P., Zhang, Y., Villalobos, F., Kanjwal, S., Shi, W., Hong, A. S., Fleming, J. B., Hogan, T. P., & Chapman, W. (2026). Identifying Distant Breast Cancer Recurrence: A Proof-of-Concept Comparison of Discrete-Field Algorithm and Large Language Model Performance and Cost. JCO Clinical Cancer Informatics, 10(4). https://doi.org/10.1200/cci-26-00103
Vancouver
Mullins MA, He L, Rana C, Abdrabbuh O, Carrillo A, Su M, et al. Identifying Distant Breast Cancer Recurrence: A Proof-of-Concept Comparison of Discrete-Field Algorithm and Large Language Model Performance and Cost. JCO Clinical Cancer Informatics. 2026;10(4). doi:10.1200/cci-26-00103
BibTeX
@article{Mullins2026,
author = {Mullins, Megan A and He, Lesi and Rana, Chahat and Abdrabbuh, Omer and Carrillo, Abril and Su, Malcolm and Pilla, Pranathi and Zhang, Yingzi and Villalobos, Francisco and Kanjwal, Shifa and Shi, Wenqi and Hong, Arthur S and Fleming, Jason B and Hogan, Timothy P and Chapman, Wendy},
title = {Identifying Distant Breast Cancer Recurrence: A Proof-of-Concept Comparison of Discrete-Field Algorithm and Large Language Model Performance and Cost},
journal = {JCO Clinical Cancer Informatics},
year = {2026},
volume = {10},
number = {4},
doi = {10.1200/cci-26-00103}
}
EUVIMED