Feasibility Study for Using Large Language Models to Identify Goals-of-Care Documentation at Scale in Patients With Advanced Cancer
JCO Oncology Practice · 2026-02 · 22 (2) · 294-305
DOI: 10.1200/op-24-00992 ↗Kurz gefasst
Fachartikel in der Zeitschrift JCO Oncology Practice, 2026-02. Verfasst von Nicole D Agaronnik und 4 weiteren Autor:innen. Beteiligte Einrichtung: Harvard Medical School, Boston, MA. Hauptthemen laut MEDLINE: Neoplasms, Documentation, Electronic Health Records, Language. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 31 Literaturangaben.
Abstract (Originalsprache)
PURPOSE The purpose of our study was to (1) use a large language model (LLM) to identify goals-of-care (GOC) conversations in a large volume of notes, and (2) explore the potential of LLMs for a novel summarization task. METHODS We included patients diagnosed with advanced cancer between April 1, 2024, and June 30, 2024. A validated LLM prompt for GOC was applied to electronic health records (EHRs) using a Health Insurance Portability and Accountability Act (HIPPA)-secure version of GPT-4o, a LLM developed by OpenAI. Output included (1) presence or absence of GOC documentation, (2) explanations with source text used to inform the LLM's determination, and (3) a hallucination score, indicating proportion of source text generated by the LLM that did not perfectly match text in the EHR. Two LLM prompts were designed to generate structured and unstructured GOC summaries. We randomly selected five patients and applied the summarization task to notes flagged by LLM as containing GOC. We reviewed LLM summaries to examine for relevant information. RESULTS Among 326 patients associated with nearly 1,400 clinical notes, LLM flagged approximately 40% of notes for GOC documentation. Subsequent review of explanation text identified that 128 patients (nearly 40% of the total patient population) had GOC documentation. The hallucination index for explanations was low, suggesting that the LLM did not produce text that was not found in EHRs. LLM prompts produced accurate summaries in less than 2 minutes per patient. CONCLUSION LLMs can capture GOC at scale and generate clinically useful summaries. Future directions include real-time implementation in the clinical setting.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Harvard Medical School, Boston, MA.
- Dana-Farber Cancer Institute, Boston, MA.
- Albany Medical College, Albany, NY.
- Brigham and Women's Hospital, Boston, MA.
ORCID-Profile: Nicole D Agaronnik, Joshua Davis, Christopher R Manz, James A Tulsky, Charlotta Lindvall
Zitieren
APA
Agaronnik, N. D., Davis, J., Manz, C. R., Tulsky, J. A., & Lindvall, C. (2026). Feasibility Study for Using Large Language Models to Identify Goals-of-Care Documentation at Scale in Patients With Advanced Cancer. JCO Oncology Practice, 22(2), 294-305. https://doi.org/10.1200/op-24-00992
Vancouver
Agaronnik ND, Davis J, Manz CR, Tulsky JA, Lindvall C. Feasibility Study for Using Large Language Models to Identify Goals-of-Care Documentation at Scale in Patients With Advanced Cancer. JCO Oncology Practice. 2026;22(2):294-305. doi:10.1200/op-24-00992
BibTeX
@article{Agaronnik2026,
author = {Agaronnik, Nicole D and Davis, Joshua and Manz, Christopher R and Tulsky, James A and Lindvall, Charlotta},
title = {Feasibility Study for Using Large Language Models to Identify Goals-of-Care Documentation at Scale in Patients With Advanced Cancer},
journal = {JCO Oncology Practice},
year = {2026},
volume = {22},
number = {2},
pages = {294--305},
doi = {10.1200/op-24-00992}
}
EUVIMED