A Large Language Model–Driven System for Advance Care Planning Training Among Health Care Providers in the Chinese Context: Development and Technical Evaluation
Journal of Medical Internet Research · 2026-07-28 · 28 · e87288-e87288
DOI: 10.2196/87288 ↗At a glance
Journal article in Journal of Medical Internet Research, published online on 2026-07-28. Written by Minghui Tan and 6 co-authors. Institution: Xiangya School of Nursing, Central South University, 172 Tongzipo Road, Changhsa, Hunan, 410013, China, 86 18711000745. Major MEDLINE topics: Large Language Models, Health Personnel, Advance Care Planning. Cited 1 times so far (Crossref); 57 references. The full text is freely available in PubMed Central.
Topics: Ethik in der Pflege, Künstliche Intelligenz in der Pflege
Abstract (original language)
Background
With the expanding need for advance care planning (ACP), innovative educational strategies for training health care providers are increasingly required. Large language model (LLM)–based ACP chatbots offer a novel and potentially effective solution to enhance health care providers’ competence in navigating complex ACP conversations.
Objective
This study aimed to develop a Chinese-context ACP corpus to support an LLM-based chatbot and evaluate the feasibility and performance of a multi-agent system for simulating complex ACP discussions as a training tool for health care providers.
Methods
This study involved dataset construction and model adaptation and evaluation. We constructed 3 structured datasets using synthetic dialogue data generated through prompts derived from ACP-related scientific literature and policy documents. Both open-source (Zhongjing) and closed-source LLMs (GPT-4o-mini) were chosen as baseline models. The Zhongjing model was adapted through fine-tuning, whereas GPT-4o-mini was adapted using both fine-tuning and prompt engineering. Model performance was assessed through automatic and human evaluations following the QUEST (Quality of information, Understanding and reasoning, Expression style and persona, Safety and harm, and Trust and confidence) framework. Statistical comparisons between baseline and adapted models were performed using repeated-measures ANOVA.
Results
Three separate datasets for the assistant, vignette, and evaluator agents were created, which collectively formed a multi-agent artificial intelligence system for Chinese ACP training. The assistant dataset included 4364 dialogues, the vignette dataset comprised 671 clinical scenarios, and the evaluator dataset contained 671 records. Both automatic and human evaluations confirmed that the adapted models significantly outperformed baseline models on most aspects of Chinese ACP conversations and summarization (η 2 p =0.12‐0.99; P values ranged from .03 to
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Xiangya School of Nursing, Central South University, 172 Tongzipo Road, Changhsa, Hunan, 410013, China, 86 18711000745.
- School of Nursing, Ningxia Medical University, Yinchuan, Ningxia, China.
- School of Computer Science and Engineering, Central South University, Changsha, Hunan, China.
- New York University Shanghai, Shanghai, Shanghai, China.
- School of Nursing, Yale University, New Haven, CT, United States.
Cite
APA
Tan, M., Tang, S., Kan, S., Wu, B., Ni, Z., Zhang, H., & Ding, J. (2026). A Large Language Model–Driven System for Advance Care Planning Training Among Health Care Providers in the Chinese Context: Development and Technical Evaluation. Journal of Medical Internet Research, 28, e87288-e87288. https://doi.org/10.2196/87288
Vancouver
Tan M, Tang S, Kan S, Wu B, Ni Z, Zhang H, et al. A Large Language Model–Driven System for Advance Care Planning Training Among Health Care Providers in the Chinese Context: Development and Technical Evaluation. Journal of Medical Internet Research. 2026;28:e87288-e87288. doi:10.2196/87288
BibTeX
@article{Tan2026,
author = {Tan, Minghui and Tang, Siyuan and Kan, Shichao and Wu, Bei and Ni, Zhao and Zhang, Haojie and Ding, Jinfeng},
title = {A Large Language Model–Driven System for Advance Care Planning Training Among Health Care Providers in the Chinese Context: Development and Technical Evaluation},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e87288--e87288},
doi = {10.2196/87288}
}
EUVIMED