Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study
Journal of Medical Internet Research · 03.09.2026 · 28 · e87061-e87061
DOI: 10.2196/87061 ↗Kurz gefasst
Mixed-Methods-Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 03.09.2026. Verfasst von Maria Lizarazo Jimenez und 17 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Division of Endocrinology, Diabetes, Metabolism and Nutrition, Knowledge and Evaluation Research Unit, Care and AI Laboratory…. Hauptthemen laut MEDLINE: Patient-Centered Care, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Large language models (LLMs) are increasingly demonstrating the potential to reach human-level performance in generating clinical summaries from patient-clinician conversations. LLMs are usually evaluated against clinical summaries that focus mainly on patients’ biology and not on their biography (eg, preferences, values, wishes, and concerns). To achieve patient-centered care, artificial intelligence clinical summarization must incorporate patient-centered domains, implemented through patient-centered summaries (PCSs).
Objective
This study aimed to develop a framework to generate PCS that capture patients’ values, preferences, and wishes while ensuring clinical utility for clinicians, and assess if current open-source LLMs can achieve human-level performance in generating PCS.
Methods
We developed a 4-step mixed methods process to define and evaluate PCS. First, 2 patient and public involvement and engagement groups were convened in the United Kingdom (10 patients and 8 clinicians), who participated in semistructured interviews exploring what personal and contextual information should be included in clinical summaries and how it should be structured for clinical use. Second, findings were translated into an annotation guideline, which was used by 8 clinician annotators to generate gold standard PCS from 88 transcribed patient-clinician consultations about the management of atrial fibrillation. Third, 16 consultations were used to iteratively develop and refine a prompt aligned with the annotation guideline. Finally, 5 LLMs (Llama-3.2-3B [Meta AI], Llama-3.1-8B [Meta AI], Mistral-8B [Mistral AI], Gemma-3-4B [Google DeepMind], and Qwen3-8B [Alibaba]) generated summaries from 72 consultations using zero-shot and few-shot prompting, which were evaluated against gold standard PCS using ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation–Longest Common Subsequence) and BERTScore (Bidirectional Encoder Representations from Transformers Score) and assessed for correctness, completeness, conciseness, patient-centeredness, and fluency.
Results
Patients emphasized that summaries should include (1) lifestyle routines and daily functioning as indicators of independence or disruption; (2) the presence and role of social support systems, especially during crises; (3) recent life events or stressors, such as trauma, loss, or caregiving demands; and (4) care preferences, values, and communication styles that provide meaning or reflect autonomy. Clinicians sought summaries that included a concise functional baseline, psychosocial context, and emotional cues, preferably in a structured, clinically digestible format. In the 72 consultations (mean age 70, SD 11 y; 32/72, 44.4% female), the best zero-shot performance was observed with Mistral-8B (ROUGE-L 0.189) and Llama-3.1-8B (BERTScore 0.673). The best few-shot prompting was found with 3 examples using Llama-3.1-8B (ROUGE-L 0.206 and BERTScore 0.683).
Conclusions
The open-source LLMs we evaluated did not achieve human-level performance in generating PCSs. Without task-specific fine-tuning, current open-source LLMs cannot reach human-level performance in this task. Our framework serves as an innovative guideline for developing gold standard PCS for artificial intelligence clinical tasks.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medicine, Division of Endocrinology, Diabetes, Metabolism and Nutrition, Knowledge and Evaluation Research Unit, Care and AI Laboratory, Mayo Clinic, 100 2nd St SW, Rochester, MN, 55905, United States, 1 (…
- Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Mayo Clinic, Rochester, MN, United States.
- Centre for Health Technology, University of Plymouth, Plymouth, United Kingdom.
- Division of Clinical Trials and Biostatistics, Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, United States.
- Usher Institute, University of Edinburgh, Edinburgh, Scotland, United Kingdom.
- Derriford Hospital, University Hospitals Plymouth NHS Trust, Plymouth, England, United Kingdom.
Zitieren
APA
Lizarazo Jimenez, M., Claros, A. G., Green, K., Toro-Tobon, D., Larios, F., Asthana, S., Wenczenovicz, C., Guevara Maldonado, K., Vilatuna-Andrango, L., Proano, A. C., Bandi, S. S. S., Bagewadi, S., Branda, M. E., Al Zahidy, M., Luz, S., Lapata, M., Brito, J. P., & Ponce-Ponte, O. J. (2026). Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study. Journal of Medical Internet Research, 28, e87061-e87061. https://doi.org/10.2196/87061
Vancouver
Lizarazo Jimenez M, Claros AG, Green K, Toro-Tobon D, Larios F, Asthana S, et al. Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study. Journal of Medical Internet Research. 2026;28:e87061-e87061. doi:10.2196/87061
BibTeX
@article{LizarazoJimenez2026,
author = {Lizarazo Jimenez, Maria and Claros, Ana Gabriela and Green, Kieran and Toro-Tobon, David and Larios, Felipe and Asthana, Sheena and Wenczenovicz, Camila and Guevara Maldonado, Kerly and Vilatuna-Andrango, Luis and Proano, Ana Cristina and Bandi, Satya Sai Sri and Bagewadi, Shubhangi and Branda, Megan E and Al Zahidy, Misk and Luz, Saturnino and Lapata, Mirella and Brito, Juan P and Ponce-Ponte, Oscar J},
title = {Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e87061--e87061},
doi = {10.2196/87061}
}
EUVIMED