EUVIMEDEUVIMED entdecken
Mixed-Methods-StudieOpen Access

Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study

Maria Lizarazo Jimenez, Ana Gabriela Claros, Kieran Green, David Toro-Tobon, Felipe Larios, Sheena Asthana, Camila Wenczenovicz, Kerly Guevara Maldonado, Luis Vilatuna-Andrango, Ana Cristina Proano, Satya Sai Sri Bandi, Shubhangi Bagewadi und 6 weitere

Journal of Medical Internet Research · 03.09.2026 · 28 · e87061-e87061

DOI: 10.2196/87061 ↗

Kurz gefasst

Mixed-Methods-Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 03.09.2026. Verfasst von Maria Lizarazo Jimenez und 17 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medicine, Division of Endocrinology, Diabetes, Metabolism and Nutrition, Knowledge and Evaluation Research Unit, Care and AI Laboratory…. Hauptthemen laut MEDLINE: Patient-Centered Care, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly demonstrating the potential to reach human-level performance in generating clinical summaries from patient-clinician conversations. LLMs are usually evaluated against clinical summaries that focus mainly on patients’ biology and not on their biography (eg, preferences, values, wishes, and concerns). To achieve patient-centered care, artificial intelligence clinical summarization must incorporate patient-centered domains, implemented through patient-centered summaries (PCSs).

Objective

This study aimed to develop a framework to generate PCS that capture patients’ values, preferences, and wishes while ensuring clinical utility for clinicians, and assess if current open-source LLMs can achieve human-level performance in generating PCS.

Methods

We developed a 4-step mixed methods process to define and evaluate PCS. First, 2 patient and public involvement and engagement groups were convened in the United Kingdom (10 patients and 8 clinicians), who participated in semistructured interviews exploring what personal and contextual information should be included in clinical summaries and how it should be structured for clinical use. Second, findings were translated into an annotation guideline, which was used by 8 clinician annotators to generate gold standard PCS from 88 transcribed patient-clinician consultations about the management of atrial fibrillation. Third, 16 consultations were used to iteratively develop and refine a prompt aligned with the annotation guideline. Finally, 5 LLMs (Llama-3.2-3B [Meta AI], Llama-3.1-8B [Meta AI], Mistral-8B [Mistral AI], Gemma-3-4B [Google DeepMind], and Qwen3-8B [Alibaba]) generated summaries from 72 consultations using zero-shot and few-shot prompting, which were evaluated against gold standard PCS using ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation–Longest Common Subsequence) and BERTScore (Bidirectional Encoder Representations from Transformers Score) and assessed for correctness, completeness, conciseness, patient-centeredness, and fluency.

Results

Patients emphasized that summaries should include (1) lifestyle routines and daily functioning as indicators of independence or disruption; (2) the presence and role of social support systems, especially during crises; (3) recent life events or stressors, such as trauma, loss, or caregiving demands; and (4) care preferences, values, and communication styles that provide meaning or reflect autonomy. Clinicians sought summaries that included a concise functional baseline, psychosocial context, and emotional cues, preferably in a structured, clinically digestible format. In the 72 consultations (mean age 70, SD 11 y; 32/72, 44.4% female), the best zero-shot performance was observed with Mistral-8B (ROUGE-L 0.189) and Llama-3.1-8B (BERTScore 0.673). The best few-shot prompting was found with 3 examples using Llama-3.1-8B (ROUGE-L 0.206 and BERTScore 0.683).

Conclusions

The open-source LLMs we evaluated did not achieve human-level performance in generating PCSs. Without task-specific fine-tuning, current open-source LLMs cannot reach human-level performance in this task. Our framework serves as an innovative guideline for developing gold standard PCS for artificial intelligence clinical tasks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Patient-Centered CareArtificial IntelligenceHumansLarge Language ModelsUnited Kingdom

Stichwörter der Autor:innen

LLMsartificial intelligencelarge language modelsmixed methodspatient-centeredqualitative research

Autor:innen und Institutionen

  • Department of Medicine, Division of Endocrinology, Diabetes, Metabolism and Nutrition, Knowledge and Evaluation Research Unit, Care and AI Laboratory, Mayo Clinic, 100 2nd St SW, Rochester, MN, 55905, United States, 1 (…
  • Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Mayo Clinic, Rochester, MN, United States.
  • Centre for Health Technology, University of Plymouth, Plymouth, United Kingdom.
  • Division of Clinical Trials and Biostatistics, Department of Quantitative Health Sciences, Mayo Clinic, Rochester, MN, United States.
  • Usher Institute, University of Edinburgh, Edinburgh, Scotland, United Kingdom.
  • Derriford Hospital, University Hospitals Plymouth NHS Trust, Plymouth, England, United Kingdom.

Zitieren

APA

Lizarazo Jimenez, M., Claros, A. G., Green, K., Toro-Tobon, D., Larios, F., Asthana, S., Wenczenovicz, C., Guevara Maldonado, K., Vilatuna-Andrango, L., Proano, A. C., Bandi, S. S. S., Bagewadi, S., Branda, M. E., Al Zahidy, M., Luz, S., Lapata, M., Brito, J. P., & Ponce-Ponte, O. J. (2026). Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study. Journal of Medical Internet Research, 28, e87061-e87061. https://doi.org/10.2196/87061

Vancouver

Lizarazo Jimenez M, Claros AG, Green K, Toro-Tobon D, Larios F, Asthana S, et al. Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study. Journal of Medical Internet Research. 2026;28:e87061-e87061. doi:10.2196/87061

BibTeX
@article{LizarazoJimenez2026,
  author = {Lizarazo Jimenez, Maria and Claros, Ana Gabriela and Green, Kieran and Toro-Tobon, David and Larios, Felipe and Asthana, Sheena and Wenczenovicz, Camila and Guevara Maldonado, Kerly and Vilatuna-Andrango, Luis and Proano, Ana Cristina and Bandi, Satya Sai Sri and Bagewadi, Shubhangi and Branda, Megan E and Al Zahidy, Misk and Luz, Saturnino and Lapata, Mirella and Brito, Juan P and Ponce-Ponte, Oscar J},
  title = {Patient-Centered Summarization Framework for AI Clinical Summarization: Mixed Methods Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e87061--e87061},
  doi = {10.2196/87061}
}
Kostenlos starten →