Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study
JMIR Research Protocols · 2026
DOI: 10.2196/84688 ↗Kurz gefasst
Studienprotokoll in der Zeitschrift JMIR Research Protocols, 2026. Verfasst von Misk Al Zahidy und 10 weiteren Autor:innen. Beteiligte Einrichtung: Care and AI Laboratory, Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, United States. Hauptthemen laut MEDLINE: Artificial Intelligence, Communication, Physician-Patient Relations. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 21 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
The promise of artificial intelligence (AI) in medicine depends on its ability to learn from data that reflect what matters to patients and clinicians in the care process. Most existing models are trained on electronic health records (EHRs), which primarily capture biological measures but rarely the interactions and relationships between patients and clinicians. These relationships, central to how care is understood, negotiated, and delivered, unfold across multiple modalities, including voice, text, and video, yet remain largely absent from current datasets. As a result, AI systems trained solely on EHRs risk perpetuating a narrow biomedical view of medicine and overlooking the lived exchanges that define clinical encounters.
Objective
This study aims to design, implement, and evaluate the feasibility of a longitudinal, multimodal system for capturing patient-clinician encounters, linking 360° video or audio recordings with postvisit surveys and EHR data, to create a foundational dataset for downstream AI research.
Methods
This single-site study was conducted in an academic outpatient specialty clinic (Division of Endocrinology, Mayo Clinic, Rochester, Minnesota, United States). Adult patients attending in-person visits with participating clinicians were invited to enroll. Encounters were recorded using a 360° 2D monocular video camera and dual-channel audio. After each visit, patients completed a brief survey assessing relational empathy, satisfaction, visit pace, and treatment burden. Demographic and clinical data were extracted from the EHR. Feasibility was assessed using 5 prespecified end points: clinician consent, patient consent, recording success, survey completion, and data linkage across modalities.
Results
Recruitment began in January 2025. By August 2025, 35 of 36 (97%) eligible clinicians and 212 of 281 (75%) approached eligible patients had consented. Of the consented encounters, 162 (76%) resulted in a complete 360° video recording, and the postvisit surveys were completed for 204 of 212 (96%) consented encounters, reflecting 1 survey per encounter. Data collection is ongoing as of December 2025, and further analyses will be reported in subsequent publications.
Conclusions
This protocol describes a longitudinal multimodal encounter capture system that links 360° audio or video with postvisit surveys and EHR data. The study specifies operational definitions, workflows, feasibility end points, and governance procedures to support implementation and replication in other clinical settings.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Care and AI Laboratory, Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, United States.
- Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Department of Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN, 55902, United States, 1 507-206-5737.
- Derriford Hospital, University Hospitals Plymouth NHS Trust, Plymouth, United Kingdom.
ORCID-Profile: Misk Al Zahidy, Kerly Guevara Maldonado, Luis Vilatuna Andrango, Ana Cristina Proano, Ana Gabriela Claros, Maria Lizarazo Jimenez, Esteban Gomez-Alvarez, David Toro-Tobon, Victor M Montori, Oscar J Ponce-Ponte, Juan P Brito
Zitieren
APA
Al Zahidy, M., Guevara Maldonado, K., Vilatuna Andrango, L., Proano, A. C., Claros, A. G., Lizarazo Jimenez, M., Gomez-Alvarez, E., Toro-Tobon, D., Montori, V. M., Ponce-Ponte, O. J., & Brito, J. P. (2026). Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study. JMIR Research Protocols. https://doi.org/10.2196/84688
Vancouver
Al Zahidy M, Guevara Maldonado K, Vilatuna Andrango L, Proano AC, Claros AG, Lizarazo Jimenez M, et al. Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study. JMIR Research Protocols. 2026. doi:10.2196/84688
BibTeX
@article{AlZahidy2026,
author = {Al Zahidy, Misk and Guevara Maldonado, Kerly and Vilatuna Andrango, Luis and Proano, Ana Cristina and Claros, Ana Gabriela and Lizarazo Jimenez, Maria and Gomez-Alvarez, Esteban and Toro-Tobon, David and Montori, Victor M and Ponce-Ponte, Oscar J and Brito, Juan P},
title = {Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study},
journal = {JMIR Research Protocols},
year = {2026},
doi = {10.2196/84688}
}
EUVIMED