EUVIMEDEUVIMED entdecken
StudienprotokollOpen Access

Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study

Misk Al Zahidy iD, Kerly Guevara Maldonado iD, Luis Vilatuna Andrango iD, Ana Cristina Proano iD, Ana Gabriela Claros iD, Maria Lizarazo Jimenez iD, Esteban Gomez-Alvarez iD, David Toro-Tobon iD, Victor M Montori iD, Oscar J Ponce-Ponte iD, Juan P Brito iD

JMIR Research Protocols · 2026

DOI: 10.2196/84688 ↗

Kurz gefasst

Studienprotokoll in der Zeitschrift JMIR Research Protocols, 2026. Verfasst von Misk Al Zahidy und 10 weiteren Autor:innen. Beteiligte Einrichtung: Care and AI Laboratory, Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, United States. Hauptthemen laut MEDLINE: Artificial Intelligence, Communication, Physician-Patient Relations. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 21 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The promise of artificial intelligence (AI) in medicine depends on its ability to learn from data that reflect what matters to patients and clinicians in the care process. Most existing models are trained on electronic health records (EHRs), which primarily capture biological measures but rarely the interactions and relationships between patients and clinicians. These relationships, central to how care is understood, negotiated, and delivered, unfold across multiple modalities, including voice, text, and video, yet remain largely absent from current datasets. As a result, AI systems trained solely on EHRs risk perpetuating a narrow biomedical view of medicine and overlooking the lived exchanges that define clinical encounters.

Objective

This study aims to design, implement, and evaluate the feasibility of a longitudinal, multimodal system for capturing patient-clinician encounters, linking 360° video or audio recordings with postvisit surveys and EHR data, to create a foundational dataset for downstream AI research.

Methods

This single-site study was conducted in an academic outpatient specialty clinic (Division of Endocrinology, Mayo Clinic, Rochester, Minnesota, United States). Adult patients attending in-person visits with participating clinicians were invited to enroll. Encounters were recorded using a 360° 2D monocular video camera and dual-channel audio. After each visit, patients completed a brief survey assessing relational empathy, satisfaction, visit pace, and treatment burden. Demographic and clinical data were extracted from the EHR. Feasibility was assessed using 5 prespecified end points: clinician consent, patient consent, recording success, survey completion, and data linkage across modalities.

Results

Recruitment began in January 2025. By August 2025, 35 of 36 (97%) eligible clinicians and 212 of 281 (75%) approached eligible patients had consented. Of the consented encounters, 162 (76%) resulted in a complete 360° video recording, and the postvisit surveys were completed for 204 of 212 (96%) consented encounters, reflecting 1 survey per encounter. Data collection is ongoing as of December 2025, and further analyses will be reported in subsequent publications.

Conclusions

This protocol describes a longitudinal multimodal encounter capture system that links 360° audio or video with postvisit surveys and EHR data. The study specifies operational definitions, workflows, feasibility end points, and governance procedures to support implementation and replication in other clinical settings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceCommunicationPhysician-Patient RelationsAdultHumansElectronic Health RecordsFeasibility StudiesLongitudinal StudiesObservational Studies as TopicVideo Recording

Stichwörter der Autor:innen

AIartificial intelligencedigital healthelectronic health recordsendocrinologyfeasibility studiesmultimodal data capturepatient-clinician communication

Autor:innen und Institutionen

  • Care and AI Laboratory, Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, United States.
  • Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Department of Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN, 55902, United States, 1 507-206-5737.
  • Derriford Hospital, University Hospitals Plymouth NHS Trust, Plymouth, United Kingdom.

ORCID-Profile: Misk Al Zahidy, Kerly Guevara Maldonado, Luis Vilatuna Andrango, Ana Cristina Proano, Ana Gabriela Claros, Maria Lizarazo Jimenez, Esteban Gomez-Alvarez, David Toro-Tobon, Victor M Montori, Oscar J Ponce-Ponte, Juan P Brito

Zitieren

APA

Al Zahidy, M., Guevara Maldonado, K., Vilatuna Andrango, L., Proano, A. C., Claros, A. G., Lizarazo Jimenez, M., Gomez-Alvarez, E., Toro-Tobon, D., Montori, V. M., Ponce-Ponte, O. J., & Brito, J. P. (2026). Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study. JMIR Research Protocols. https://doi.org/10.2196/84688

Vancouver

Al Zahidy M, Guevara Maldonado K, Vilatuna Andrango L, Proano AC, Claros AG, Lizarazo Jimenez M, et al. Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study. JMIR Research Protocols. 2026. doi:10.2196/84688

BibTeX
@article{AlZahidy2026,
  author = {Al Zahidy, Misk and Guevara Maldonado, Kerly and Vilatuna Andrango, Luis and Proano, Ana Cristina and Claros, Ana Gabriela and Lizarazo Jimenez, Maria and Gomez-Alvarez, Esteban and Toro-Tobon, David and Montori, Victor M and Ponce-Ponte, Oscar J and Brito, Juan P},
  title = {Longitudinal and Multimodal Recording System to Capture Real-World Patient-Clinician Conversations for AI and Encounter Research: Protocol for an Observational Study},
  journal = {JMIR Research Protocols},
  year = {2026},
  doi = {10.2196/84688}
}
Kostenlos starten →