EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Observer: creation of a novel multimodal dataset for outpatient care research

Kevin B Johnson, Basam Alasaly iD, Kuk Jin Jang, Eric Eaton, Sriharsha Mopidevi, Ross Koppel iD

Journal of the American Medical Informatics Association · 2026

DOI: 10.1093/jamia/ocaf182 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2026. Verfasst von Kevin B Johnson und 5 weiteren Autor:innen. Beteiligte Einrichtung: Division of Biomedical Informatics, Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine at the University of Penn…. Hauptthemen laut MEDLINE: Electronic Health Records, Ambulatory Care, Datasets as Topic. Die Arbeit nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NLM NIH HHS, NIH HHS, National Library of Medicine and the NIH Office of the Director.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To support ambulatory care innovation, we created Observer, a multimodal dataset comprising videotaped outpatient visits, electronic health record (EHR) data, and structured surveys. This paper describes the data collection procedures and summarizes the clinical and contextual features of the dataset. Materials and Methods A multistakeholder steering group shaped recruitment strategies, survey design, and privacy-preserving design. Consented patients and primary care providers (PCPs) were recorded using room-view and egocentric cameras. EHR data, metadata, and audit logs were also captured. A custom de-identification pipeline, combining transcript redaction, voice masking, and facial blurring, ensured video and EHR HIPAA compliance.

Results

We report on the first 100 visits in this continually growing dataset. Thirteen PCPs from 4 clinics participated. Recording the first 100 visits required approaching 210 patients, from which 129 consented (61%), with 29 patients missing their scheduled encounter after consenting. Visit lengths ranged from 5 to 100 minutes, covering preventive care to chronic disease management. Survey responses revealed high satisfaction: 4.24/5 (patients) and 3.94/5 (PCPs). Visit experience was unaffected by the presence of video recording technology.

Discussion

We demonstrate the feasibility of capturing rich, real-world primary care interactions using scalable, privacy-sensitive methods. Room layout and camera placement were key influences on recorded communication and are now added to the dataset. The Observer dataset enables future clinical AI research/development, communication studies, and informatics education among public and private user groups.

Conclusion

Observer is a new, shareable, real-world clinic encounter research and teaching resource with a representative sample of adult primary care data.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsAmbulatory CareDatasets as TopicHumansPrimary Health CareData CollectionVideo RecordingFemaleMaleVideotape RecordingAdultSurveys and QuestionnairesMiddle Aged

Stichwörter der Autor:innen

ambulatory careartificial intelligencedata curationprimary health carevideo recording

Autor:innen und Institutionen

  • Division of Biomedical Informatics, Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, 19104, United States.
  • Department of Computer and Information Science, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, 19104, United States.
  • School of Arts and Sciences, University of Pennsylvania, Philadelphia, PA, 19104, United States.

ORCID-Profile: Basam Alasaly, Ross Koppel

Förderung

  • NLM NIH HHS
  • NIH HHS
  • National Library of Medicine and the NIH Office of the Director
  • Helping Doctors Doctor: Using AI to Automate Documentation and 'De-Autonomate' Health Care
  • National Library of Medicine
  • NIH
  • Helping Doctors Doctor: Using AI to Automate Documentation and ‘De-Autonomate’ Health Care

Zitieren

APA

Johnson, K. B., Alasaly, B., Jang, K. J., Eaton, E., Mopidevi, S., & Koppel, R. (2026). Observer: creation of a novel multimodal dataset for outpatient care research. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocaf182

Vancouver

Johnson KB, Alasaly B, Jang KJ, Eaton E, Mopidevi S, Koppel R. Observer: creation of a novel multimodal dataset for outpatient care research. Journal of the American Medical Informatics Association. 2026. doi:10.1093/jamia/ocaf182

BibTeX
@article{Johnson2026,
  author = {Johnson, Kevin B and Alasaly, Basam and Jang, Kuk Jin and Eaton, Eric and Mopidevi, Sriharsha and Koppel, Ross},
  title = {Observer: creation of a novel multimodal dataset for outpatient care research},
  journal = {Journal of the American Medical Informatics Association},
  year = {2026},
  doi = {10.1093/jamia/ocaf182}
}
Kostenlos starten →