EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Three Ambient Listening Systems in Simulated Dutch-Language Multispeaker and Multidialect Clinical Encounters: Comparative Study

Anne H Hoekman iD, Tarannom Mehri iD, Job N Doornberg iD, Mirjam Plantinga iD, Tom P van der Laan iD, Charlotte M H H T Bootsma-Robroeks iD, Rosanne C Schoonbeek iD

JMIR Medical Informatics · 2026

DOI: 10.2196/94586 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Informatics, 2026. Verfasst von Anne H Hoekman und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Information Office, Information Management Healthcare, University Medical Center Groningen, Groningen, The Netherlands. Hauptthemen laut MEDLINE: Physician-Patient Relations, Electronic Health Records, Documentation. Die Arbeit nennt 22 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Clinicians spend a lot of time on electronic health record (EHR) documentation, often at the expense of direct patient interaction. Ambient listening technology uses AI to passively record and summarize clinical encounters. Although early studies are promising, there is limited evidence on system performance in complex, non-English settings.

Objective

This study aimed to compare the documentation performance of 3 commercially available ambient listening systems in simulated Dutch-language outpatient consultations by assessing note completeness, correctness, and conciseness under predefined linguistic and interactional challenges.

Methods

Standardized audio recordings of 10 scripted physician-patient interactions across 2 specialties were used. Scenarios included multispeaker dynamics (patient companion), conversational disruptions (nurse interruption), evasive patient communication, and a regional dialect (Gronings). Three ambient listening providers (provider 1, provider 2, and provider 3) processed the audio files. Generated notes were divided into 4 note sections: patient history, physical examination, diagnostic impression, and treatment plan. For each system output, each note section was scored for completeness, conciseness, and correctness using a 5-point ordinal scale against physician-developed reference summaries. In total, 8 clinicians participated in scoring: 4 junior clinicians evaluated all outputs, and 4 senior clinicians each reviewed a subset of scenarios. Analyses were based on 351 rater-averaged section-domain scores. Interrater agreement was assessed using Gwet agreement coefficient 2 (Gwet AC-2). System-level technical characteristics were assessed alongside clinical performance to aid interpretation of between-provider differences.

Results

Across 351 rater-averaged section-domain scores on a 5-point scale, overall interrater agreement was high (Gwet AC-2=0.827). Provider-level mean scores ranged from 3.82 to 4.26 (4.26, SD 0.73 for provider 3, 4.00, SD 0.90 for provider 2, and 3.82, SD 1.07 for provider 1). Mean scores were higher in otolaryngology than in surgical oncology (4.36, SD 0.56 vs 3.68, SD 1.10). Across scoring domains, correctness received the highest mean score (4.21, SD 0.98), while completeness received the lowest (3.81, SD 0.90). Variation in mean scores was observed across script scenarios. Dialect-containing scenarios had the lowest mean score overall (3.77, SD 1.21), although this pattern was not uniform across specialties and systems. Median summary generation times ranged from 13.5 seconds (IQR 12-15; provider 3) to 22.0 seconds (IQR 18-24; provider 2).

Conclusions

Ambient listening systems demonstrated generally strong performance in this simulated Dutch-language evaluation, including under several common conversational challenges. However, mean scores varied across scripted scenarios, with lower scores observed in some dialect-containing and otherwise challenging interactions. Future deployment studies must prioritize linguistic equity, real-world validation of efficiency gains, and evaluation of both clinician and patient perceptions to understand how these systems influence consultation dynamics and care delivery across diverse patient populations.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Physician-Patient RelationsElectronic Health RecordsDocumentationHumansNetherlandsCommunicationLanguageMale

Stichwörter der Autor:innen

AI adoptionambient listeningartificial intelligence adoptionelectronic health recordsgenerative AIgenerative artificial intelligencelarge language models

Autor:innen und Institutionen

  • Department of Health Information Office, Information Management Healthcare, University Medical Center Groningen, Groningen, The Netherlands.
  • Department of Trauma Surgery / Orthopedics, University Medical Center Groningen, Groningen, The Netherlands.
  • Department of Orthopaedic Trauma, Flinders University, Adelaide, Australia.
  • ELSA AI Lab Northern Netherlands, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.
  • Department of Genetics and Data Science Center in Health, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.
  • Department of Otolaryngology - Head and Neck Surgery, University Medical Center Groningen, Groningen, The Netherlands.

ORCID-Profile: Anne H Hoekman, Tarannom Mehri, Job N Doornberg, Mirjam Plantinga, Tom P van der Laan, Charlotte M H H T Bootsma-Robroeks, Rosanne C Schoonbeek

Zitieren

APA

Hoekman, A. H., Mehri, T., Doornberg, J. N., Plantinga, M., van der Laan, T. P., Bootsma-Robroeks, C. M. H. H. T., & Schoonbeek, R. C. (2026). Three Ambient Listening Systems in Simulated Dutch-Language Multispeaker and Multidialect Clinical Encounters: Comparative Study. JMIR Medical Informatics. https://doi.org/10.2196/94586

Vancouver

Hoekman AH, Mehri T, Doornberg JN, Plantinga M, van der Laan TP, Bootsma-Robroeks CMHHT, et al. Three Ambient Listening Systems in Simulated Dutch-Language Multispeaker and Multidialect Clinical Encounters: Comparative Study. JMIR Medical Informatics. 2026. doi:10.2196/94586

BibTeX
@article{Hoekman2026,
  author = {Hoekman, Anne H and Mehri, Tarannom and Doornberg, Job N and Plantinga, Mirjam and van der Laan, Tom P and Bootsma-Robroeks, Charlotte M H H T and Schoonbeek, Rosanne C},
  title = {Three Ambient Listening Systems in Simulated Dutch-Language Multispeaker and Multidialect Clinical Encounters: Comparative Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  doi = {10.2196/94586}
}
Kostenlos starten →