EUVIMEDDescoperă EUVIMED
Randomisierte kontrollierte StudieOpen access

Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study

Echtzeit-KI-Diagnose-Copilot in simulierten Hausarztkonsultationen: randomisierte Simulationsstudie

Ivan Cusacovich iD, Manuel Pinilla iD, Ruben Garcia Castro iD

JMIR Formative Research · 2026-09-22 · 10 · e104579-e104579

DOI: 10.2196/104579 ↗

Auf einen Blick

In 260 simulierten Sprechstunden war die Top-3-Diagnosegenauigkeit mit Echtzeit-KI höher als ohne externe Hilfen. Hinweise auf Überabhängigkeit waren möglich, aber statistisch nicht eindeutig.

Fragestellung

Verbessert ein sprachbasierter Echtzeit-KI-Diagnoseassistent die diagnostische Genauigkeit von Ärzt:innen in simulierten hausärztlichen Konsultationen und zeigt sich Überabhängigkeit?

Methode

Case-level randomisierte, verblindete Simulationsstudie in einer webbasierten virtuellen Hausarztpraxis; 13 Fachärzt:innen; 260 sprachbasierte Konsultationen; 3 verblindete Gutachter:innen.

Ergebnisse

Top-3-Genauigkeit: 62,3 % (95 %-KI 49,1 % bis 75,7 %) ohne Hilfe vs. 74,6 % (95 %-KI 63,7 % bis 85,4 %) mit KI; AOR 2,68 (95 %-KI 1,24 bis 6,12), P =,01. Top-1 war nicht eindeutig.

Schlussfolgerung der Autor:innen

Die Echtzeit-KI-Unterstützung war in dieser randomisierten Hochdifficultäts-Simulation mit höherer Top-3-Diagnosegenauigkeit verbunden. Die Autor:innen sehen weiteren Bedarf an Refinement und prospektiver Prüfung in klinischen Arbeitsabläufen.

Kernaussagen

  • Top-3-Genauigkeit höher mit KI-Unterstützung
  • AOR 2,68 (95 %-KI 1,24 bis 6,12)
  • Überabhängigkeit möglich, aber nicht statistisch gesichert

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Randomisierte kontrollierte Studie in JMIR Formative Research, published online on 2026-09-22. Written by Ivan Cusacovich and 2 co-authors. Institution: Internal Medicine Department, Hospital Clínico Universitario de Valladolid, Valladolid, Spain. Major MEDLINE topics: Artificial Intelligence, Referral and Consultation. Cited 1 times so far (Crossref); 33 references. The full text is freely available in PubMed Central.

Auf einen BlickRezumatSubjectsDate bibliograficeCiteRelated studies

Abstract (original language)

Background

Diagnostic errors in primary care contribute substantially to avoidable morbidity. Real-time, voice-based artificial intelligence (AI) diagnostic assistants may support diagnostic reasoning during clinical encounters, but formative evidence that can be used to assess both diagnostic benefit and overreliance risk before clinical deployment is needed from interactive settings.

Objective

This study aimed to conduct a formative, high-difficulty simulation stress test of whether a real-time, voice-based AI diagnostic assistant was associated with improved physician diagnostic accuracy and measurable overreliance during simulated primary care consultations.

Methods

We conducted a case-level randomized, adjudicator-blinded simulation study in a web-based virtual primary care clinic. Thirteen board-certified family and community medicine physicians managed 260 simulated voice-based consultations. Cases were assigned to either an AI-assisted condition or an unassisted, resource-restricted simulation condition without any external diagnostic aids. No real patients were enrolled, and no clinical care was delivered or modified. The primary outcome was Top-3 diagnostic accuracy, defined as at least one correct diagnosis among up to 3 submitted diagnoses assessed by 3 blinded adjudicators and analyzed using a frequentist binomial generalized linear mixed model fitted by maximum likelihood with random intercepts for physician and case.

Results

All 260 consultations were completed and analyzed. Adjusted Top-3 diagnostic accuracy was 62.3% (95% CI 49.1% to 75.7%) in the unassisted condition and 74.6% (95% CI 63.7% to 85.4%) in the AI-assisted condition (adjusted odds ratio [AOR] 2.68, 95% CI 1.24 to 6.12; χ 2 1 =6.4, P =.01). This corresponds to an adjusted absolute difference of +12.3 percentage points (95% CI +2.7 to +22.6) and a simulation-context number needed to treat equivalent of 8.1 (95% CI 4.4 to 37.2). The post hoc Top-2 analysis was supportive (AOR 2.59, 95% CI 1.23 to 5.77; χ 2 1 =6.3, P =.01). Top-1 point estimates favored AI assistance but were inconclusive: Adjusted accuracy was 47.8% in the unassisted condition versus 57% with AI assistance (AOR 1.83, 95% CI 0.93 to 3.80; χ 2 1 =3.1, P =.08). Leave-one-physician-out and virtual-patient simulator-exclusion sensitivity analyses were consistent with the primary finding. In risk scenarios with an incorrect operative AI suggestion, adjusted concordance was 38.8% in the unassisted condition and 55.8% in the AI-assisted condition, but the between-condition comparison was not statistically significant (AOR 2.08, 95% CI 0.89 to 5.19; χ 2 1 =2.9, P =.09). Mean consultation time increased by 10.7%, and physicians rated the system highly for usefulness and satisfaction.

Conclusions

In this randomized, high-difficulty simulation study, real-time, voice-based AI assistance was associated with higher physician Top-3 diagnostic accuracy than an unassisted, resource-restricted comparator. Some sensitivity analyses were supportive, while others were directionally favorable but inconclusive, and the safety analysis suggested possible overreliance without statistically conclusive difference between conditions. These findings support further refinement and prospective evaluation in real clinical workflows before conclusions are drawn about routine clinical effectiveness or implementation.

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceReferral and ConsultationHumansPrimary Health CareMaleFemaleAdultDiagnostic ErrorsComputer Simulation

Author keywords

artificial intelligenceautomation biasclinical decision supportdiagnostic accuracyformative evaluationhuman-AI interactionphysician performanceprimary caresimulation studyvirtual patients

Authors and institutions

  • Internal Medicine Department, Hospital Clínico Universitario de Valladolid, Valladolid, Spain.
  • Medsys AI, SL, Madrid, Madrid, Spain.
  • Department of Medical-Surgical Dermatology and Venereology, Hospital Universitario Fundación Jiménez Díaz, Avenida de los Reyes Católicos, 2, Madrid, Madrid, 28040, Spain, 34 915504800.

ORCID profiles: Ivan Cusacovich, Manuel Pinilla, Ruben Garcia Castro

Cite

APA

Cusacovich, I., Pinilla, M., & Garcia Castro, R. (2026). Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study. JMIR Formative Research, 10, e104579-e104579. https://doi.org/10.2196/104579

Vancouver

Cusacovich I, Pinilla M, Garcia Castro R. Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study. JMIR Formative Research. 2026;10:e104579-e104579. doi:10.2196/104579

BibTeX
@article{Cusacovich2026,
  author = {Cusacovich, Ivan and Pinilla, Manuel and Garcia Castro, Ruben},
  title = {Real-Time Artificial Intelligence Diagnostic Copilot in Simulated Primary Care Consultations: Randomized Simulation Study},
  journal = {JMIR Formative Research},
  year = {2026},
  volume = {10},
  pages = {e104579--e104579},
  doi = {10.2196/104579}
}
Start for free →