Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care
JMIR Formative Research · 2026
DOI: 10.2196/88905 ↗Kurz gefasst
Multizentrische Studie in der Zeitschrift JMIR Formative Research, 2026. Verfasst von Chris Jacobs und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, University of Bath, Bath, England, United Kingdom. Hauptthemen laut MEDLINE: Artificial Intelligence, Education, Medical. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Conversational artificial intelligence (AI) systems offer potential solutions to traditional constraints in medical consultation skills training, including high costs, scheduling difficulties, and varied standardization. There is limited evidence evaluating medical professionals’ perceptions of AI-generated patient interactions across multiple fidelity dimensions and assessing the educational value of conversational AI for consultation skills training.
Objective
This study aimed to evaluate perceptions of conversational AI patient simulations in primary care consultation training, examining functional fidelity, conversational realism, educational value, and implementation readiness.
Methods
A cross-sectional evaluation study at a UK medical school (medical students and general practitioners) yielded 47 grouped and individual responses. Participants completed standardized clinical scenarios using the SimFlow conversational AI system, a conversational AI system, followed by a multidomain questionnaire evaluating AI realism, medical content, educational value, feedback, and usability. Data were analyzed using the Wilcoxon signed rank test, Spearman correlation, and Firth logistic regression to assess domain performance and participant characteristics.
Results
Medical content received the highest ratings (median 4.5, IQR 4.0-5.0), with 97.8% (45/46) rating clinical plausibility highly. Educational value was rated positively (median 4.0, IQR 3.0-4.0), although AI realism received moderate scores (median 3.0, IQR 2.0-4.0). Participants with prior AI experience gave significantly higher ratings for AI realism than those without prior experience (mean 3.81, SD 0.63 vs 3.07, SD 0.72; P=.03). Concordance analysis demonstrated moderate-to-strong agreement between individual- and group-level domain rankings (mean Spearman ρ=0.685), supporting consistency between collaborative and individual survey evaluations. Qualitative analysis revealed 4 themes: clinical authenticity, interactional limitations, educational potential, and implementation considerations.
Conclusions
Conversational AI demonstrates strong capabilities in functional fidelity (clinical accuracy) despite limitations in conversational fidelity (realism). The technology shows promise as a supplementary tool for clinical skills training rather than higher-stakes assessment, with future development needed in dialogue naturalness and feedback capabilities.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Psychology, University of Bath, Bath, England, United Kingdom.
- Faculty of Life Sciences & Medicine, King's College London, London, England, United Kingdom.
- Medical School, University of Bristol, Bristol, England, United Kingdom.
ORCID-Profile: Chris Jacobs, Hans Johnson, Kirsty Brownlie, Richard Joiner, Trevor Thompson
Zitieren
APA
Jacobs, C., Johnson, H., Brownlie, K., Joiner, R., & Thompson, T. (2026). Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care. JMIR Formative Research. https://doi.org/10.2196/88905
Vancouver
Jacobs C, Johnson H, Brownlie K, Joiner R, Thompson T. Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care. JMIR Formative Research. 2026. doi:10.2196/88905
BibTeX
@article{Jacobs2026,
author = {Jacobs, Chris and Johnson, Hans and Brownlie, Kirsty and Joiner, Richard and Thompson, Trevor},
title = {Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care},
journal = {JMIR Formative Research},
year = {2026},
doi = {10.2196/88905}
}
EUVIMED