EUVIMEDTutustu EUVIMEDiin
Multizentrische StudieQuerschnittstudieOpen access

Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care

Chris Jacobs iD, Hans Johnson iD, Kirsty Brownlie iD, Richard Joiner iD, Trevor Thompson iD

JMIR Formative Research · 2026

DOI: 10.2196/88905 ↗

At a glance

Multizentrische Studie in JMIR Formative Research, 2026. Written by Chris Jacobs and 4 co-authors. Institution: Department of Psychology, University of Bath, Bath, England, United Kingdom. Major MEDLINE topics: Artificial Intelligence, Education, Medical. Cited 3 times so far (Crossref); 45 references. The full text is freely available in PubMed Central.

TiivistelmäSubjectsBibliografiset tiedotCiteRelated studies

Abstract (original language)

Background

Conversational artificial intelligence (AI) systems offer potential solutions to traditional constraints in medical consultation skills training, including high costs, scheduling difficulties, and varied standardization. There is limited evidence evaluating medical professionals’ perceptions of AI-generated patient interactions across multiple fidelity dimensions and assessing the educational value of conversational AI for consultation skills training.

Objective

This study aimed to evaluate perceptions of conversational AI patient simulations in primary care consultation training, examining functional fidelity, conversational realism, educational value, and implementation readiness.

Methods

A cross-sectional evaluation study at a UK medical school (medical students and general practitioners) yielded 47 grouped and individual responses. Participants completed standardized clinical scenarios using the SimFlow conversational AI system, a conversational AI system, followed by a multidomain questionnaire evaluating AI realism, medical content, educational value, feedback, and usability. Data were analyzed using the Wilcoxon signed rank test, Spearman correlation, and Firth logistic regression to assess domain performance and participant characteristics.

Results

Medical content received the highest ratings (median 4.5, IQR 4.0-5.0), with 97.8% (45/46) rating clinical plausibility highly. Educational value was rated positively (median 4.0, IQR 3.0-4.0), although AI realism received moderate scores (median 3.0, IQR 2.0-4.0). Participants with prior AI experience gave significantly higher ratings for AI realism than those without prior experience (mean 3.81, SD 0.63 vs 3.07, SD 0.72; P=.03). Concordance analysis demonstrated moderate-to-strong agreement between individual- and group-level domain rankings (mean Spearman ρ=0.685), supporting consistency between collaborative and individual survey evaluations. Qualitative analysis revealed 4 themes: clinical authenticity, interactional limitations, educational potential, and implementation considerations.

Conclusions

Conversational AI demonstrates strong capabilities in functional fidelity (clinical accuracy) despite limitations in conversational fidelity (realism). The technology shows promise as a supplementary tool for clinical skills training rather than higher-stakes assessment, with future development needed in dialogue naturalness and feedback capabilities.

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceEducation, MedicalCross-Sectional StudiesHumansPrimary Health CareMaleFemaleSurveys and QuestionnairesUnited KingdomAdultClinical CompetenceStudents, Medical

Author keywords

AI: medicalartificial intelligenceclinicalconversationaleducationrealism

Authors and institutions

  • Department of Psychology, University of Bath, Bath, England, United Kingdom.
  • Faculty of Life Sciences & Medicine, King's College London, London, England, United Kingdom.
  • Medical School, University of Bristol, Bristol, England, United Kingdom.

ORCID profiles: Chris Jacobs, Hans Johnson, Kirsty Brownlie, Richard Joiner, Trevor Thompson

Cite

APA

Jacobs, C., Johnson, H., Brownlie, K., Joiner, R., & Thompson, T. (2026). Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care. JMIR Formative Research. https://doi.org/10.2196/88905

Vancouver

Jacobs C, Johnson H, Brownlie K, Joiner R, Thompson T. Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care. JMIR Formative Research. 2026. doi:10.2196/88905

BibTeX
@article{Jacobs2026,
  author = {Jacobs, Chris and Johnson, Hans and Brownlie, Kirsty and Joiner, Richard and Thompson, Trevor},
  title = {Scaling Multimodal Agentic AI in Medical Education: Multisite Cross-Sectional Study of Simulation Effectiveness in Primary Care},
  journal = {JMIR Formative Research},
  year = {2026},
  doi = {10.2196/88905}
}
Start for free →