EUVIMEDOpdag EUVIMED
BeobachtungsstudieKohortenstudie

Association of Preexamination AI Virtual-Patient Trainer Scores With Scores on a Local Single-Station History-Taking Objective Structured Clinical Examination: Observational Cohort Study

Weishan Zhang iD, Yang Liu iD, Yiying Zhu iD, Xian Lu iD, Liping Wu iD, Minghui Yue iD, Xiaolin Hong iD, Oudong Xia iD, Chujun Shi iD

JMIR Medical Education · 2026-11-03 · 12 · e109752

DOI: 10.2196/109752 ↗

At a glance

Beobachtungsstudie in JMIR Medical Education, published online on 2026-11-03. Written by Weishan Zhang and 8 co-authors. Institution: Medical Simulation Center, Shantou University Medical College, Shantou, Guangdong, China. Major MEDLINE topics: Medical History Taking, Clinical Competence, Educational Measurement, Artificial Intelligence. 39 references.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Large language model–based virtual-patient systems score history-taking practice, but whether scores correspond to independently scored examination performance remains untested. Existing evaluations emphasize internal platform metrics or simulated-patient fidelity, leaving criterion-related evidence limited.

Objective

This study tests whether mean preexamination AI-trainer practice scores were associated with performance on 1 local objective structured clinical examination (OSCE) history-taking station, whether the association depended on the complaint-matched training case, and whether a model including the trainer score retained predictive performance in held-out students.

Methods

At 1 medical school, second-year students received a required 6-case assignment on the AI-Powered Medical History-Taking Training and Evaluation System (AMTES) and completed an end-of-course OSCE. Each student completed 1 station rated by 1 examiner; interrater and cross-station reliability could not be estimated. The 4 OSCE diagnoses differed from the training diagnoses: 3 shared a presenting-complaint category with an AMTES case, whereas 1 OSCE case had no presenting-complaint match. Platform-to-OSCE record linkage was 474 out of 480 (98.8%); 473 students with analyzable preexamination practice formed the analytic sample. The primary predictor was the mean of normalized scores across practice records time-stamped before each student’s examination-day cut-off. Analyses used Spearman correlations, OSCE-case fixed-effects regression, and repeated 5-fold cross-validation.

Results

OSCE scores (mean 81.5, SD 6.7) were associated with platform practice scores (Spearman ρ=0.388; P

Subjects

MeSH terms (MEDLINE)

Medical History TakingClinical CompetenceEducational MeasurementArtificial IntelligenceHumansCohort StudiesMaleEducation, Medical, UndergraduateStudents, Medical

Author keywords

artificial intelligencecriterion-related validityeducational measurementlarge language modelslearning analyticsmedical educationmedical history takingobjective structured clinical examinationvirtual patients

Authors and institutions

  • Medical Simulation Center, Shantou University Medical College, Shantou, Guangdong, China.
  • Department of Medical Physics and Informatics, Shantou University Medical College, Shantou, China.
  • Shantou University Medical College, Shantou, China.

ORCID profiles: Weishan Zhang, Yang Liu, Yiying Zhu, Xian Lu, Liping Wu, Minghui Yue, Xiaolin Hong, Oudong Xia, Chujun Shi

Cite

APA

Zhang, W., Liu, Y., Zhu, Y., Lu, X., Wu, L., Yue, M., Hong, X., Xia, O., & Shi, C. (2026). Association of Preexamination AI Virtual-Patient Trainer Scores With Scores on a Local Single-Station History-Taking Objective Structured Clinical Examination: Observational Cohort Study. JMIR Medical Education, 12, e109752. https://doi.org/10.2196/109752

Vancouver

Zhang W, Liu Y, Zhu Y, Lu X, Wu L, Yue M, et al. Association of Preexamination AI Virtual-Patient Trainer Scores With Scores on a Local Single-Station History-Taking Objective Structured Clinical Examination: Observational Cohort Study. JMIR Medical Education. 2026;12:e109752. doi:10.2196/109752

BibTeX
@article{Zhang2026,
  author = {Zhang, Weishan and Liu, Yang and Zhu, Yiying and Lu, Xian and Wu, Liping and Yue, Minghui and Hong, Xiaolin and Xia, Oudong and Shi, Chujun},
  title = {Association of Preexamination AI Virtual-Patient Trainer Scores With Scores on a Local Single-Station History-Taking Objective Structured Clinical Examination: Observational Cohort Study},
  journal = {JMIR Medical Education},
  year = {2026},
  volume = {12},
  pages = {e109752},
  doi = {10.2196/109752}
}
Start for free →