EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group

Marco Montagna iD, Filippo Chiabrando iD, Rebecca De Lorenzo iD, Patrizia Rovere Querini iD, Medical Students

JMIR Medical Education · 18.03.2025 · 11 · e55709-e55709

DOI: 10.2196/55709 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Education, online erschienen am 18.03.2025. Verfasst von Marco Montagna und 4 weiteren Autor:innen. Beteiligte Einrichtung: School of Medicine, Vita-Salute San Raffaele University, Via Olgettina 58, Milan, 20132, Italy. Hauptthemen laut MEDLINE: Students, Medical, Decision Support Systems, Clinical, Clinical Competence, Problem Solving. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 20 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Health care practitioners use clinical decision support systems (CDSS) as an aid in the crucial task of clinical reasoning and decision-making. Traditional CDSS are online repositories (ORs) and clinical practice guidelines (CPG). Recently, large language models (LLMs) such as ChatGPT have emerged as potential alternatives. They have proven to be powerful, innovative tools, yet they are not devoid of worrisome risks.

Objective

This study aims to explore how medical students perform in an evaluated clinical case through the use of different CDSS tools.

Methods

The authors randomly divided medical students into 3 groups, CPG, n=6 (38%); OR, n=5 (31%); and ChatGPT, n=5 (31%); and assigned each group a different type of CDSS for guidance in answering prespecified questions, assessing how students’ speed and ability at resolving the same clinical case varied accordingly. External reviewers evaluated all answers based on accuracy and completeness metrics (score: 1‐5). The authors analyzed and categorized group scores according to the skill investigated: differential diagnosis, diagnostic workup, and clinical decision-making.

Results

Answering time showed a trend for the ChatGPT group to be the fastest. The mean scores for completeness were as follows: CPG 4.0, OR 3.7, and ChatGPT 3.8 ( P =.49). The mean scores for accuracy were as follows: CPG 4.0, OR 3.3, and ChatGPT 3.7 ( P =.02). Aggregating scores according to the 3 students’ skill domains, trends in differences among the groups emerge more clearly, with the CPG group that performed best in nearly all domains and maintained almost perfect alignment between its completeness and accuracy.

Conclusions

This hands-on session provided valuable insights into the potential perks and associated pitfalls of LLMs in medical education and practice. It suggested the critical need to include teachings in medical degree courses on how to properly take advantage of LLMs, as the potential for misuse is evident and real.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Students, MedicalDecision Support Systems, ClinicalClinical CompetenceProblem SolvingClinical ReasoningHumansFemaleMaleClinical Decision-MakingEducation, Medical, Undergraduate

Stichwörter der Autor:innen

AICDSSLLMsMLalgorithmartificial intelligencechatGPTchatbotclinical decision support systemsclinical decision-makingdecision supportdecision support tooldeep learninginnovative teachinglarge language models

Autor:innen und Institutionen

  • School of Medicine, Vita-Salute San Raffaele University, Via Olgettina 58, Milan, 20132, Italy.
  • Unit of Medical Specialties and Healthcare Continuity, IRCCS San Raffaele Scientific Institute, Milan, Italy.

ORCID-Profile: Marco Montagna, Filippo Chiabrando, Rebecca De Lorenzo, Patrizia Rovere Querini

Zitieren

APA

Montagna, M., Chiabrando, F., De Lorenzo, R., Rovere Querini, P., & Medical Students (2025). Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group. JMIR Medical Education, 11, e55709-e55709. https://doi.org/10.2196/55709

Vancouver

Montagna M, Chiabrando F, De Lorenzo R, Rovere Querini P, Medical Students. Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group. JMIR Medical Education. 2025;11:e55709-e55709. doi:10.2196/55709

BibTeX
@article{Montagna2025,
  author = {Montagna, Marco and Chiabrando, Filippo and De Lorenzo, Rebecca and Rovere Querini, Patrizia and Medical Students, Medical Students},
  title = {Impact of Clinical Decision Support Systems on Medical Students’ Case-Solving Performance: Comparison Study with a Focus Group},
  journal = {JMIR Medical Education},
  year = {2025},
  volume = {11},
  pages = {e55709--e55709},
  doi = {10.2196/55709}
}
Kostenlos starten →