EUVIMEDAvasta EUVIMED
Open access

Health Care Trainees’ and Professionals’ Perceptions of ChatGPT in Improving Medical Knowledge Training: Rapid Survey Study

Je-Ming Hu iD, Feng-Cheng Liu iD, Chi-Ming Chu iD, Yu-Tien Chang iD

Journal of Medical Internet Research · 2023

DOI: 10.2196/49385 ↗

At a glance

Journal article in Journal of Medical Internet Research, 2023. Written by Je-Ming Hu and 3 co-authors. Institution: Division of Colorectal Surgery, Department of Surgery, Tri-service General Hospital, National Defense Medical Center, Taipei, Taiwan. Major MEDLINE topics: Attitude of Health Personnel, Students, Artificial Intelligence. Cited 49 times so far (Crossref); 39 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliograafilised andmedCiteRelated studies

Abstract (original language)

Background

ChatGPT is a powerful pretrained large language model. It has both demonstrated potential and raised concerns related to knowledge translation and knowledge transfer. To apply and improve knowledge transfer in the real world, it is essential to assess the perceptions and acceptance of the users of ChatGPT-assisted training.

Objective

We aimed to investigate the perceptions of health care trainees and professionals on ChatGPT-assisted training, using biomedical informatics as an example.

Methods

We used purposeful sampling to include all health care undergraduate trainees and graduate professionals (n=195) from January to May 2023 in the School of Public Health at the National Defense Medical Center in Taiwan. Subjects were asked to watch a 2-minute video introducing 5 scenarios about ChatGPT-assisted training in biomedical informatics and then answer a self-designed online (web- and mobile-based) questionnaire according to the Kirkpatrick model. The survey responses were used to develop 4 constructs: “perceived knowledge acquisition,” “perceived training motivation,” “perceived training satisfaction,” and “perceived training effectiveness.” The study used structural equation modeling (SEM) to evaluate and test the structural model and hypotheses.

Results

The online questionnaire response rate was 152 of 195 (78%); 88 of 152 participants (58%) were undergraduate trainees and 90 of 152 participants (59%) were women. The ages ranged from 18 to 53 years (mean 23.3, SD 6.0 years). There was no statistical difference in perceptions of training evaluation between men and women. Most participants were enthusiastic about the ChatGPT-assisted training, while the graduate professionals were more enthusiastic than undergraduate trainees. Nevertheless, some concerns were raised about potential cheating on training assessment. The average scores for knowledge acquisition, training motivation, training satisfaction, and training effectiveness were 3.84 (SD 0.80), 3.76 (SD 0.93), 3.75 (SD 0.87), and 3.72 (SD 0.91), respectively (Likert scale 1-5: strongly disagree to strongly agree). Knowledge acquisition had the highest score and training effectiveness the lowest. In the SEM results, training effectiveness was influenced predominantly by knowledge acquisition and partially met the hypotheses in the research framework. Knowledge acquisition had a direct effect on training effectiveness, training satisfaction, and training motivation, with β coefficients of .80, .87, and .97, respectively (all P

Subjects

MeSH terms (MEDLINE)

Attitude of Health PersonnelStudentsArtificial IntelligenceAdolescentAdultFemaleHumansMaleMiddle AgedYoung AdultMedicinePerceptionSurveys and QuestionnairesTaiwan

Author keywords

ChatGPTSEMinternet surveylarge language modelmedicineperception evaluationstructural equation modeling

Authors and institutions

  • Division of Colorectal Surgery, Department of Surgery, Tri-service General Hospital, National Defense Medical Center, Taipei, Taiwan.
  • Graduate Institute of Medical Sciences, National Defense Medical Center, Taipei, Taiwan.
  • School of Medicine, National Defense Medical Center, Taipei, Taiwan.
  • Division of Rheumatology/Immunology and Allergy, Department of Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan.
  • School of Public Health, National Defense Medical Center, Taipei, Taiwan.
  • Graduate Institute of Life Sciences, National Defense Medical Center, Taipei, Taiwan.

ORCID profiles: Je-Ming Hu, Feng-Cheng Liu, Chi-Ming Chu, Yu-Tien Chang

Cite

APA

Hu, J. M., Liu, F. C., Chu, C. M., & Chang, Y. T. (2023). Health Care Trainees’ and Professionals’ Perceptions of ChatGPT in Improving Medical Knowledge Training: Rapid Survey Study. Journal of Medical Internet Research. https://doi.org/10.2196/49385

Vancouver

Hu JM, Liu FC, Chu CM, Chang YT. Health Care Trainees’ and Professionals’ Perceptions of ChatGPT in Improving Medical Knowledge Training: Rapid Survey Study. Journal of Medical Internet Research. 2023. doi:10.2196/49385

BibTeX
@article{Hu2023,
  author = {Hu, Je-Ming and Liu, Feng-Cheng and Chu, Chi-Ming and Chang, Yu-Tien},
  title = {Health Care Trainees’ and Professionals’ Perceptions of ChatGPT in Improving Medical Knowledge Training: Rapid Survey Study},
  journal = {Journal of Medical Internet Research},
  year = {2023},
  doi = {10.2196/49385}
}
Start for free →