EUVIMEDObjevte EUVIMED
EditorialOpen access

ChatGPT in Medical Education: A Precursor for Automation Bias?

Tina Nguyen iD

JMIR Medical Education · 2024-01-17 · 10 · e50174

DOI: 10.2196/50174 ↗

At a glance

Editorial in JMIR Medical Education, published online on 2024-01-17. Written by Tina Nguyen. Institution: The University of Texas Medical Branch, Galveston, TX, United States. Major MEDLINE topics: Artificial Intelligence, Education, Medical. Cited 69 times so far (Crossref); 29 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Artificial intelligence (AI) in health care has the promise of providing accurate and efficient results. However, AI can also be a black box, where the logic behind its results is nonrational. There are concerns if these questionable results are used in patient care. As physicians have the duty to provide care based on their clinical judgment in addition to their patients’ values and preferences, it is crucial that physicians validate the results from AI. Yet, there are some physicians who exhibit a phenomenon known as automation bias, where there is an assumption from the user that AI is always right. This is a dangerous mindset, as users exhibiting automation bias will not validate the results, given their trust in AI systems. Several factors impact a user’s susceptibility to automation bias, such as inexperience or being born in the digital age. In this editorial, I argue that these factors and a lack of AI education in the medical school curriculum cause automation bias. I also explore the harms of automation bias and why prospective physicians need to be vigilant when using AI. Furthermore, it is important to consider what attitudes are being taught to students when introducing ChatGPT, which could be some students’ first time using AI, prior to their use of AI in the clinical setting. Therefore, in attempts to avoid the problem of automation bias in the long-term, in addition to incorporating AI education into the curriculum, as is necessary, the use of ChatGPT in medical education should be limited to certain tasks. Otherwise, having no constraints on what ChatGPT should be used for could lead to automation bias.

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceEducation, MedicalHumansProspective StudiesAutomationEducational Status

Author keywords

AIChatGPTLLMsartificial intelligenceautomation biasbiaslarge language modelsmedical educationmedical school curriculummedical studentsresidents

Authors and institutions

  • The University of Texas Medical Branch, Galveston, TX, United States.

ORCID profiles: Tina Nguyen

Cite

APA

Nguyen, T. (2024). ChatGPT in Medical Education: A Precursor for Automation Bias?. JMIR Medical Education, 10, e50174. https://doi.org/10.2196/50174

Vancouver

Nguyen T. ChatGPT in Medical Education: A Precursor for Automation Bias?. JMIR Medical Education. 2024;10:e50174. doi:10.2196/50174

BibTeX
@article{Nguyen2024,
  author = {Nguyen, Tina},
  title = {ChatGPT in Medical Education: A Precursor for Automation Bias?},
  journal = {JMIR Medical Education},
  year = {2024},
  volume = {10},
  pages = {e50174},
  doi = {10.2196/50174}
}
Start for free →