EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study

Hiroyasu Sato, Katsuhiko Ogasawara

Journal of Educational Evaluation for Health Professions · 28.02.2024 · 21 · 4

DOI: 10.3352/jeehp.2024.21.4 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Educational Evaluation for Health Professions, online erschienen am 28.02.2024. Verfasst von Hiroyasu Sato und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pharmacy, Abashiri Kosei General Hospital, Abashiri, Japan. Hauptthemen laut MEDLINE: Pharmacists. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 14 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

The objective of this study was to assess the performance of ChatGPT (GPT-4) on all items, including those with diagrams, in the Japanese National License Examination for Pharmacists (JNLEP) and compare it with the previous GPT-3.5 model’s performance.

Methods

The 107th JNLEP, conducted in 2022, with 344 items input into the GPT-4 model, was targeted for this study. Separately, 284 items, excluding those with diagrams, were entered into the GPT-3.5 model. The answers were categorized and analyzed to determine accuracy rates based on categories, subjects, and presence or absence of diagrams. The accuracy rates were compared to the main passing criteria (overall accuracy rate ≥62.9%).

Results

The overall accuracy rate for all items in the 107th JNLEP in GPT-4 was 72.5%, successfully meeting all the passing criteria. For the set of items without diagrams, the accuracy rate was 80.0%, which was significantly higher than that of the GPT-3.5 model (43.5%). The GPT-4 model demonstrated an accuracy rate of 36.1% for items that included diagrams.

Conclusion

Advancements that allow GPT-4 to process images have made it possible for LLMs to answer all items in medical-related license examinations. This study’s findings confirm that ChatGPT (GPT-4) possesses sufficient knowledge to meet the passing criteria.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PharmacistsHumansJapan

Stichwörter der Autor:innen

JapanPharmacistsArtificial intelligencePharmacy licensureTask performance

Autor:innen und Institutionen

  • Department of Pharmacy, Abashiri Kosei General Hospital, Abashiri, Japan.
  • Graduate School of Health Sciences, Hokkaido University, Sapporo, Japan.
  • Graduate School of Engineering, Muroran Institute of Technology, Muroran, Japan

Zitieren

APA

Sato, H., & Ogasawara, K. (2024). ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study. Journal of Educational Evaluation for Health Professions, 21, 4. https://doi.org/10.3352/jeehp.2024.21.4

Vancouver

Sato H, Ogasawara K. ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study. Journal of Educational Evaluation for Health Professions. 2024;21:4. doi:10.3352/jeehp.2024.21.4

BibTeX
@article{Sato2024,
  author = {Sato, Hiroyasu and Ogasawara, Katsuhiko},
  title = {ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study},
  journal = {Journal of Educational Evaluation for Health Professions},
  year = {2024},
  volume = {21},
  pages = {4},
  doi = {10.3352/jeehp.2024.21.4}
}
Kostenlos starten →