A PRACTICAL FRAMEWORK FOR EVALUATING AI CHATBOTS IN NON-ENGLISH DRUG INFORMATION SERVICES
Thai Bulletin of Pharmaceutical Sciences · 20.07.2026 · 21 (2) · 251-261
DOI: 10.69598/tbps.21.2.251-261 ↗Kurz gefasst
Fachartikel in der Zeitschrift Thai Bulletin of Pharmaceutical Sciences, online erschienen am 20.07.2026. Verfasst von Inthira Kanchanaphibool und 6 weiteren Autor:innen.
Abstract (Originalsprache)
Artificial intelligence, particularly AI-based chatbots, has gained increasing attention as a tool to support pharmacists in providing drug information services, especially in settings with pharmacist shortages. This study compared the competencies of ChatGPT, ChatGPT Pro, Perplexity, and Perplexity Pro in responding to drug-related clinical questions, and examined the effect of an Enhanced Task Translation (ETT) technique on chatbot performance, after English technical terms were embedded with Thai-language prompts. An experimental comparative design was employed. Two sets of Thai-language multiple-choice questions (MCQs), each comprising 120 items based on the Thai Pharmacy Licensing Examination, were administered: one standard set and one ETT set. Cardiology-focused clinical case scenarios were additionally presented using a SOAP-note format. Performance was assessed using a rubric adapted from the American Society of Health-System Pharmacists (ASHP) guidelines across four domains: question classification, source citation, evidence application, and communication. All four chatbots surpassed the 60% passing threshold: 72.08–83.33% for the standard set and 73.75–80.83% for the ETT set. Perplexity Pro demonstrated the highest MCQ performance, while the inclusion of English technical terms did not consistently improve results across models. In the clinical case assessment, ChatGPT Pro achieved the highest rubric score, showing strong evidence use and clinical reasoning. These findings suggest that AI chatbot competency in drug-related queries is broadly comparable to that of pharmacists. The ETT technique did not produce notable performance differences between Thai-only and mixed-language prompts. AI chatbots may support pharmacists by generating initial structured responses to drug information requests. The proposed evaluation framework provides a potential model for assessing AI competency in non-English settings and underscores the importance of jointly evaluating knowledge accuracy and clinical judgment to ensure safe integration into drug information services.
Zitieren
APA
Kanchanaphibool, I., Aonpong, P., Dabngoen, T., Kongmee, K., Wongcharee, T., Tantasuparuk, T., & Hirunrassamee, S. (2026). A PRACTICAL FRAMEWORK FOR EVALUATING AI CHATBOTS IN NON-ENGLISH DRUG INFORMATION SERVICES. Thai Bulletin of Pharmaceutical Sciences, 21(2), 251-261. https://doi.org/10.69598/tbps.21.2.251-261
Vancouver
Kanchanaphibool I, Aonpong P, Dabngoen T, Kongmee K, Wongcharee T, Tantasuparuk T, et al. A PRACTICAL FRAMEWORK FOR EVALUATING AI CHATBOTS IN NON-ENGLISH DRUG INFORMATION SERVICES. Thai Bulletin of Pharmaceutical Sciences. 2026;21(2):251-261. doi:10.69598/tbps.21.2.251-261
BibTeX
@article{Kanchanaphibool2026,
author = {Kanchanaphibool, Inthira and Aonpong, Panyanat and Dabngoen, Thanaphat and Kongmee, Krittin and Wongcharee, Thanawat and Tantasuparuk, Tanut and Hirunrassamee, Sanita},
title = {A PRACTICAL FRAMEWORK FOR EVALUATING AI CHATBOTS IN NON-ENGLISH DRUG INFORMATION SERVICES},
journal = {Thai Bulletin of Pharmaceutical Sciences},
year = {2026},
volume = {21},
number = {2},
pages = {251--261},
doi = {10.69598/tbps.21.2.251-261}
}
EUVIMED