Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study
JMIR Formative Research · 2026
DOI: 10.2196/95851 ↗At a glance
Vergleichsstudie in JMIR Formative Research, 2026. Written by Xin Liu and 1 co-authors. Institution: Thomas College of Business & Economics, University of North Carolina at Pembroke, Pembroke, NC, United States. Major MEDLINE topics: Acupuncture Therapy, Physicians, Medicine, Chinese Traditional. 78 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Traditional Chinese medicine (TCM) has received increasing attention in evidence-based medicine. However, its largely experience-based and practice-oriented nature has slowed modernization and standardization. Advances in AI provide new opportunities for integrating AI with TCM.
Objective
This study evaluates the application of large language models (LLMs), including ChatGPT (OpenAI) and DeepSeek, in generating acupuncture treatment protocols. We compare the validity and clinical effectiveness of physician-generated protocols with those produced by LLMs, examine differences in outputs across language corpora (Chinese and English), and establish a standardized assessment framework for systematically assessing the effectiveness of acupuncture point selection. Ultimately, this research aims to support the standardization of acupoint selection and promote greater consistency, reliability, and evidence-based development in acupuncture practice.
Methods
Qualified acupuncture treatment cases published in Acupuncture in Medicine were identified and translated from English into Chinese to create a bilingual case dataset. Standardized case prompts were independently input into DeepSeek and ChatGPT to generate acupuncture treatment protocols. Ten senior TCM physicians evaluated the AI-generated and physician-generated protocols using 7 criteria: local point selection, distal point selection, syndrome-based point selection, meridian-tracing point selection, neuroanatomical point selection, core acupoint selection, and synergistic effects. Repeated-measures ANOVA with sphericity assessment was used to examine differences across LLMs, language corpora, and protocol groups.
Results
Blinded expert evaluations revealed significant differences among the 5 acupuncture groups for local point selection (P=.006), syndrome-based point selection (P=.005), neuroanatomical point selection (P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Thomas College of Business & Economics, University of North Carolina at Pembroke, Pembroke, NC, United States.
- Northeast Agricultural University, Harbin, Heilongjiang, China.
ORCID profiles: Xin Liu, Zining Guo
Cite
APA
Liu, X., & Guo, Z. (2026). Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study. JMIR Formative Research. https://doi.org/10.2196/95851
Vancouver
Liu X, Guo Z. Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study. JMIR Formative Research. 2026. doi:10.2196/95851
BibTeX
@article{Liu2026,
author = {Liu, Xin and Guo, Zining},
title = {Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study},
journal = {JMIR Formative Research},
year = {2026},
doi = {10.2196/95851}
}
EUVIMED