EUVIMEDOpdag EUVIMED
VergleichsstudieQuerschnittstudieOpen access

Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study

Xin Liu iD, Zining Guo iD

JMIR Formative Research · 2026

DOI: 10.2196/95851 ↗

At a glance

Vergleichsstudie in JMIR Formative Research, 2026. Written by Xin Liu and 1 co-authors. Institution: Thomas College of Business & Economics, University of North Carolina at Pembroke, Pembroke, NC, United States. Major MEDLINE topics: Acupuncture Therapy, Physicians, Medicine, Chinese Traditional. 78 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Traditional Chinese medicine (TCM) has received increasing attention in evidence-based medicine. However, its largely experience-based and practice-oriented nature has slowed modernization and standardization. Advances in AI provide new opportunities for integrating AI with TCM.

Objective

This study evaluates the application of large language models (LLMs), including ChatGPT (OpenAI) and DeepSeek, in generating acupuncture treatment protocols. We compare the validity and clinical effectiveness of physician-generated protocols with those produced by LLMs, examine differences in outputs across language corpora (Chinese and English), and establish a standardized assessment framework for systematically assessing the effectiveness of acupuncture point selection. Ultimately, this research aims to support the standardization of acupoint selection and promote greater consistency, reliability, and evidence-based development in acupuncture practice.

Methods

Qualified acupuncture treatment cases published in Acupuncture in Medicine were identified and translated from English into Chinese to create a bilingual case dataset. Standardized case prompts were independently input into DeepSeek and ChatGPT to generate acupuncture treatment protocols. Ten senior TCM physicians evaluated the AI-generated and physician-generated protocols using 7 criteria: local point selection, distal point selection, syndrome-based point selection, meridian-tracing point selection, neuroanatomical point selection, core acupoint selection, and synergistic effects. Repeated-measures ANOVA with sphericity assessment was used to examine differences across LLMs, language corpora, and protocol groups.

Results

Blinded expert evaluations revealed significant differences among the 5 acupuncture groups for local point selection (P=.006), syndrome-based point selection (P=.005), neuroanatomical point selection (P

Subjects

MeSH terms (MEDLINE)

Acupuncture TherapyPhysiciansMedicine, Chinese TraditionalLarge Language ModelsHumansCross-Sectional StudiesGenerative Artificial IntelligenceChinaReproducibility of ResultsLanguage

Author keywords

AIChatGPTDeepSeekTCMacupoint selectionacupunctureintegrative medicinelanguage corporalarge language modeltraditional Chinese medicine

Authors and institutions

  • Thomas College of Business & Economics, University of North Carolina at Pembroke, Pembroke, NC, United States.
  • Northeast Agricultural University, Harbin, Heilongjiang, China.

ORCID profiles: Xin Liu, Zining Guo

Cite

APA

Liu, X., & Guo, Z. (2026). Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study. JMIR Formative Research. https://doi.org/10.2196/95851

Vancouver

Liu X, Guo Z. Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study. JMIR Formative Research. 2026. doi:10.2196/95851

BibTeX
@article{Liu2026,
  author = {Liu, Xin and Guo, Zining},
  title = {Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study},
  journal = {JMIR Formative Research},
  year = {2026},
  doi = {10.2196/95851}
}
Start for free →