EUVIMEDOpdag EUVIMED
Open access

Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study

Zhengyue Liu iD, Yi Ding iD, Jingxia Chen iD, Ziqiang Yan iD, Xuxi Cheng iD, Wei Zhou iD, Peng Fu iD, Zhuo Wang iD

JMIR Medical Informatics · 2026

DOI: 10.2196/97520 ↗

At a glance

Journal article in JMIR Medical Informatics, 2026. Written by Zhengyue Liu and 7 co-authors. Institution: School of Clinical Pharmacy, Shenyang pharmaceutical university, Shenyang, China. Major MEDLINE topics: Large Language Models, Artificial Intelligence, Outpatients, Drug Prescriptions. Cited 1 times so far (Crossref); 17 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Pharmacists’ prescription review is key to medication safety, but rising numbers of outpatient prescriptions and expanding formularies leave less time per case, increasing error risk. Large language models (LLMs) show promise, yet two barriers hinder routine use: first, most systems rely on cloud-based commercial models, risking data breaches by transmitting protected patient information externally, and second, retrieval-augmented generation (RAG) pipelines used to reduce hallucinations depend on complex text vectorization and vector databases, which hospitals with limited IT resources struggle to build and maintain.

Objective

This study aimed to evaluate the feasibility and usefulness of a locally deployed, knowledge-augmented LLM as a decision-support tool for pharmacist-led outpatient prescription review.

Methods

The open-source Qwen3-14B model was deployed on a hospital intranet server using the Ollama framework. A structured knowledge base derived from drug package inserts served as the principal reference and supported lightweight knowledge augmentation through exact-match injection. A 2-period crossover design was used: 2 pharmacists independently reviewed 213 outpatient prescriptions under both unaided and AI-assisted conditions, yielding paired unaided and collaborative results for each prescription. Plain AI review and knowledge-augmented AI review were also run on the same 213-prescription test set to quantify hallucination rates, and stand-alone AI performance was assessed against the reference standard. The reference standard was established by independent consensus between two supervising pharmacists, with disagreements adjudicated by a deputy chief pharmacist. Accuracy, sensitivity, and specificity were compared between conditions using paired McNemar tests, and review time was compared using the Wilcoxon signed-rank test.

Results

Overall review accuracy was 97.2% (207/213, 95% CI 94.0%-98.7%) in the human-AI collaborative condition vs 82.6% (176/213, 95% CI 77.0%-87.1%) in the pharmacist-alone condition (P

Subjects

MeSH terms (MEDLINE)

Large Language ModelsArtificial IntelligenceOutpatientsDrug PrescriptionsHumansCross-Over Studies

Author keywords

Ollamahuman-AI collaborationhuman–artificial intelligence collaborationknowledge augmentationlarge language modelon-premise deploymentprescription review

Authors and institutions

  • School of Clinical Pharmacy, Shenyang pharmaceutical university, Shenyang, China.
  • Department of Pharmacy, Shanghai Changhai Hospital, The First Affiliated Hospital of Navy Medical University, Shanghai, China.

ORCID profiles: Zhengyue Liu, Yi Ding, Jingxia Chen, Ziqiang Yan, Xuxi Cheng, Wei Zhou, Peng Fu, Zhuo Wang

Cite

APA

Liu, Z., Ding, Y., Chen, J., Yan, Z., Cheng, X., Zhou, W., Fu, P., & Wang, Z. (2026). Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study. JMIR Medical Informatics. https://doi.org/10.2196/97520

Vancouver

Liu Z, Ding Y, Chen J, Yan Z, Cheng X, Zhou W, et al. Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study. JMIR Medical Informatics. 2026. doi:10.2196/97520

BibTeX
@article{Liu2026,
  author = {Liu, Zhengyue and Ding, Yi and Chen, Jingxia and Yan, Ziqiang and Cheng, Xuxi and Zhou, Wei and Fu, Peng and Wang, Zhuo},
  title = {Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  doi = {10.2196/97520}
}
Start for free →