EUVIMEDEUVIMED entdecken
Open Access

Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study

Zhengyue Liu iD, Yi Ding iD, Jingxia Chen iD, Ziqiang Yan iD, Xuxi Cheng iD, Wei Zhou iD, Peng Fu iD, Zhuo Wang iD

JMIR Medical Informatics · 2026

DOI: 10.2196/97520 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Informatics, 2026. Verfasst von Zhengyue Liu und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Clinical Pharmacy, Shenyang pharmaceutical university, Shenyang, China. Hauptthemen laut MEDLINE: Large Language Models, Artificial Intelligence, Outpatients, Drug Prescriptions. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 17 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Pharmacists’ prescription review is key to medication safety, but rising numbers of outpatient prescriptions and expanding formularies leave less time per case, increasing error risk. Large language models (LLMs) show promise, yet two barriers hinder routine use: first, most systems rely on cloud-based commercial models, risking data breaches by transmitting protected patient information externally, and second, retrieval-augmented generation (RAG) pipelines used to reduce hallucinations depend on complex text vectorization and vector databases, which hospitals with limited IT resources struggle to build and maintain.

Objective

This study aimed to evaluate the feasibility and usefulness of a locally deployed, knowledge-augmented LLM as a decision-support tool for pharmacist-led outpatient prescription review.

Methods

The open-source Qwen3-14B model was deployed on a hospital intranet server using the Ollama framework. A structured knowledge base derived from drug package inserts served as the principal reference and supported lightweight knowledge augmentation through exact-match injection. A 2-period crossover design was used: 2 pharmacists independently reviewed 213 outpatient prescriptions under both unaided and AI-assisted conditions, yielding paired unaided and collaborative results for each prescription. Plain AI review and knowledge-augmented AI review were also run on the same 213-prescription test set to quantify hallucination rates, and stand-alone AI performance was assessed against the reference standard. The reference standard was established by independent consensus between two supervising pharmacists, with disagreements adjudicated by a deputy chief pharmacist. Accuracy, sensitivity, and specificity were compared between conditions using paired McNemar tests, and review time was compared using the Wilcoxon signed-rank test.

Results

Overall review accuracy was 97.2% (207/213, 95% CI 94.0%-98.7%) in the human-AI collaborative condition vs 82.6% (176/213, 95% CI 77.0%-87.1%) in the pharmacist-alone condition (P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsArtificial IntelligenceOutpatientsDrug PrescriptionsHumansCross-Over Studies

Stichwörter der Autor:innen

Ollamahuman-AI collaborationhuman–artificial intelligence collaborationknowledge augmentationlarge language modelon-premise deploymentprescription review

Autor:innen und Institutionen

  • School of Clinical Pharmacy, Shenyang pharmaceutical university, Shenyang, China.
  • Department of Pharmacy, Shanghai Changhai Hospital, The First Affiliated Hospital of Navy Medical University, Shanghai, China.

ORCID-Profile: Zhengyue Liu, Yi Ding, Jingxia Chen, Ziqiang Yan, Xuxi Cheng, Wei Zhou, Peng Fu, Zhuo Wang

Zitieren

APA

Liu, Z., Ding, Y., Chen, J., Yan, Z., Cheng, X., Zhou, W., Fu, P., & Wang, Z. (2026). Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study. JMIR Medical Informatics. https://doi.org/10.2196/97520

Vancouver

Liu Z, Ding Y, Chen J, Yan Z, Cheng X, Zhou W, et al. Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study. JMIR Medical Informatics. 2026. doi:10.2196/97520

BibTeX
@article{Liu2026,
  author = {Liu, Zhengyue and Ding, Yi and Chen, Jingxia and Yan, Ziqiang and Cheng, Xuxi and Zhou, Wei and Fu, Peng and Wang, Zhuo},
  title = {Locally Deployed Large Language Models for AI-Assisted Outpatient Prescription Review: Crossover Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  doi = {10.2196/97520}
}
Kostenlos starten →