EUVIMEDOpdag EUVIMED
Open access

A Knowledge Graph–Enhanced Question-Answering System for Chinese Patent Medicines in Pediatric Influenza: Development and Evaluation Study

Haoyuan Li, Guozhen Zhao, Xingyu Zong, Ning Liang, Huizhen Li, Wandi Zhang, Jiahao Lin, Bohan Ma, Jinghan Jia, Yanping Wang, Nannan Shi

JMIR AI · 2026-11-13 · 5 · e101648-e101648

DOI: 10.2196/101648 ↗

At a glance

Journal article in JMIR AI, published online on 2026-05-18. Written by Haoyuan Li and 10 co-authors. Institution: Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, No.16, Nanxiaojie, Dongzhimen Nei Street, Dongcheng Dist…. 13 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Pediatric influenza is an important public health concern. Chinese patent medicines are used in its treatment, but relevant clinical evidence is dispersed across guidelines, expert consensus statements, and clinical studies. The multidimensional knowledge required for syndrome-based prescribing complicates information retrieval and standardized clinical use.

Objective

This study aimed to develop a traceable knowledge graph of Chinese patent medicines for pediatric influenza, integrate it with a large language model (LLM) to support evidence-based question answering, and evaluate answer quality and preliminary usability.

Methods

We systematically searched for clinical practice guidelines, expert consensus statements, and clinical studies. Evidence segmentation, ontology development, LLM-assisted extraction with manual verification, terminology normalization, and knowledge integration were used to construct a Neo4j knowledge graph. Structured graph retrieval was combined with retrieval-augmented generation to develop a question-answering system. A standalone LLM and the knowledge graph–enhanced LLM (KG-LLM) each answered 80 standardized questions. Three clinical experts independently evaluated anonymized, randomly ordered answers for core task accuracy, completeness, and unsupported factual content. Five physicians participated in an exploratory usability evaluation using the System Usability Scale (SUS), study-specific Likert items, and self-reported evidence-location outcomes.

Results

We included 142 publications and generated 1294 evidence units. The knowledge graph contained 3211 nodes and 19,671 relationships, linking structured knowledge to evidence units and source publications. Across 80 paired questions, KG-LLM achieved higher accuracy scores than the standalone LLM (mean 3.05, SD 1.88 vs mean 2.02, SD 1.54; Holm-adjusted P

Subjects

Author keywords

Chinese patent medicineclinical decision supportevidence provenanceknowledge graphlarge language modelpediatric influenzaretrieval-augmented generation

Authors and institutions

  • Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, No.16, Nanxiaojie, Dongzhimen Nei Street, Dongcheng District, Beijing, China, 86 010-64093251.
  • School of Acupuncture-Moxibustion and Tuina, Beijing University of Chinese Medicine, Beijing, China.

Cite

APA

Li, H., Zhao, G., Zong, X., Liang, N., Li, H., Zhang, W., Lin, J., Ma, B., Jia, J., Wang, Y., & Shi, N. (2026). A Knowledge Graph-Enhanced Question-Answering System for Chinese Patent Medicines in Pediatric Influenza: Development and Evaluation Study (Preprint). JMIR AI. https://doi.org/10.2196/101648

Vancouver

Li H, Zhao G, Zong X, Liang N, Li H, Zhang W, et al. A Knowledge Graph-Enhanced Question-Answering System for Chinese Patent Medicines in Pediatric Influenza: Development and Evaluation Study (Preprint). JMIR AI. 2026. doi:10.2196/101648

BibTeX
@article{Li2026,
  author = {Li, Haoyuan and Zhao, Guozhen and Zong, Xingyu and Liang, Ning and Li, Huizhen and Zhang, Wandi and Lin, Jiahao and Ma, Bohan and Jia, Jinghan and Wang, Yanping and Shi, Nannan},
  title = {A Knowledge Graph-Enhanced Question-Answering System for Chinese Patent Medicines in Pediatric Influenza: Development and Evaluation Study (Preprint)},
  journal = {JMIR AI},
  year = {2026},
  doi = {10.2196/101648}
}
Start for free →