An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study
JMIR Medical Informatics · 2025
DOI: 10.2196/80072 ↗At a glance
Journal article in JMIR Medical Informatics, 2025. Written by Ya-Han Hu and 4 co-authors. Institution: Department of Information Management, National Central University, Taoyuan, Taiwan. Major MEDLINE topics: Natural Language Processing, Eligibility Determination, User-Computer Interface, Clinical Trials as Topic. Cited 3 times so far (Crossref); 48 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Clinical trial eligibility screening using electronic medical records (EMRs) is challenging due to the complexity of patient data and the varied clinical terminologies. Manual screening is time-consuming, requires specialized knowledge, and can lead to inconsistent participant selection, potentially compromising patient safety and research outcomes. This is critical in time-sensitive conditions like acute ischemic stroke. While computerized clinical decision support tools offer solutions, most require software engineering expertise to update, limiting their practical utility when eligibility criteria change.
Objective
We developed and evaluated the intelligent trial eligibility screening tool (iTEST), which combines natural language processing with a block-based visual programming interface designed to enable clinicians to create and modify eligibility screening rules independently. In this study, we assessed iTEST’s rule evaluation module using pre-configured rules and compared its effectiveness with that of standard EMR interfaces.
Methods
We conducted an experiment at a tertiary teaching hospital in Taiwan with 12 clinicians using a 2-period crossover design. The clinicians assessed the eligibility of 4 patients with stroke for 2 clinical trials using both standard EMR and iTEST in a counterbalanced order, resulting in 48 evaluation scenarios. The iTEST comprised a rule authoring module using Google Blockly and a rule evaluation module utilizing MetaMap Lite for extracting medical concepts from unstructured EMR documents and structured laboratory data. Primary outcomes included accuracy in determining eligibility. Secondary outcomes measured task completion time, cognitive workload using the National Aeronautics and Space Administration Task Load Index scale (range 0‐100, with lower scores indicating a lower cognitive workload), and system usability through the system usability scale (range: 0‐100, with higher scores indicating higher system usability).
Results
The iTEST significantly improved accuracy scores (from 0.91 to 1.00, P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Information Management, National Central University, Taoyuan, Taiwan.
- Asian Institute for Impact Measurement and Management, National Central University, Taoyuan, Taiwan.
- Division of Neurology, Department of Internal Medicine, Ditmanson Medical Foundation Chia-Yi Christian Hospital, 539 Zhongxiao Road, East District, Chiayi City, 60002, Taiwan, +886-5-2765041 ext 8655.
- Department of Nursing, Fooyin University, Kaohsiung, Taiwan.
ORCID profiles: Ya-Han Hu, Yi-Ying Cheng, Yu-Hsiang Su, Sheng-Feng Sung
Cite
APA
Hu, Y. H., Cheng, Y. Y., Lan, C. C., Su, Y. H., & Sung, S. F. (2025). An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study. JMIR Medical Informatics. https://doi.org/10.2196/80072
Vancouver
Hu YH, Cheng YY, Lan CC, Su YH, Sung SF. An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study. JMIR Medical Informatics. 2025. doi:10.2196/80072
BibTeX
@article{Hu2025,
author = {Hu, Ya-Han and Cheng, Yi-Ying and Lan, Chung-Ching and Su, Yu-Hsiang and Sung, Sheng-Feng},
title = {An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study},
journal = {JMIR Medical Informatics},
year = {2025},
doi = {10.2196/80072}
}
EUVIMED