EUVIMEDObjevte EUVIMED
Open access

An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study

Ya-Han Hu iD, Yi-Ying Cheng iD, Chung-Ching Lan, Yu-Hsiang Su iD, Sheng-Feng Sung iD

JMIR Medical Informatics · 2025

DOI: 10.2196/80072 ↗

At a glance

Journal article in JMIR Medical Informatics, 2025. Written by Ya-Han Hu and 4 co-authors. Institution: Department of Information Management, National Central University, Taoyuan, Taiwan. Major MEDLINE topics: Natural Language Processing, Eligibility Determination, User-Computer Interface, Clinical Trials as Topic. Cited 3 times so far (Crossref); 48 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Clinical trial eligibility screening using electronic medical records (EMRs) is challenging due to the complexity of patient data and the varied clinical terminologies. Manual screening is time-consuming, requires specialized knowledge, and can lead to inconsistent participant selection, potentially compromising patient safety and research outcomes. This is critical in time-sensitive conditions like acute ischemic stroke. While computerized clinical decision support tools offer solutions, most require software engineering expertise to update, limiting their practical utility when eligibility criteria change.

Objective

We developed and evaluated the intelligent trial eligibility screening tool (iTEST), which combines natural language processing with a block-based visual programming interface designed to enable clinicians to create and modify eligibility screening rules independently. In this study, we assessed iTEST’s rule evaluation module using pre-configured rules and compared its effectiveness with that of standard EMR interfaces.

Methods

We conducted an experiment at a tertiary teaching hospital in Taiwan with 12 clinicians using a 2-period crossover design. The clinicians assessed the eligibility of 4 patients with stroke for 2 clinical trials using both standard EMR and iTEST in a counterbalanced order, resulting in 48 evaluation scenarios. The iTEST comprised a rule authoring module using Google Blockly and a rule evaluation module utilizing MetaMap Lite for extracting medical concepts from unstructured EMR documents and structured laboratory data. Primary outcomes included accuracy in determining eligibility. Secondary outcomes measured task completion time, cognitive workload using the National Aeronautics and Space Administration Task Load Index scale (range 0‐100, with lower scores indicating a lower cognitive workload), and system usability through the system usability scale (range: 0‐100, with higher scores indicating higher system usability).

Results

The iTEST significantly improved accuracy scores (from 0.91 to 1.00, P

Subjects

MeSH terms (MEDLINE)

Natural Language ProcessingEligibility DeterminationUser-Computer InterfaceClinical Trials as TopicPatient SelectionHumansElectronic Health RecordsTaiwanMaleDecision Support Systems, ClinicalFemale

Author keywords

block-based visual programmingclinical decision supportclinical trialselectronic medical recordseligibility screeningnatural language processingpatient safety

Authors and institutions

  • Department of Information Management, National Central University, Taoyuan, Taiwan.
  • Asian Institute for Impact Measurement and Management, National Central University, Taoyuan, Taiwan.
  • Division of Neurology, Department of Internal Medicine, Ditmanson Medical Foundation Chia-Yi Christian Hospital, 539 Zhongxiao Road, East District, Chiayi City, 60002, Taiwan, +886-5-2765041 ext 8655.
  • Department of Nursing, Fooyin University, Kaohsiung, Taiwan.

ORCID profiles: Ya-Han Hu, Yi-Ying Cheng, Yu-Hsiang Su, Sheng-Feng Sung

Cite

APA

Hu, Y. H., Cheng, Y. Y., Lan, C. C., Su, Y. H., & Sung, S. F. (2025). An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study. JMIR Medical Informatics. https://doi.org/10.2196/80072

Vancouver

Hu YH, Cheng YY, Lan CC, Su YH, Sung SF. An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study. JMIR Medical Informatics. 2025. doi:10.2196/80072

BibTeX
@article{Hu2025,
  author = {Hu, Ya-Han and Cheng, Yi-Ying and Lan, Chung-Ching and Su, Yu-Hsiang and Sung, Sheng-Feng},
  title = {An Intelligent Trial Eligibility Screening Tool Using Natural Language Processing With a Block-Based Visual Programming Interface: Development and Usability Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  doi = {10.2196/80072}
}
Start for free →