Analysis of eligibility criteria clusters based on large language models for clinical trial design
Journal of the American Medical Informatics Association · 01.03.2025 · 32 (3) · 447-458
DOI: 10.1093/jamia/ocae311 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 26.12.2024. Verfasst von Alban Bornet und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Radiology and Medical Informatics, University of Geneva, 1202 Geneva, Switzerland. Hauptthemen laut MEDLINE: Clinical Trials as Topic, Eligibility Determination, Natural Language Processing, Patient Selection. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Swiss Innovation Agency Innosuisse, CTxAI: quality by design of clinical studies using explainable AI.
Abstract (Originalsprache)
Objectives
Clinical trials (CTs) are essential for improving patient care by evaluating new treatments’ safety and efficacy. A key component in CT protocols is the study population defined by the eligibility criteria. This study aims to evaluate the effectiveness of large language models (LLMs) in encoding eligibility criterion information to support CT-protocol design. Materials and Methods We extracted eligibility criterion sections, phases, conditions, and interventions from CT protocols available in the ClinicalTrials.gov registry. Eligibility sections were split into individual rules using a criterion tokenizer and embedded using LLMs. The obtained representations were clustered. The quality and relevance of the clusters for protocol design was evaluated through 3 experiments: intrinsic alignment with protocol information and human expert cluster coherence assessment, extrinsic evaluation through CT-level classification tasks, and eligibility section generation.
Results
Sentence embeddings fine-tuned using biomedical corpora produce clusters with the highest alignment to CT-level information. Human expert evaluation confirms that clusters are well structured and coherent. Despite the high information compression, clusters retain significant CT information, up to 97% of the classification performance obtained with raw embeddings. Finally, eligibility sections automatically generated using clusters achieve 95% of the ROUGE scores obtained with a generative LLM prompted with CT-protocol details, suggesting that clusters encapsulate information useful to CT-protocol design.
Discussion
Clusters derived from sentence-level LLM embeddings effectively summarize complex eligibility criterion data while retaining relevant CT-protocol details. Clustering-based approaches provide a scalable enhancement in CT design that balances information compression with accuracy.
Conclusions
Clustering eligibility criteria using LLM embeddings provides a practical and efficient method to summarize critical protocol information. We provide an interactive visualization of the pipeline here.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Radiology and Medical Informatics, University of Geneva, 1202 Geneva, Switzerland.
- Risklick AG, 3013 Bern, Switzerland.
ORCID-Profile: Alban Bornet, Philipp Khlebnikov, Florian Meer, Anthony Yazdani, Boya Zhang, Poorya Amini, Douglas Teodoro
Förderung
- Swiss Innovation Agency Innosuisse
- CTxAI: quality by design of clinical studies using explainable AI
Zitieren
APA
Bornet, A., Khlebnikov, P., Meer, F., Haas, Q., Yazdani, A., Zhang, B., Amini, P., & Teodoro, D. (2025). Analysis of eligibility criteria clusters based on large language models for clinical trial design. Journal of the American Medical Informatics Association, 32(3), 447-458. https://doi.org/10.1093/jamia/ocae311
Vancouver
Bornet A, Khlebnikov P, Meer F, Haas Q, Yazdani A, Zhang B, et al. Analysis of eligibility criteria clusters based on large language models for clinical trial design. Journal of the American Medical Informatics Association. 2025;32(3):447-458. doi:10.1093/jamia/ocae311
BibTeX
@article{Bornet2025,
author = {Bornet, Alban and Khlebnikov, Philipp and Meer, Florian and Haas, Quentin and Yazdani, Anthony and Zhang, Boya and Amini, Poorya and Teodoro, Douglas},
title = {Analysis of eligibility criteria clusters based on large language models for clinical trial design},
journal = {Journal of the American Medical Informatics Association},
year = {2025},
volume = {32},
number = {3},
pages = {447--458},
doi = {10.1093/jamia/ocae311}
}
EUVIMED