EUVIMEDEUVIMED entdecken
Open Access

Distilling large language models for matching patients to clinical trials

Mauro Nievas, Aditya Basu, Yanshan Wang, Hrituraj Singh

Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 1953-1963

DOI: 10.1093/jamia/ocae073 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 19.04.2024. Verfasst von Mauro Nievas und 3 weiteren Autor:innen. Beteiligte Einrichtung: Triomics Research, Triomics, Inc., San Francisco, CA 94105, United States. Hauptthemen laut MEDLINE: Clinical Trials as Topic, Patient Selection. Die Arbeit wurde bisher 71-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

The objective of this study is to systematically examine the efficacy of both proprietary (GPT-3.5, GPT-4) and open-source large language models (LLMs) (LLAMA 7B, 13B, 70B) in the context of matching patients to clinical trials in healthcare. Materials and methods The study employs a multifaceted evaluation framework, incorporating extensive automated and human-centric assessments along with a detailed error analysis for each model, and assesses LLMs’ capabilities in analyzing patient eligibility against clinical trial’s inclusion and exclusion criteria. To improve the adaptability of open-source LLMs, a specialized synthetic dataset was created using GPT-4, facilitating effective fine-tuning under constrained data conditions.

Results

The findings indicate that open-source LLMs, when fine-tuned on this limited and synthetic dataset, achieve performance parity with their proprietary counterparts, such as GPT-3.5.

Discussion

This study highlights the recent success of LLMs in the high-stakes domain of healthcare, specifically in patient-trial matching. The research demonstrates the potential of open-source models to match the performance of proprietary models when fine-tuned appropriately, addressing challenges like cost, privacy, and reproducibility concerns associated with closed-source proprietary LLMs.

Conclusion

The study underscores the opportunity for open-source LLMs in patient-trial matching. To encourage further research and applications in this field, the annotated evaluation dataset and the fine-tuned LLM, Trial-LLAMA, are released for public use.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Clinical Trials as TopicPatient SelectionHumansProgramming LanguagesNatural Language Processing

Stichwörter der Autor:innen

GPT-3.5GPT-4LLAMAclinical trial matchingdistillationlarge language models

Autor:innen und Institutionen

  • Triomics Research, Triomics, Inc., San Francisco, CA 94105, United States.
  • Triomics Research, Triomics, Inc., Bengaluru, Karnataka 560102, India.
  • Department of Health Information Management, University of Pittsburgh, Pittsburgh, PA 15260, United States.

Zitieren

APA

Nievas, M., Basu, A., Wang, Y., & Singh, H. (2024). Distilling large language models for matching patients to clinical trials. Journal of the American Medical Informatics Association, 31(9), 1953-1963. https://doi.org/10.1093/jamia/ocae073

Vancouver

Nievas M, Basu A, Wang Y, Singh H. Distilling large language models for matching patients to clinical trials. Journal of the American Medical Informatics Association. 2024;31(9):1953-1963. doi:10.1093/jamia/ocae073

BibTeX
@article{Nievas2024,
  author = {Nievas, Mauro and Basu, Aditya and Wang, Yanshan and Singh, Hrituraj},
  title = {Distilling large language models for matching patients to clinical trials},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  volume = {31},
  number = {9},
  pages = {1953--1963},
  doi = {10.1093/jamia/ocae073}
}
Kostenlos starten →