EUVIMEDEUVIMED entdecken
Crossref

Enhancing Oncology-Specific Question Answering With Large Language Models Through Fine-Tuned Embeddings With Synthetic Data

Kun-Han Lu iD, Sina Mehdinia, Kingson Man, Chi Wah Wong iD, Allen Mao iD, Zahra Eftekhari

JCO Clinical Cancer Informatics · 2025

DOI: 10.1200/cci-25-00011 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JCO Clinical Cancer Informatics, 2025. Verfasst von Kun-Han Lu und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Applied AI and Data Science, City of Hope, Duarte, CA. Hauptthemen laut MEDLINE: Electronic Health Records, Medical Oncology, Natural Language Processing, Neoplasms. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

PURPOSE The recent advancements of retrieval-augmented generation (RAG) and large language models (LLMs) have revolutionized the extraction of real-world evidence from unstructured electronic health records (EHRs) in oncology. This study aims to enhance RAG's effectiveness by implementing a retriever encoder specifically designed for oncology EHRs, with the goal of improving the precision and relevance of retrieved clinical notes for oncology-related queries. METHODS Our model was pretrained with more than six million oncology notes from 209,135 patients at City of Hope. The model was subsequently fine-tuned into a sentence transformer model using 12,371 query-passage training pairs. Specifically, the passages were obtained from actual patient notes, whereas the query was synthesized by an LLM. We evaluated the retrieval performance of our model by comparing it with six widely used embedding models on 50 oncology questions across 10 categories based on Normalized Discounted Cumulative Gain (NDCG), Precision, and Recall. RESULTS In our test data set comprising 53 patients, our model exceeded the performance of the runner-up model by 9% for NDCG (evaluated at the top 10 results), 7% for Precision (top 10), and 6% for Recall (top 10). Our model showed exceptional retrieval performance across all metrics for oncology-specific categories, including biomarkers assessed, current diagnosis, disease status, laboratory results, tumor characteristics, and tumor staging. CONCLUSION Our findings highlight the effectiveness of pretrained contextual embeddings and sentence transformers in retrieving pertinent notes from oncology EHRs. The innovative use of LLM-synthesized query-passage pairs for data augmentation was proven to be effective. This fine-tuning approach holds significant promise in specialized fields like health care, where acquiring annotated data is challenging.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsMedical OncologyNatural Language ProcessingNeoplasmsInformation Storage and RetrievalHumansLanguageLarge Language Models

Autor:innen und Institutionen

  • Department of Applied AI and Data Science, City of Hope, Duarte, CA.
  • Division of Mathematics for Cancer Evolution and Early Detection, Beckman Research Institute, City of Hope, Duarte, CA.
  • Division of Research Informatics, City of Hope, Duarte, CA.

ORCID-Profile: Kun-Han Lu, Chi Wah Wong, Allen Mao

Zitieren

APA

Lu, K. H., Mehdinia, S., Man, K., Wong, C. W., Mao, A., & Eftekhari, Z. (2025). Enhancing Oncology-Specific Question Answering With Large Language Models Through Fine-Tuned Embeddings With Synthetic Data. JCO Clinical Cancer Informatics. https://doi.org/10.1200/cci-25-00011

Vancouver

Lu KH, Mehdinia S, Man K, Wong CW, Mao A, Eftekhari Z. Enhancing Oncology-Specific Question Answering With Large Language Models Through Fine-Tuned Embeddings With Synthetic Data. JCO Clinical Cancer Informatics. 2025. doi:10.1200/cci-25-00011

BibTeX
@article{Lu2025,
  author = {Lu, Kun-Han and Mehdinia, Sina and Man, Kingson and Wong, Chi Wah and Mao, Allen and Eftekhari, Zahra},
  title = {Enhancing Oncology-Specific Question Answering With Large Language Models Through Fine-Tuned Embeddings With Synthetic Data},
  journal = {JCO Clinical Cancer Informatics},
  year = {2025},
  doi = {10.1200/cci-25-00011}
}
Kostenlos starten →