EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Extracting TNFi switching reasons and trajectories from real-world data using large language models

Brenda Y Miao, Marie Binvignat, Augusto Garcia-Agundez, Maxim Bravo, Christopher Yk Williams, Claire Q Miao, Ahmed Alaa, Vivek Rudrapatna iD, Atul J Butte, Gabriela Schmajuk, Jinoos Yazdany iD

JAMIA Open · 03.11.2025 · 8 (6)

DOI: 10.1093/jamiaopen/ooaf132 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 11.11.2025. Verfasst von Brenda Y Miao und 10 weiteren Autor:innen. Beteiligte Einrichtung: Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, CA 94143, United States. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIAMS NIH HHS, NLM NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background and significance To evaluate whether large language models (LLMs) can automate chart review to identify tumor necrosis factor inhibitor (TNFi) switching patterns and reasons for switching in a large real-world cohort. Materials and methods We conducted an observational study using de-identified electronic health record (EHR) data from 2012 to 2023 at a single academic medical center (University of California, San Francisco). TNFi medication orders and linked clinical notes were extracted, requiring at least 6 months of follow-up to identify treatment switches, defined as a change from one TNFi to another at consecutive encounters. Using GPT-4, we extracted which TNFi was stopped and started and classified the reason for switching. Performance was benchmarked against eight open-source LLMs, structured EHR data, and expert annotation.

Results

A total of 9187 patients (mean [SD] age, 39.9 [19.0] years; 57.1% female) received ≥1 TNFi with sufficient follow-up. We identified 3104 TNFi switches among 2112 patients. GPT-4 achieved micro-F1 scores of 0.75 (stopped drug), 0.80 (started drug), and 0.83 (reason). Among open-source models, Starling-7B-beta and Llama-3-8B performed most competitively. The most common reason identified by GPT-4 was lack of efficacy (56.9%), followed by adverse events (13.5%) and insurance/cost (10.8%).

Conclusions

Both GPT-4 and locally deployable LLMs effectively extracted complex treatment trajectories and rationale from clinical notes, supporting their broader utility in scalable EHR review and real-world evidence generation.

Schlagwörter

Stichwörter der Autor:innen

LLMsTNF-iTreatment switching

Autor:innen und Institutionen

  • Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, CA 94143, United States.
  • Immunology, Immunopathology, Immunotherapy I3 Lab, INSERM UMRS-959, Sorbonne Université, 75013 Paris, France.
  • Rheumatology Department and Research Center of Saint Antoine (CRSA), INSERM UMRS-938, Sorbonne Université, 75012 Paris, France.
  • Division of Rheumatology, University of California San Francisco, San Francisco, CA 94143, United States.
  • University of California, Berkeley, Berkeley, CA 94720, United States.
  • Division of Gastroenterology, University of California San Francisco, San Francisco, CA 94143, United States.

ORCID-Profile: Vivek Rudrapatna, Jinoos Yazdany

Förderung

  • NIAMS NIH HHS
  • NLM NIH HHS

Zitieren

APA

Miao, B. Y., Binvignat, M., Garcia-Agundez, A., Bravo, M., Williams, C. Y., Miao, C. Q., Alaa, A., Rudrapatna, V., Butte, A. J., Schmajuk, G., & Yazdany, J. (2025). Extracting TNFi switching reasons and trajectories from real-world data using large language models. JAMIA Open, 8(6). https://doi.org/10.1093/jamiaopen/ooaf132

Vancouver

Miao BY, Binvignat M, Garcia-Agundez A, Bravo M, Williams CY, Miao CQ, et al. Extracting TNFi switching reasons and trajectories from real-world data using large language models. JAMIA Open. 2025;8(6). doi:10.1093/jamiaopen/ooaf132

BibTeX
@article{Miao2025,
  author = {Miao, Brenda Y and Binvignat, Marie and Garcia-Agundez, Augusto and Bravo, Maxim and Williams, Christopher Yk and Miao, Claire Q and Alaa, Ahmed and Rudrapatna, Vivek and Butte, Atul J and Schmajuk, Gabriela and Yazdany, Jinoos},
  title = {Extracting TNFi switching reasons and trajectories from real-world data using large language models},
  journal = {JAMIA Open},
  year = {2025},
  volume = {8},
  number = {6},
  doi = {10.1093/jamiaopen/ooaf132}
}
Kostenlos starten →