EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

ChatGPT-4 extraction of heart failure symptoms and signs from electronic health records

T Elizabeth Workman, Ali Ahmed, Helen M Sheriff, Venkatesh K Raman, Sijian Zhang, Yijun Shao, Charles Faselis, Gregg C Fonarow, Qing Zeng-Treitler

Progress in Cardiovascular Diseases · 2024-11 · 87 · 44-49

DOI: 10.1016/j.pcad.2024.10.010 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Progress in Cardiovascular Diseases, 2024-11. Verfasst von T Elizabeth Workman und 8 weiteren Autor:innen. Beteiligte Einrichtung: Washington DC VA Medical Center, Washington DC, USA; The George Washington University, Washington DC, USA. Electronic address: lizworkman@gwu.edu. Hauptthemen laut MEDLINE: Electronic Health Records, Heart Failure, Natural Language Processing. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: HSRD VA, Health Services Research and Development, US Department of Veterans Affairs.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Natural language processing (NLP) can facilitate research utilizing data from electronic health records (EHRs). Large language models can potentially improve NLP applications leveraging EHR notes. The objective of this study was to assess the performance of zero-shot learning using Chat Generative Pre-trained Transformer 4 (ChatGPT-4) for extraction of symptoms and signs, and compare its performance to baseline machine learning and rule-based methods developed using annotated data. METHODS AND RESULTS: From unstructured clinical notes of the national EHR data of the Veterans healthcare system, we extracted 1999 text snippets containing relevant keywords for heart failure symptoms and signs, which were then annotated by two clinicians. We also created 102 synthetic snippets that were semantically similar to snippets randomly selected from the original 1999 snippets. The authors applied zero-shot learning, using two different forms of prompt engineering in a symptom and sign extraction task with ChatGPT-4, utilizing the synthetic snippets. For comparison, baseline models using machine learning and rule-based methods were trained using the original 1999 annotated text snippets, and then used to classify the 102 synthetic snippets. The best zero-shot learning application achieved 90.6 % precision, 100 % recall, and 95 % F1 score, outperforming the best baseline method, which achieved 54.9 % precision, 82.4 % recall, and 65.5 % F1 score. Prompt style and temperature settings influenced zero-shot learning performance. CONCLUSIONS: Zero-shot learning utilizing ChatGPT-4 significantly outperformed traditional machine learning and rule-based NLP. Prompt type and temperature settings affected zero-shot learning performance. These findings suggest a more efficient means of symptoms and signs extraction than traditional machine learning and rule-based methods.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsHeart FailureNatural Language ProcessingHumansData MiningMachine LearningPredictive Value of TestsUnited StatesSymptom Assessment

Stichwörter der Autor:innen

Heart failureInformation extractionLarge language modelsZero-shot learning

Autor:innen und Institutionen

  • Washington DC VA Medical Center, Washington DC, USA; The George Washington University, Washington DC, USA. Electronic address: lizworkman@gwu.edu.
  • Washington DC VA Medical Center, Washington DC, USA; The George Washington University, Washington DC, USA; Georgetown University, Washington DC, USA.
  • Washington DC VA Medical Center, Washington DC, USA; The George Washington University, Washington DC, USA.
  • Washington DC VA Medical Center, Washington DC, USA; Georgetown University, Washington DC, USA.
  • Washington DC VA Medical Center, Washington DC, USA.
  • University of California, Los Angeles, CA, USA.

Förderung

  • HSRD VA
  • Health Services Research and Development
  • US Department of Veterans Affairs

Zitieren

APA

Workman, T. E., Ahmed, A., Sheriff, H. M., Raman, V. K., Zhang, S., Shao, Y., Faselis, C., Fonarow, G. C., & Zeng-Treitler, Q. (2024). ChatGPT-4 extraction of heart failure symptoms and signs from electronic health records. Progress in Cardiovascular Diseases, 87, 44-49. https://doi.org/10.1016/j.pcad.2024.10.010

Vancouver

Workman TE, Ahmed A, Sheriff HM, Raman VK, Zhang S, Shao Y, et al. ChatGPT-4 extraction of heart failure symptoms and signs from electronic health records. Progress in Cardiovascular Diseases. 2024;87:44-49. doi:10.1016/j.pcad.2024.10.010

BibTeX
@article{Workman2024,
  author = {Workman, T Elizabeth and Ahmed, Ali and Sheriff, Helen M and Raman, Venkatesh K and Zhang, Sijian and Shao, Yijun and Faselis, Charles and Fonarow, Gregg C and Zeng-Treitler, Qing},
  title = {ChatGPT-4 extraction of heart failure symptoms and signs from electronic health records},
  journal = {Progress in Cardiovascular Diseases},
  year = {2024},
  volume = {87},
  pages = {44--49},
  doi = {10.1016/j.pcad.2024.10.010}
}
Kostenlos starten →