Disparities in seizure outcomes revealed by large language models
Journal of the American Medical Informatics Association · 2024
DOI: 10.1093/jamia/ocae047 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2024. Verfasst von Kevin Xie und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104, United States. Hauptthemen laut MEDLINE: Healthcare Disparities, Seizures, Epilepsy. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NINDS NIH HHS, National Science Foundation Research, NIH HHS.
Abstract (Originalsprache)
Objective
Large-language models (LLMs) can potentially revolutionize health care delivery and research, but risk propagating existing biases or introducing new ones. In epilepsy, social determinants of health are associated with disparities in care access, but their impact on seizure outcomes among those with access remains unclear. Here we (1) evaluated our validated, epilepsy-specific LLM for intrinsic bias, and (2) used LLM-extracted seizure outcomes to determine if different demographic groups have different seizure outcomes. Materials and Methods We tested our LLM for differences and equivalences in prediction accuracy and confidence across demographic groups defined by race, ethnicity, sex, income, and health insurance, using manually annotated notes. Next, we used LLM-classified seizure freedom at each office visit to test for demographic outcome disparities, using univariable and multivariable analyses.
Results
We analyzed 84 675 clinic visits from 25 612 unique patients seen at our epilepsy center. We found little evidence of bias in the prediction accuracy or confidence of outcome classifications across demographic groups. Multivariable analysis indicated worse seizure outcomes for female patients (OR 1.33, P ≤ .001), those with public insurance (OR 1.53, P ≤ .001), and those from lower-income zip codes (OR ≥1.22, P ≤ .007). Black patients had worse outcomes than White patients in univariable but not multivariable analysis (OR 1.03, P = .66).
Conclusion
We found little evidence that our LLM was intrinsically biased against any demographic group. Seizure freedom extracted by LLM revealed disparities in seizure outcomes across several demographic groups. These findings quantify the critical need to reduce disparities in the care of people with epilepsy.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104, United States.
- Center for Neuroengineering and Therapeutics, University of Pennsylvania, Philadelphia, PA 19104, United States.
- Department of Neurology, University of Pennsylvania, Philadelphia, PA 19104, United States.
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA 19104, United States.
- Department of Neurology, University of Michigan, Ann Arbor, MI 48109, United States.
- Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, United States.
ORCID-Profile: Kevin Xie, William K S Ojemann, Colin A Ellis
Förderung
- NINDS NIH HHS
- National Science Foundation Research
- NIH HHS
- Mirowski Family Foundation
- American Academy of Neurology Susan S. Spencer Clinical Research Training Scholarship
- Office of Naval Research
- National Institute of Neurological Disorders and Stroke
- National Institutes of Health
Zitieren
APA
Xie, K., Ojemann, W. K. S., Gallagher, R. S., Shinohara, R. T., Lucas, A., Hill, C. E., Hamilton, R. H., Johnson, K. B., Roth, D., Litt, B., & Ellis, C. A. (2024). Disparities in seizure outcomes revealed by large language models. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocae047
Vancouver
Xie K, Ojemann WKS, Gallagher RS, Shinohara RT, Lucas A, Hill CE, et al. Disparities in seizure outcomes revealed by large language models. Journal of the American Medical Informatics Association. 2024. doi:10.1093/jamia/ocae047
BibTeX
@article{Xie2024,
author = {Xie, Kevin and Ojemann, William K S and Gallagher, Ryan S and Shinohara, Russell T and Lucas, Alfredo and Hill, Chloé E and Hamilton, Roy H and Johnson, Kevin B and Roth, Dan and Litt, Brian and Ellis, Colin A},
title = {Disparities in seizure outcomes revealed by large language models},
journal = {Journal of the American Medical Informatics Association},
year = {2024},
doi = {10.1093/jamia/ocae047}
}
EUVIMED