EUVIMEDEUVIMED entdecken
Open Access

Disparities in seizure outcomes revealed by large language models

Kevin Xie iD, William K S Ojemann iD, Ryan S Gallagher, Russell T Shinohara, Alfredo Lucas, Chloé E Hill, Roy H Hamilton, Kevin B Johnson, Dan Roth, Brian Litt, Colin A Ellis iD

Journal of the American Medical Informatics Association · 2024

DOI: 10.1093/jamia/ocae047 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2024. Verfasst von Kevin Xie und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104, United States. Hauptthemen laut MEDLINE: Healthcare Disparities, Seizures, Epilepsy. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NINDS NIH HHS, National Science Foundation Research, NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Large-language models (LLMs) can potentially revolutionize health care delivery and research, but risk propagating existing biases or introducing new ones. In epilepsy, social determinants of health are associated with disparities in care access, but their impact on seizure outcomes among those with access remains unclear. Here we (1) evaluated our validated, epilepsy-specific LLM for intrinsic bias, and (2) used LLM-extracted seizure outcomes to determine if different demographic groups have different seizure outcomes. Materials and Methods We tested our LLM for differences and equivalences in prediction accuracy and confidence across demographic groups defined by race, ethnicity, sex, income, and health insurance, using manually annotated notes. Next, we used LLM-classified seizure freedom at each office visit to test for demographic outcome disparities, using univariable and multivariable analyses.

Results

We analyzed 84 675 clinic visits from 25 612 unique patients seen at our epilepsy center. We found little evidence of bias in the prediction accuracy or confidence of outcome classifications across demographic groups. Multivariable analysis indicated worse seizure outcomes for female patients (OR 1.33, P ≤ .001), those with public insurance (OR 1.53, P ≤ .001), and those from lower-income zip codes (OR ≥1.22, P ≤ .007). Black patients had worse outcomes than White patients in univariable but not multivariable analysis (OR 1.03, P = .66).

Conclusion

We found little evidence that our LLM was intrinsically biased against any demographic group. Seizure freedom extracted by LLM revealed disparities in seizure outcomes across several demographic groups. These findings quantify the critical need to reduce disparities in the care of people with epilepsy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Healthcare DisparitiesSeizuresEpilepsyHumansFemaleMaleAdultMiddle AgedNatural Language ProcessingSocial Determinants of HealthAdolescentYoung AdultLanguage

Stichwörter der Autor:innen

clinical informaticselectronic health recordhealth disparitiesnatural language processing

Autor:innen und Institutionen

  • Department of Bioengineering, University of Pennsylvania, Philadelphia, PA 19104, United States.
  • Center for Neuroengineering and Therapeutics, University of Pennsylvania, Philadelphia, PA 19104, United States.
  • Department of Neurology, University of Pennsylvania, Philadelphia, PA 19104, United States.
  • Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA 19104, United States.
  • Department of Neurology, University of Michigan, Ann Arbor, MI 48109, United States.
  • Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, United States.

ORCID-Profile: Kevin Xie, William K S Ojemann, Colin A Ellis

Förderung

  • NINDS NIH HHS
  • National Science Foundation Research
  • NIH HHS
  • Mirowski Family Foundation
  • American Academy of Neurology Susan S. Spencer Clinical Research Training Scholarship
  • Office of Naval Research
  • National Institute of Neurological Disorders and Stroke
  • National Institutes of Health

Zitieren

APA

Xie, K., Ojemann, W. K. S., Gallagher, R. S., Shinohara, R. T., Lucas, A., Hill, C. E., Hamilton, R. H., Johnson, K. B., Roth, D., Litt, B., & Ellis, C. A. (2024). Disparities in seizure outcomes revealed by large language models. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocae047

Vancouver

Xie K, Ojemann WKS, Gallagher RS, Shinohara RT, Lucas A, Hill CE, et al. Disparities in seizure outcomes revealed by large language models. Journal of the American Medical Informatics Association. 2024. doi:10.1093/jamia/ocae047

BibTeX
@article{Xie2024,
  author = {Xie, Kevin and Ojemann, William K S and Gallagher, Ryan S and Shinohara, Russell T and Lucas, Alfredo and Hill, Chloé E and Hamilton, Roy H and Johnson, Kevin B and Roth, Dan and Litt, Brian and Ellis, Colin A},
  title = {Disparities in seizure outcomes revealed by large language models},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  doi = {10.1093/jamia/ocae047}
}
Kostenlos starten →