EUVIMEDEUVIMED entdecken
VergleichsstudieValidierungsstudieOpen Access

Efficient Detection of Stigmatizing Language in Electronic Health Records via In-Context Learning: Comparative Analysis and Validation Study

Hongbo Chen iD, Myrtede Alfred iD, Eldan Cohen iD

JMIR Medical Informatics · 18.08.2025 · 13 · e68955

DOI: 10.2196/68955 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 18.08.2025. Verfasst von Hongbo Chen und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada. Hauptthemen laut MEDLINE: Electronic Health Records, Machine Learning, Language, Social Stigma. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 107 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The presence of stigmatizing language within electronic health records (EHRs) poses significant risks to patient care by perpetuating biases. While numerous studies have explored the use of supervised machine learning models to detect stigmatizing language automatically, these models require large, annotated datasets, which may not always be readily available. In-context learning (ICL) has emerged as a data-efficient alternative, allowing large language models to adapt to tasks using only instructions and examples.

Objective

We aimed to investigate the efficacy of ICL in detecting stigmatizing language within EHRs under data-scarce conditions.

Methods

We analyzed 5043 sentences from the Medical Information Mart for Intensive Care–IV dataset, which contains EHRs from patients admitted to the emergency department at the Beth Israel Deaconess Medical Center. We compared ICL with zero-shot (textual entailment), few-shot (SetFit), and supervised fine-tuning approaches. The ICL approach used 4 prompting strategies: generic, chain of thought, clue and reasoning prompting, and a newly introduced stigma detection guided prompt. Model fairness was evaluated using the equal performance criterion, measuring true positive rate, false positive rate, and F1-score disparities across protected attributes, including sex, age, and race.

Results

In the zero-shot setting, the best-performing ICL model, GEMMA-2, achieved a mean F1-score of 0.858 (95% CI 0.854-0.862), showing an 18.7% improvement over the best textual entailment model, DEBERTA-M (mean F1-score 0.723, 95% CI 0.718-0.728; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Electronic Health RecordsMachine LearningLanguageSocial StigmaStereotypingHumansMaleFemale

Stichwörter der Autor:innen

artificial intelligenceelectronic health recordfairnessfew-shotin-context learninglarge language modelmachine learningprompting strategystigmatizing languagetext classificationzero-shot

Autor:innen und Institutionen

  • Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON, Canada.

ORCID-Profile: Hongbo Chen, Myrtede Alfred, Eldan Cohen

Zitieren

APA

Chen, H., Alfred, M., & Cohen, E. (2025). Efficient Detection of Stigmatizing Language in Electronic Health Records via In-Context Learning: Comparative Analysis and Validation Study. JMIR Medical Informatics, 13, e68955. https://doi.org/10.2196/68955

Vancouver

Chen H, Alfred M, Cohen E. Efficient Detection of Stigmatizing Language in Electronic Health Records via In-Context Learning: Comparative Analysis and Validation Study. JMIR Medical Informatics. 2025;13:e68955. doi:10.2196/68955

BibTeX
@article{Chen2025,
  author = {Chen, Hongbo and Alfred, Myrtede and Cohen, Eldan},
  title = {Efficient Detection of Stigmatizing Language in Electronic Health Records via In-Context Learning: Comparative Analysis and Validation Study},
  journal = {JMIR Medical Informatics},
  year = {2025},
  volume = {13},
  pages = {e68955},
  doi = {10.2196/68955}
}
Kostenlos starten →