EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages

Matthew Farmer, Kimberly Powell, Mihail Popescu

Stacks Journal · 13.03.2026

DOI: 10.60102/stacks-26001 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Stacks Journal, online erschienen am 13.03.2026. Verfasst von Matthew Farmer und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Institutes of Health.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rapid improvement in generative language model capabilities and adoption has increased interest in their application for healthcare, particularly in extracting meaningful information and language from unstructured text. This study investigates the validity of Chain-of-Thought (CoT) prompting on a small language model’s ability to extract clinical concepts from nursing home staff text messages. We focus on concepts within the Age-Friendly Health Systems 4M framework (What Matters, Medication, Mentation, and Mobility). We evaluated the classification performance of four locally-run language models on an expert-annotated dataset of 860 text messages, using classification metrics to compare with or without CoT prompting. Our results demonstrate that CoT prompting improves the capabilities of Gemma 2 9b in extracting expressions related to the 4M framework. However, there was not an improvement observed in the Medication concept. CoT continues to demonstrate potential in improving language model capabilities in the clinical domain, particularly from unstructured text data. This has implications for ontology development, information retrieval, and clinical decision support. Additional contexts or data sources would improve the generalizability of these findings.

Förderung

  • National Institutes of Health

Zitieren

APA

Farmer, M., Powell, K., & Popescu, M. (2026). Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages. Stacks Journal. https://doi.org/10.60102/stacks-26001

Vancouver

Farmer M, Powell K, Popescu M. Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages. Stacks Journal. 2026. doi:10.60102/stacks-26001

BibTeX
@article{Farmer2026,
  author = {Farmer, Matthew and Powell, Kimberly and Popescu, Mihail},
  title = {Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages},
  journal = {Stacks Journal},
  year = {2026},
  doi = {10.60102/stacks-26001}
}
Kostenlos starten →