Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages
Stacks Journal · 13.03.2026
DOI: 10.60102/stacks-26001 ↗Kurz gefasst
Fachartikel in der Zeitschrift Stacks Journal, online erschienen am 13.03.2026. Verfasst von Matthew Farmer und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Institutes of Health.
Abstract (Originalsprache)
The rapid improvement in generative language model capabilities and adoption has increased interest in their application for healthcare, particularly in extracting meaningful information and language from unstructured text. This study investigates the validity of Chain-of-Thought (CoT) prompting on a small language model’s ability to extract clinical concepts from nursing home staff text messages. We focus on concepts within the Age-Friendly Health Systems 4M framework (What Matters, Medication, Mentation, and Mobility). We evaluated the classification performance of four locally-run language models on an expert-annotated dataset of 860 text messages, using classification metrics to compare with or without CoT prompting. Our results demonstrate that CoT prompting improves the capabilities of Gemma 2 9b in extracting expressions related to the 4M framework. However, there was not an improvement observed in the Medication concept. CoT continues to demonstrate potential in improving language model capabilities in the clinical domain, particularly from unstructured text data. This has implications for ontology development, information retrieval, and clinical decision support. Additional contexts or data sources would improve the generalizability of these findings.
Förderung
- National Institutes of Health
Zitieren
APA
Farmer, M., Powell, K., & Popescu, M. (2026). Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages. Stacks Journal. https://doi.org/10.60102/stacks-26001
Vancouver
Farmer M, Powell K, Popescu M. Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages. Stacks Journal. 2026. doi:10.60102/stacks-26001
BibTeX
@article{Farmer2026,
author = {Farmer, Matthew and Powell, Kimberly and Popescu, Mihail},
title = {Validation of Chain-of-Thought prompt structuring to improve a language model’s capability to extract new clinical concepts from clinical text messages},
journal = {Stacks Journal},
year = {2026},
doi = {10.60102/stacks-26001}
}
EUVIMED