Prompt-Driven LLM Pipeline for Topic Modeling of Multiple Sclerosis Social Media Discussions
Applied Sciences · 26.05.2026 · 16 (11) · 5316
DOI: 10.3390/app16115316 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 26.05.2026. Verfasst von Yasmeen Alamoudi und 2 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: KAU Endowment (WAQF) at King Abdulaziz University.
Abstract (Originalsprache)
Despite advances in topic modeling, extracting coherent themes from short, noisy, health-related social media texts remains a methodological challenge. This paper presents a preliminary and exploratory empirical investigation of a prompt-driven Large Language Model (LLM) pipeline for topic modeling of 504 multiple sclerosis (MS)-related posts from Platform X. Two in-context learning strategies—zero-shot and few-shot prompting—were compared using GPT-4o-mini, with topic quality assessed through a multi-layered evaluation framework that leveraged GPT-4 for automated coherence and diversity scoring, complemented by human validation for coherence and document-level faithfulness. Within this experimental setting, few-shot prompting achieved the highest human agreement score (87.5%) alongside high coherence (4.9/5.0), while zero-shot prompting yielded the highest coherence score (5.0/5.0), with 79.2% human agreement. Both configurations demonstrated high thematic diversity (4.6/5.0) and produced themes largely judged as faithfully grounded in the source posts across the full corpus, with few-shot prompting demonstrating consistently stronger faithfulness and greater thematic stability. Nonetheless, these findings suggest that prompt-driven LLM pipelines — particularly few-shot approaches—show promise as a human-aligned and interpretable method for topic modeling of short, noisy, health-related social media texts.
Förderung
- KAU Endowment (WAQF) at King Abdulaziz University
Zitieren
APA
Alamoudi, Y., Babour, A., & Almatrafi, O. (2026). Prompt-Driven LLM Pipeline for Topic Modeling of Multiple Sclerosis Social Media Discussions. Applied Sciences, 16(11), 5316. https://doi.org/10.3390/app16115316
Vancouver
Alamoudi Y, Babour A, Almatrafi O. Prompt-Driven LLM Pipeline for Topic Modeling of Multiple Sclerosis Social Media Discussions. Applied Sciences. 2026;16(11):5316. doi:10.3390/app16115316
BibTeX
@article{Alamoudi2026,
author = {Alamoudi, Yasmeen and Babour, Amal and Almatrafi, Omaima},
title = {Prompt-Driven LLM Pipeline for Topic Modeling of Multiple Sclerosis Social Media Discussions},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {11},
pages = {5316},
doi = {10.3390/app16115316}
}
EUVIMED