Evaluation of unsupervised static topic models’ emergence detection ability
PeerJ Computer Science · 30.05.2025 · 11 · e2875
DOI: 10.7717/peerj-cs.2875 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 30.05.2025. Verfasst von Xue Li und 5 weiteren Autor:innen. Beteiligte Einrichtung: Informatics Institute, University of Amsterdam, Amsterdam, Netherlands. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Dutch Research Council.
Abstract (Originalsprache)
Detecting emerging topics is crucial for understanding research trends, technological advancements, and shifts in public discourse. While unsupervised topic modeling techniques such as Latent Dirichlet allocation (LDA), BERTopic, and CoWords clustering are widely used for topic extraction, their ability to retrospectively detect emerging topics without relying on ground truth labels has not been systematically compared. This gap largely stems from the lack of a dedicated evaluation metric for measuring emergence detection. In this study, we introduce a quantitative evaluation metric to assess the effectiveness of topic models in detecting emerging topics. We evaluate three topic modeling approaches using both qualitative analysis and our proposed emergence detection metric. Our results indicate that, qualitatively, CoWords identifies emerging topics earlier than LDA and BERTopics. Quantitatively, our evaluation metric demonstrates that LDA achieves an average F1 score of 80.6% in emergence detection, outperforming BERTopic by 24.0%. These findings highlight the strengths and limitations of different topic models for emergence detection, while our proposed metric provides a robust framework for future benchmarking in this area.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Informatics Institute, University of Amsterdam, Amsterdam, Netherlands.
- Business School, University of Amsterdam, Amsterdam, Netherlands.
- University of Johannesburg, Johannesburg, South Africa.
ORCID-Profile: Xue Li, Ciro D Esposito, Paul Groth
Förderung
- Dutch Research Council
Zitieren
APA
Li, X., Esposito, C. D., Groth, P., Sitruk, J., Szatmari, B., & Wijnberg, N. (2025). Evaluation of unsupervised static topic models’ emergence detection ability. PeerJ Computer Science, 11, e2875. https://doi.org/10.7717/peerj-cs.2875
Vancouver
Li X, Esposito CD, Groth P, Sitruk J, Szatmari B, Wijnberg N. Evaluation of unsupervised static topic models’ emergence detection ability. PeerJ Computer Science. 2025;11:e2875. doi:10.7717/peerj-cs.2875
BibTeX
@article{Li2025,
author = {Li, Xue and Esposito, Ciro D and Groth, Paul and Sitruk, Jonathan and Szatmari, Balazs and Wijnberg, Nachoem},
title = {Evaluation of unsupervised static topic models’ emergence detection ability},
journal = {PeerJ Computer Science},
year = {2025},
volume = {11},
pages = {e2875},
doi = {10.7717/peerj-cs.2875}
}
EUVIMED