The interpretation of topic models for scholarly analysis: An evaluation and critique of current practice
Digital Scholarship in the Humanities · 31.05.2023 · 38 (2) · 530-543
DOI: 10.1093/llc/fqac075 ↗Kurz gefasst
Fachartikel in der Zeitschrift Digital Scholarship in the Humanities, online erschienen am 22.12.2022. Verfasst von Mathew Gillings und 1 weiteren Autor:innen. Die Arbeit wurde bisher 47-mal zitiert (Crossref) und nennt 30 Literaturangaben. Gefördert von: Economic and Social Research Council, Centre for Corpus Approaches to Social Science.
Abstract (Originalsprache)
Topic modelling is a method of statistical data mining of a corpus of documents, popular in the digital humanities and, increasingly, in social sciences. A critical methodological issue is how ‘topics’ (groups of co-selected word types) can be interpreted in analytically meaningful terms. In the current literature, this is typically done by ‘eyeballing’; that is, cursory and largely unsystematic examination of the ‘top’ words in each algorithmically identified word group. We critically evaluate this approach in a dual analysis, comparing the ‘eyeballing’ approach with an alternative using sample close reading across the corpus. We used MALLET to extract two topic models from a test corpus: one with stopwords included, another with stopwords excluded. We then used the aforementioned methods to assign labels to these topics. The results suggest that a close-reading approach is more effective not only in level of detail but even in terms of accuracy. In particular, we found that: assigning labels via eyeballing yields incomplete or incorrect topic labels; removing stopwords drastically affects the analysis outcome; topic labelling and interpretation depend considerably on the analysts’ specialist knowledge; and differences of perspective or construal are unlikely to be captured through a topic model. We conclude that an interpretive paradigm founded in close reading may make topic modelling more appealing to humanities researchers.
Förderung
- Economic and Social Research Council
- Centre for Corpus Approaches to Social Science
Zitieren
APA
Gillings, M., & Hardie, A. (2023). The interpretation of topic models for scholarly analysis: An evaluation and critique of current practice. Digital Scholarship in the Humanities, 38(2), 530-543. https://doi.org/10.1093/llc/fqac075
Vancouver
Gillings M, Hardie A. The interpretation of topic models for scholarly analysis: An evaluation and critique of current practice. Digital Scholarship in the Humanities. 2023;38(2):530-543. doi:10.1093/llc/fqac075
BibTeX
@article{Gillings2023,
author = {Gillings, Mathew and Hardie, Andrew},
title = {The interpretation of topic models for scholarly analysis: An evaluation and critique of current practice},
journal = {Digital Scholarship in the Humanities},
year = {2023},
volume = {38},
number = {2},
pages = {530--543},
doi = {10.1093/llc/fqac075}
}
EUVIMED