EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Exploiting Contextual Embeddings in Hierarchical Topic Modeling and Investigating the Limits of the Current Evaluation Metrics

Felipe Viegas, Antonio Pereira, Washington Cunha, Celso França, Claudio Andrade, Elisa Tuler, Leonardo Rocha, Marcos André Gonçalves

Computational Linguistics · 01.09.2025 · 51 (3) · 843-883

DOI: 10.1162/coli_a_00543 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computational Linguistics, online erschienen am 01.09.2025. Verfasst von Felipe Viegas und 7 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We investigate two essential challenges in the context of hierarchical topic modeling (HTM)—(i) the impact of data representation and (ii) topic evaluation. The data representation directly influences the performance of the topic generation, and the impact of new representations such as contextual embeddings in this task has been under-investigated. Topic evaluation, responsible for driving the advances in the field, assesses the overall quality of the topic generation process. HTM studies exploit the exact topic modeling (TM) evaluation metrics as traditional TM to measure the quality of topics. One significant result of our work is demonstrating that the HTM’s hierarchical nature demands novel ways of evaluating the quality of topics. As our main contribution, we propose two new topic quality metrics to assess the topical quality of the hierarchical structures. Uniqueness considers topic topological consistency, while the Semantic Hierarchical Structure (SHS) captures the semantic relatedness of the hierarchies. We also present an additional advance to the state-of-the-art by proposing the c-CluHTM. To the best of our knowledge, c-CluHTM is the first method that exploits contextual embeddings into NMF in HTM tasks. c-CluHTM enhances the topics’ semantics while preserving the hierarchical structure. We perform an experimental evaluation, and our results demonstrate the superiority of our proposal with gains between 12% and 21%, regarding NPMI and Coherence over the best baselines. Regarding the newly proposed metrics, our results reveal that Uniqueness and SHS can capture relevant information about the structure of the hierarchical topics that traditional metrics cannot.

Zitieren

APA

Viegas, F., Pereira, A., Cunha, W., França, C., Andrade, C., Tuler, E., Rocha, L., & Gonçalves, M. A. (2025). Exploiting Contextual Embeddings in Hierarchical Topic Modeling and Investigating the Limits of the Current Evaluation Metrics. Computational Linguistics, 51(3), 843-883. https://doi.org/10.1162/coli_a_00543

Vancouver

Viegas F, Pereira A, Cunha W, França C, Andrade C, Tuler E, et al. Exploiting Contextual Embeddings in Hierarchical Topic Modeling and Investigating the Limits of the Current Evaluation Metrics. Computational Linguistics. 2025;51(3):843-883. doi:10.1162/coli_a_00543

BibTeX
@article{Viegas2025,
  author = {Viegas, Felipe and Pereira, Antonio and Cunha, Washington and França, Celso and Andrade, Claudio and Tuler, Elisa and Rocha, Leonardo and Gonçalves, Marcos André},
  title = {Exploiting Contextual Embeddings in Hierarchical Topic Modeling and Investigating the Limits of the Current Evaluation Metrics},
  journal = {Computational Linguistics},
  year = {2025},
  volume = {51},
  number = {3},
  pages = {843--883},
  doi = {10.1162/coli_a_00543}
}
Kostenlos starten →