Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT
International Journal on Advanced Science, Engineering and Information Technology · 17.06.2024 · 14 (3) · 912-919
DOI: 10.18517/ijaseit.14.3.19347 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal on Advanced Science, Engineering and Information Technology, online erschienen am 17.06.2024. Verfasst von Maresha Caroline Wijanto und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Topic modeling has emerged as a successful approach to uncovering topics from textual data. Various topic modeling techniques have been introduced, ranging from traditional algorithms to those based on neural networks. In this research, we explore advanced topic modeling techniques, including BERT-based approaches, to enhance the analysis of scientific articles. We first investigate a widely used Latent Dirichlet Allocation (LDA) model and then explore the capabilities of BERT, to automatically uncover latent topics within scientific papers. The goal of this study is to identify the optimal hyperparameter setting for BERT-based topic modeling of scientific articles. We conduct experiments across several scenarios involving combinations of word embedding, dimension reduction, and clustering methods. The results were analyzed based on the coherence values, average execution time, number of topics generated, visualization through the inter-topic distance map, and the top-N-words of each topic. Our findings suggest that combination of RoBERTa for word embedding, PCA for dimension reduction, and K-Means for clustering yields superior results among the tested scenarios. Further evaluation of BERT-based topic modeling is necessary to validate these findings and explore its applications in various academic and industrial contexts. The implications of these advanced techniques could significantly streamline the process of staying updated with scientific literature, potentially revolutionizing research methodologies across disciplines.
Zitieren
APA
Wijanto, M. C., Widiastuti, I., & Yong, H. (2024). Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT. International Journal on Advanced Science, Engineering and Information Technology, 14(3), 912-919. https://doi.org/10.18517/ijaseit.14.3.19347
Vancouver
Wijanto MC, Widiastuti I, Yong H. Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT. International Journal on Advanced Science, Engineering and Information Technology. 2024;14(3):912-919. doi:10.18517/ijaseit.14.3.19347
BibTeX
@article{Wijanto2024,
author = {Wijanto, Maresha Caroline and Widiastuti, Ika and Yong, Hwan-Seung},
title = {Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT},
journal = {International Journal on Advanced Science, Engineering and Information Technology},
year = {2024},
volume = {14},
number = {3},
pages = {912--919},
doi = {10.18517/ijaseit.14.3.19347}
}
EUVIMED