EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT

Maresha Caroline Wijanto, Ika Widiastuti, Hwan-Seung Yong

International Journal on Advanced Science, Engineering and Information Technology · 17.06.2024 · 14 (3) · 912-919

DOI: 10.18517/ijaseit.14.3.19347 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal on Advanced Science, Engineering and Information Technology, online erschienen am 17.06.2024. Verfasst von Maresha Caroline Wijanto und 2 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Topic modeling has emerged as a successful approach to uncovering topics from textual data. Various topic modeling techniques have been introduced, ranging from traditional algorithms to those based on neural networks. In this research, we explore advanced topic modeling techniques, including BERT-based approaches, to enhance the analysis of scientific articles. We first investigate a widely used Latent Dirichlet Allocation (LDA) model and then explore the capabilities of BERT, to automatically uncover latent topics within scientific papers. The goal of this study is to identify the optimal hyperparameter setting for BERT-based topic modeling of scientific articles. We conduct experiments across several scenarios involving combinations of word embedding, dimension reduction, and clustering methods. The results were analyzed based on the coherence values, average execution time, number of topics generated, visualization through the inter-topic distance map, and the top-N-words of each topic. Our findings suggest that combination of RoBERTa for word embedding, PCA for dimension reduction, and K-Means for clustering yields superior results among the tested scenarios. Further evaluation of BERT-based topic modeling is necessary to validate these findings and explore its applications in various academic and industrial contexts. The implications of these advanced techniques could significantly streamline the process of staying updated with scientific literature, potentially revolutionizing research methodologies across disciplines.

Zitieren

APA

Wijanto, M. C., Widiastuti, I., & Yong, H. (2024). Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT. International Journal on Advanced Science, Engineering and Information Technology, 14(3), 912-919. https://doi.org/10.18517/ijaseit.14.3.19347

Vancouver

Wijanto MC, Widiastuti I, Yong H. Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT. International Journal on Advanced Science, Engineering and Information Technology. 2024;14(3):912-919. doi:10.18517/ijaseit.14.3.19347

BibTeX
@article{Wijanto2024,
  author = {Wijanto, Maresha Caroline and Widiastuti, Ika and Yong, Hwan-Seung},
  title = {Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT},
  journal = {International Journal on Advanced Science, Engineering and Information Technology},
  year = {2024},
  volume = {14},
  number = {3},
  pages = {912--919},
  doi = {10.18517/ijaseit.14.3.19347}
}
Kostenlos starten →