Evaluation of DistilBERT and BiLSTM Models for the Development of Islamic Chatbots Based on Tag Classification
Khazanah Journal of Religion and Technology · 29.06.2026 · 4 (1) · 7-14
DOI: 10.66948/kjrt.41.51 ↗Kurz gefasst
Fachartikel in der Zeitschrift Khazanah Journal of Religion and Technology, online erschienen am 29.06.2026. Verfasst von Muhammad Rizki Al-Fathir und 4 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study evaluates the performance of DistilBERT and Bidirectional Long Short-Term Memory (BiLSTM) models for intent classification in Islamic chatbots, with the main challenge being a highly imbalanced dataset containing 2,031 unique intents. Following the CRISP-DM methodology, the DistilBERT model was fine-tuned using Focal Loss to address class imbalance, while the BiLSTM model was built from scratch with a standard loss function. The evaluation results demonstrated the absolute superiority of DistilBERT, achieving an accuracy of 65.15%, far surpassing BiLSTM, which achieved only 34.50% due to severe overfitting. Although the final model sizes of both were similar, DistilBERT training proved to be significantly more efficient. These findings demonstrate that a Transformer-based architecture combined with an appropriate strategy, such as Focal Loss, is a much more robust and effective solution for large-scale, imbalanced text classification in specific domains. The practical feasibility of this approach was validated through its successful implementation in a publicly accessible, functional chatbot prototype.
Zitieren
APA
Al-Fathir, M. R., 'Ijazi, M. S., Lailatanzila, N., Ridlo, N. R., & Fadil, R. A. (2026). Evaluation of DistilBERT and BiLSTM Models for the Development of Islamic Chatbots Based on Tag Classification. Khazanah Journal of Religion and Technology, 4(1), 7-14. https://doi.org/10.66948/kjrt.41.51
Vancouver
Al-Fathir MR, 'Ijazi MS, Lailatanzila N, Ridlo NR, Fadil RA. Evaluation of DistilBERT and BiLSTM Models for the Development of Islamic Chatbots Based on Tag Classification. Khazanah Journal of Religion and Technology. 2026;4(1):7-14. doi:10.66948/kjrt.41.51
BibTeX
@article{AlFathir2026,
author = {Al-Fathir, Muhammad Rizki and 'Ijazi, Muhammad Saifurridwani and Lailatanzila, Nabila and Ridlo, Nirwan Rasyid and Fadil, Riza Anwar},
title = {Evaluation of DistilBERT and BiLSTM Models for the Development of Islamic Chatbots Based on Tag Classification},
journal = {Khazanah Journal of Religion and Technology},
year = {2026},
volume = {4},
number = {1},
pages = {7--14},
doi = {10.66948/kjrt.41.51}
}
EUVIMED