SPBEBOT: A Hierarchy-Aware Retrieval-Augmented Generation Framework for Evidence-Based Audit Assessment of Indonesian Electronic-Based Government
Journal of Artificial Intelligence and Data Science · 2026
DOI: 10.67561/jaids.v2i1.5 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence and Data Science, 2026. Verfasst von Regi Muhammar und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Electronic-Based Government System (SPBE) has emerged for Indonesia’s digital transformation in public services, as regulated by Presidential Regulation No. 95 of 2018. The implementation of SPBE is evaluated annually by assessors in accordance with the Regulation of the Ministry of Administrative and Bureaucratic Reform (MABR) No. 3 of 2024 on “Pedoman Tata Cara Pemantauan dan Evaluasi SPBE.” This guideline defines an evaluation framework consisting of 4 domains (Policy, Governance, Management, and Services), 8 aspects, and 47 indicators, each assessed across 5 maturity levels and followed by documentary evidence. Due to the hierarchical structure and large volume of regulatory documents, assessors still rely on manual processes to retrieve information and interpret assessment criteria, making the process time-consuming. This paper introduces SPBEBOT, a chatbot powered by a hierarchy-aware Vector Retrieval-Augmented Generation (RAG) approach to help SPBE assessors access and interpret evaluation guidelines. The system implements hierarchical document splitting, parent document retrieval, multi-query expansion, and prompt routing to provide the SPBE information and audit assessment tasks. The system's performance is evaluated using the Retrieval-Augmented Generation Assessment (RAGAS) and Meta’s Llama 3.3 70b model, which achieves 83% Context Recall, 68% Context Precision, 72% Faithfulness, and 81% Answer Relevancy. Additionally, testing by external SPBE assessors obtained 84.6% user satisfaction. These results indicate that SPBEBOT can help assessors work more efficiently, but the system still needs to be used together with expert judgment, especially when indicators look similar.
Zitieren
APA
Muhammar, R., Yustanti, W., & Widada, B. P. P. U. (2026). SPBEBOT: A Hierarchy-Aware Retrieval-Augmented Generation Framework for Evidence-Based Audit Assessment of Indonesian Electronic-Based Government. Journal of Artificial Intelligence and Data Science. https://doi.org/10.67561/jaids.v2i1.5
Vancouver
Muhammar R, Yustanti W, Widada BPPU. SPBEBOT: A Hierarchy-Aware Retrieval-Augmented Generation Framework for Evidence-Based Audit Assessment of Indonesian Electronic-Based Government. Journal of Artificial Intelligence and Data Science. 2026. doi:10.67561/jaids.v2i1.5
BibTeX
@article{Muhammar2026,
author = {Muhammar, Regi and Yustanti, Wiyli and Widada, Bartolomeus Priya Perkasa Utama},
title = {SPBEBOT: A Hierarchy-Aware Retrieval-Augmented Generation Framework for Evidence-Based Audit Assessment of Indonesian Electronic-Based Government},
journal = {Journal of Artificial Intelligence and Data Science},
year = {2026},
doi = {10.67561/jaids.v2i1.5}
}
EUVIMED