EUVIMEDEUVIMED entdecken
Crossref

A Knowledge-Guided Approach to LLM-Assisted Government Performance Assessment

Taneth RUANGRAJITPAKORN, Wasan NA CHAI, Thepchai SUPNITHI

IEICE Transactions on Information and Systems · 2026

DOI: 10.1587/transinf.2026dkp0011 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEICE Transactions on Information and Systems, 2026. Verfasst von Taneth RUANGRAJITPAKORN und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have shown potential for evaluating textual information against predefined criteria, but criteria-based assessment often requires more than semantic interpretation. Evidence must be identified and judged against explicit conditions, while complex criteria may require sequential scoring and detailed procedural rules. This study presents an LLM-based approach for criteria-based assessment and feedback generation that separates evidence interpretation from procedural assessment. The approach uses instructional prompts and a procedural decision graph to represent assessment knowledge and applies LLMs to interpret organisational reports, performcriterion-level assessment, and generate reasoning and improvement feedback. PMQA 4.0 is used as a real-world case study involving heterogeneous organisational reports, multiple assessment criteria, and detailed binary and sequential scoring procedures. Evaluation on real assessment data showed that the system produced results generally consistent with human assessment across both binary and sequential scoring. Expert evaluation further indicated high correctness and relevance of the generated reasoning and feedback, with high inter-rater agreement among the assessors. Analysis of disagreements showed complementary knowledge boundaries between human and automated assessment. Human assessors could incorporate contextual knowledge beyond the submitted reports, while the system provided consistent application of explicitly represented assessment rules and calculations. The results demonstrate the potential of combining LLM-based evidence interpretation with explicit procedural knowledge for supporting complex criteria-based assessment while retaining human judgement for cases requiring contextual interpretation.

Zitieren

APA

RUANGRAJITPAKORN, T., CHAI, W. N., & SUPNITHI, T. (2026). A Knowledge-Guided Approach to LLM-Assisted Government Performance Assessment. IEICE Transactions on Information and Systems. https://doi.org/10.1587/transinf.2026dkp0011

Vancouver

RUANGRAJITPAKORN T, CHAI WN, SUPNITHI T. A Knowledge-Guided Approach to LLM-Assisted Government Performance Assessment. IEICE Transactions on Information and Systems. 2026. doi:10.1587/transinf.2026dkp0011

BibTeX
@article{RUANGRAJITPAKORN2026,
  author = {RUANGRAJITPAKORN, Taneth and CHAI, Wasan NA and SUPNITHI, Thepchai},
  title = {A Knowledge-Guided Approach to LLM-Assisted Government Performance Assessment},
  journal = {IEICE Transactions on Information and Systems},
  year = {2026},
  doi = {10.1587/transinf.2026dkp0011}
}
Kostenlos starten →