Large Language Model-Based Automated Classification of Institutional IT Support Requests: A Multi-Channel Evaluation Against a Keyword Baseline
Future Internet · 14.11.2026 · 18 (11) · 626
DOI: 10.3390/fi18110626 ↗Kurz gefasst
Fachartikel in der Zeitschrift Future Internet, online erschienen am 14.11.2026. Verfasst von Byron David Aguilar-Tigre und 2 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universidad Tecnica de Machala.
Abstract (Originalsprache)
Support request management in public institutions is often fragmented, limiting oversight and management indicators. This study developed a natural language processing module for the automated classification of technical and information technology support requests, integrated into the institutional email system of a provincial government in Ecuador. Using a Design Science Research approach, the module was evaluated against a keyword-based baseline and a human consensus reference across three analyses: a primary corpus of institutional emails, a complementary validation of in-person requests, and an aggregate analysis of both channels. The module achieved significantly higher performance than the baseline in the email corpus (macro-F1 0.733 vs. 0.412; p = 0.0001) and in the aggregate analysis (macro-F1 0.713 vs. 0.510; p = 0.0002), while for in-person requests, no statistically significant difference was detected between the two methods (macro-F1 0.533 vs. 0.635; Δ = −0.101; 95% CI [−0.211, +0.120]; p = 0.221), with the keyword baseline nominally higher. Inter-rater agreement was substantial to almost perfect across subsets. These findings indicate that the module outperformed the implemented keyword baseline on institutional email. A supervised classifier and a second language model showed no detectable difference from the module on anonymized text.
Förderung
- Universidad Tecnica de Machala
Zitieren
APA
Aguilar-Tigre, B. D., Loja-Mora, N. M., & Célleri-Pacheco, J. (2026). Large Language Model-Based Automated Classification of Institutional IT Support Requests: A Multi-Channel Evaluation Against a Keyword Baseline. Future Internet, 18(11), 626. https://doi.org/10.3390/fi18110626
Vancouver
Aguilar-Tigre BD, Loja-Mora NM, Célleri-Pacheco J. Large Language Model-Based Automated Classification of Institutional IT Support Requests: A Multi-Channel Evaluation Against a Keyword Baseline. Future Internet. 2026;18(11):626. doi:10.3390/fi18110626
BibTeX
@article{AguilarTigre2026,
author = {Aguilar-Tigre, Byron David and Loja-Mora, Nancy Magaly and Célleri-Pacheco, Jennifer},
title = {Large Language Model-Based Automated Classification of Institutional IT Support Requests: A Multi-Channel Evaluation Against a Keyword Baseline},
journal = {Future Internet},
year = {2026},
volume = {18},
number = {11},
pages = {626},
doi = {10.3390/fi18110626}
}
EUVIMED