Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing
Information Technology Education Journal · 13.06.2026 · 402-412
DOI: 10.59562/intec.v5i2.13018 ↗Kurz gefasst
Fachartikel in der Zeitschrift Information Technology Education Journal, online erschienen am 13.06.2026. Verfasst von Agustina Ginting und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Purpose – This study proposes a conceptual framework for AI-based detection of handwritten and technical English writing in engineering education by investigating how different writing modalities influence AI detection performance. Design/methodology/approach – The proposed framework integrates four representative writing modalities, namely handwritten, typed, hybrid, and transcribed texts. A framework-based analytical evaluation was conducted using representative detection scores together with accuracy, precision, recall, F1-score, confidence intervals, and heatmap visualization to compare AI detection performance across writing modalities. Findings – The analytical evaluation suggests that AI detection performance varies across writing modalities, with representative accuracy ranging from 0.72 to 0.89. Detection scores progressively increase from handwritten (0.41–0.50) to typed (0.52–0.58), hybrid (0.60–0.66), and transcribed texts (0.68–0.76). Transcribed texts demonstrate the highest performance, with precision and recall reaching 0.87–0.89, whereas handwritten texts exhibit lower recall (0.44) and broader confidence intervals, indicating greater classification uncertainty. Hybrid writing shows intermediate performance due to overlapping human and AI-assisted linguistic characteristics. Research implications/limitations – This study is limited to a framework-based analytical evaluation using representative writing scenarios rather than empirical observations. Nevertheless, the proposed framework provides practical guidance for developing context-aware AI detection systems that improve fairness and reliability in evaluating technical and English as a Second Language (ESL) writing. Originality/value – This research presents an integrated analytical framework that combines multiple writing modalities with AI detection performance evaluation to investigate modality-dependent differences in technical English writing. The proposed framework contributes to the development of more robust, context-aware, and equitable AI-assisted writing assessment for engineering education.
Zitieren
APA
Ginting, A., Aminuddin, H., & Hasnita, U. (2026). Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing. Information Technology Education Journal, 402-412. https://doi.org/10.59562/intec.v5i2.13018
Vancouver
Ginting A, Aminuddin H, Hasnita U. Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing. Information Technology Education Journal. 2026:402-412. doi:10.59562/intec.v5i2.13018
BibTeX
@article{Ginting2026,
author = {Ginting, Agustina and Aminuddin, Harris and Hasnita, Ulfa},
title = {Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing},
journal = {Information Technology Education Journal},
year = {2026},
pages = {402--412},
doi = {10.59562/intec.v5i2.13018}
}
EUVIMED