EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing

Agustina Ginting, Harris Aminuddin, Ulfa Hasnita

Information Technology Education Journal · 13.06.2026 · 402-412

DOI: 10.59562/intec.v5i2.13018 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information Technology Education Journal, online erschienen am 13.06.2026. Verfasst von Agustina Ginting und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose – This study proposes a conceptual framework for AI-based detection of handwritten and technical English writing in engineering education by investigating how different writing modalities influence AI detection performance. Design/methodology/approach – The proposed framework integrates four representative writing modalities, namely handwritten, typed, hybrid, and transcribed texts. A framework-based analytical evaluation was conducted using representative detection scores together with accuracy, precision, recall, F1-score, confidence intervals, and heatmap visualization to compare AI detection performance across writing modalities. Findings – The analytical evaluation suggests that AI detection performance varies across writing modalities, with representative accuracy ranging from 0.72 to 0.89. Detection scores progressively increase from handwritten (0.41–0.50) to typed (0.52–0.58), hybrid (0.60–0.66), and transcribed texts (0.68–0.76). Transcribed texts demonstrate the highest performance, with precision and recall reaching 0.87–0.89, whereas handwritten texts exhibit lower recall (0.44) and broader confidence intervals, indicating greater classification uncertainty. Hybrid writing shows intermediate performance due to overlapping human and AI-assisted linguistic characteristics. Research implications/limitations – This study is limited to a framework-based analytical evaluation using representative writing scenarios rather than empirical observations. Nevertheless, the proposed framework provides practical guidance for developing context-aware AI detection systems that improve fairness and reliability in evaluating technical and English as a Second Language (ESL) writing. Originality/value – This research presents an integrated analytical framework that combines multiple writing modalities with AI detection performance evaluation to investigate modality-dependent differences in technical English writing. The proposed framework contributes to the development of more robust, context-aware, and equitable AI-assisted writing assessment for engineering education.

Zitieren

APA

Ginting, A., Aminuddin, H., & Hasnita, U. (2026). Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing. Information Technology Education Journal, 402-412. https://doi.org/10.59562/intec.v5i2.13018

Vancouver

Ginting A, Aminuddin H, Hasnita U. Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing. Information Technology Education Journal. 2026:402-412. doi:10.59562/intec.v5i2.13018

BibTeX
@article{Ginting2026,
  author = {Ginting, Agustina and Aminuddin, Harris and Hasnita, Ulfa},
  title = {Analytical Framework for Evaluating AI Detection Performance Across Writing Modalities in Technical English Writing},
  journal = {Information Technology Education Journal},
  year = {2026},
  pages = {402--412},
  doi = {10.59562/intec.v5i2.13018}
}
Kostenlos starten →