Vision-Based Human–Computer Interaction System for Mexican Sign Language Recognition and Robotic Reproduction: Toward Inclusive Educational Applications
Sensors · 05.11.2026 · 26 (21) · 6990
DOI: 10.3390/s26216990 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 05.11.2026. Verfasst von David Alexis Cano-Ibarra und 6 weiteren Autor:innen. Beteiligte Einrichtung: Laboratorio de Optomecatrónica, UPIIH, Instituto Politécnico Nacional, Distrito de Educación, Salud, Ciencia, Tecnología e Innovación, San Agustín Tl…. Hauptthemen laut MEDLINE: Sign Language, Robotics, User-Computer Interface. Die Arbeit nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Programa de Iniciación en Proyectos de Investigación, Instituto Politécnico Nacional.
Abstract (Originalsprache)
Mexican Sign Language (MSL) plays an essential role in communication within the Deaf community, yet access to sign-language learning resources remains limited in some educational settings. This study presents a vision-based Human–Computer Interaction (HCI) system for MSL recognition and robotic reproduction, developed as a proof of concept toward inclusive educational applications. The system uses an RGB camera and MediaPipe Hands for hand-landmark extraction, a multilayer perceptron (MLP) for recognition of 29 static classes, and a TimeDistributed–Long Short-Term Memory (LSTM) model for 10 dynamic classes. Recognized signs are mapped to predefined motion sequences executed by an ATmega2560-controlled robotic arm. Under stratified random and sequential partitioning protocols, the static model achieved accuracies of 100.00% and 99.88%, respectively, while the dynamic model achieved 100.00% under both protocols. Additional comparisons with Support Vector Machine (SVM) and Random Forest classifiers, using three random seeds, demonstrated competitive performance and low variability within the original single-participant dataset. To assess generalization beyond the training participant, a preliminary signer-independent evaluation was conducted with three previously unseen participants, without model retraining or adaptation. Across 585 recognition attempts involving 39 classes, the system achieved an overall accuracy of 89.06%, providing initial evidence of recognition performance across different signers. Furthermore, 92 accepted recognition trials successfully triggered their corresponding robotic sequences, demonstrating functional integration between recognition and actuation. These findings support the technical feasibility of the proposed contactless HCI architecture as a potential foundation for interactive MSL learning tools. Nevertheless, broader signer-independent validation and user-centered evaluations are required to establish its accessibility, usability, and educational effectiveness.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Laboratorio de Optomecatrónica, UPIIH, Instituto Politécnico Nacional, Distrito de Educación, Salud, Ciencia, Tecnología e Innovación, San Agustín Tlaxiaca 42162, Mexico.
- Unidad Profesional Interdisciplinaria de Energía y Movilidad, Instituto Politécnico Nacional, Av. Wilfrido Massieu, Adolfo López Mateos S/N, Nueva Industrial Vallejo, Gustavo A. Madero, Mexico City 07738, Mexico.
ORCID-Profile: David Alexis Cano-Ibarra, Wendy Flores-Carmona, Karen Roa-Tort, Diego A Fabila-Bustos, Macaria Hernández-Chávez, J Alejandro Aguirre-Anaya, Josué D Rivera-Fernández
Förderung
- Programa de Iniciación en Proyectos de Investigación
- Instituto Politécnico Nacional
Zitieren
APA
Cano-Ibarra, D. A., Flores-Carmona, W., Roa-Tort, K., Fabila-Bustos, D. A., Hernández-Chávez, M., Aguirre-Anaya, J. A., & Rivera-Fernández, J. D. (2026). Vision-Based Human–Computer Interaction System for Mexican Sign Language Recognition and Robotic Reproduction: Toward Inclusive Educational Applications. Sensors, 26(21), 6990. https://doi.org/10.3390/s26216990
Vancouver
Cano-Ibarra DA, Flores-Carmona W, Roa-Tort K, Fabila-Bustos DA, Hernández-Chávez M, Aguirre-Anaya JA, et al. Vision-Based Human–Computer Interaction System for Mexican Sign Language Recognition and Robotic Reproduction: Toward Inclusive Educational Applications. Sensors. 2026;26(21):6990. doi:10.3390/s26216990
BibTeX
@article{CanoIbarra2026,
author = {Cano-Ibarra, David Alexis and Flores-Carmona, Wendy and Roa-Tort, Karen and Fabila-Bustos, Diego A and Hernández-Chávez, Macaria and Aguirre-Anaya, J Alejandro and Rivera-Fernández, Josué D},
title = {Vision-Based Human–Computer Interaction System for Mexican Sign Language Recognition and Robotic Reproduction: Toward Inclusive Educational Applications},
journal = {Sensors},
year = {2026},
volume = {26},
number = {21},
pages = {6990},
doi = {10.3390/s26216990}
}
EUVIMED