A Vision-Based Assistive Robotic System with Real-Time Gesture Recognition for Communication Support in Speech-Impaired Cancer Patients: A Pilot Feasibility Study
Faculty of Natural and Applied Sciences Journal of Computing and Applications · 31.03.2026 · 3 (1) · 19-34
DOI: 10.63561/jca.v3i1.1199 ↗Kurz gefasst
Fachartikel in der Zeitschrift Faculty of Natural and Applied Sciences Journal of Computing and Applications, online erschienen am 31.03.2026. Verfasst von Abugor Ejaita Okpako und 4 weiteren Autor:innen.
Themen: Onkologische Pflege
Abstract (Originalsprache)
Assistive robotics in healthcare frequently lacks seamless integration between human-robot interaction (HRI) and diagnostic support. This challenge is especially pronounced for speech-impaired cancer patients (e.g., those with head and neck, oral, or laryngeal cancer, or post-treatment voice loss), who often face significant barriers in non-verbal communication and control of medical interfaces. This pilot feasibility study presents a vision-based assistive robotic system that combines gesture-driven HRI with preliminary histopathology-based cancer detection in a closed-loop architecture. I propose a Cross-Domain Adaptive Multi-Task Network (CDAM-Net) that mitigates negative transfer between heterogeneous visual domains natural hand gestures and microscopic tissue textures through domainadaptive feature modulation and dynamic uncertainty-based task weighting. The system integrates AI inference with an Arduino-controlled 4-DOF robotic arm and real-time clinician notification via WebSocket. In a controlled laboratory evaluation (n = 10 volunteers, 100 trials), the framework achieved 85% gesture top-1 accuracy (F1 = 0.83), 94% cancer classification accuracy (ROC-AUC = 0.98), 90% actuation success, and sub-second end-to-end latency. Adaptive parameter sharing reduced trainable weights by approximately 15% compared to separate models while maintaining performance. These results demonstrate the technical feasibility of an efficiency-aware, cross-domain adaptive assistive robotic framework for simulated tele-oncology support, establishing a foundation for future clinical validation.
Zitieren
APA
Okpako, A. E., Oyem, C. B., Ojie, D. V., Iyamah, B. E., & Chiemeke, C. S. (2026). A Vision-Based Assistive Robotic System with Real-Time Gesture Recognition for Communication Support in Speech-Impaired Cancer Patients: A Pilot Feasibility Study. Faculty of Natural and Applied Sciences Journal of Computing and Applications, 3(1), 19-34. https://doi.org/10.63561/jca.v3i1.1199
Vancouver
Okpako AE, Oyem CB, Ojie DV, Iyamah BE, Chiemeke CS. A Vision-Based Assistive Robotic System with Real-Time Gesture Recognition for Communication Support in Speech-Impaired Cancer Patients: A Pilot Feasibility Study. Faculty of Natural and Applied Sciences Journal of Computing and Applications. 2026;3(1):19-34. doi:10.63561/jca.v3i1.1199
BibTeX
@article{Okpako2026,
author = {Okpako, Abugor Ejaita and Oyem, Chinyere Blossom and Ojie, Deborah Voke and Iyamah, Boyce Egobudiken and Chiemeke, Chinye Stella},
title = {A Vision-Based Assistive Robotic System with Real-Time Gesture Recognition for Communication Support in Speech-Impaired Cancer Patients: A Pilot Feasibility Study},
journal = {Faculty of Natural and Applied Sciences Journal of Computing and Applications},
year = {2026},
volume = {3},
number = {1},
pages = {19--34},
doi = {10.63561/jca.v3i1.1199}
}
EUVIMED