EUVIMEDEUVIMED entdecken
Open Access · CC BY-ND 4.0

Explainability Evaluation of Transfer Learning Models Utilized in Multimodal Translation of Sign Language

H. M. L. S. Kumari, C. K. Walgampaya

Engineer: Journal of the Institution of Engineers, Sri Lanka · 12.05.2026 · 59 (2) · 99-110

DOI: 10.4038/engineer.v59i2.7752 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Engineer: Journal of the Institution of Engineers, Sri Lanka, online erschienen am 12.05.2026. Verfasst von H. M. L. S. Kumari und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sign language is a mode of communication for people with hearing loss and speaking disabilities; however, gesture meanings vary from region to region. The proposed study evaluates the efficacy of transfer learning for converting sign gestures into natural language text and audio. American Sign Language and Sinhala Sign Language datasets were used to demonstrate the impact of data augmentation and image preprocessing on model performance across RGB, grayscale, and binary formats. Transfer learning models, including MobileNet, InceptionV3, DenseNet121, and DenseNet201 were utilized, and results indicate that RGB data significantly outperforms other formats. MobileNet achieved the highest accuracies of 98.62% for the ASL dataset and 99.66% for the Sinhala Sign dataset. Real-time validation was conducted via webcam inference to ensure practical reliability. Furthermore, SHAP (SHapley Additive exPlanations) was used to evaluate the proposed model's interpretability. The study concludes that combining lightweight transfer learning architectures with robust data augmentation can lead to significant improvements in communication of sign language users.

Zitieren

APA

Kumari, H. M. L. S., & Walgampaya, C. K. (2026). Explainability Evaluation of Transfer Learning Models Utilized in Multimodal Translation of Sign Language. Engineer: Journal of the Institution of Engineers, Sri Lanka, 59(2), 99-110. https://doi.org/10.4038/engineer.v59i2.7752

Vancouver

Kumari HMLS, Walgampaya CK. Explainability Evaluation of Transfer Learning Models Utilized in Multimodal Translation of Sign Language. Engineer: Journal of the Institution of Engineers, Sri Lanka. 2026;59(2):99-110. doi:10.4038/engineer.v59i2.7752

BibTeX
@article{Kumari2026,
  author = {Kumari, H. M. L. S. and Walgampaya, C. K.},
  title = {Explainability Evaluation of Transfer Learning Models Utilized in Multimodal Translation of Sign Language},
  journal = {Engineer: Journal of the Institution of Engineers, Sri Lanka},
  year = {2026},
  volume = {59},
  number = {2},
  pages = {99--110},
  doi = {10.4038/engineer.v59i2.7752}
}
Kostenlos starten →