EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SIGNIFY: Leveraging Machine Learning and Gesture Recognition for Sign Language Teaching Through a Serious Game

Luca Ulrich, Giulio Carmassi, Paolo Garelli, Gianluca Lo Presti, Gioele Ramondetti, Giorgia Marullo, Chiara Innocente, Enrico Vezzetti

Future Internet · 01.12.2024 · 16 (12) · 447

DOI: 10.3390/fi16120447 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Future Internet, online erschienen am 01.12.2024. Verfasst von Luca Ulrich und 7 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: FSE REACT-EU—PON Ricerca e Innovazione.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Italian Sign Language (LIS) is the primary form of communication for many members of the Italian deaf community. Despite being recognized as a fully fledged language with its own grammar and syntax, LIS still faces challenges in gaining widespread recognition and integration into public services, education, and media. In recent years, advancements in technology, including artificial intelligence and machine learning, have opened up new opportunities to bridge communication gaps between the deaf and hearing communities. This paper presents a novel educational tool designed to teach LIS through SIGNIFY, a Machine Learning-based interactive serious game. The game incorporates a tutorial section, guiding users to learn the sign alphabet, and a classic hangman game that reinforces learning through practice. The developed system employs advanced hand gesture recognition techniques for learning and perfecting sign language gestures. The proposed solution detects and overlays 21 hand landmarks and a bounding box on live camera feeds, making use of an open-source framework to provide real-time visual feedback. Moreover, the study compares the effectiveness of two camera systems: the Azure Kinect, which provides RGB-D information, and a standard RGB laptop camera. Results highlight both systems’ feasibility and educational potential, showcasing their respective advantages and limitations. Evaluations with primary school children demonstrate the tool’s ability to make sign language education more accessible and engaging. This article emphasizes the work’s contribution to inclusive education, highlighting the integration of technology to enhance learning experiences for deaf and hard-of-hearing individuals.

Förderung

  • FSE REACT-EU—PON Ricerca e Innovazione

Zitieren

APA

Ulrich, L., Carmassi, G., Garelli, P., Presti, G. L., Ramondetti, G., Marullo, G., Innocente, C., & Vezzetti, E. (2024). SIGNIFY: Leveraging Machine Learning and Gesture Recognition for Sign Language Teaching Through a Serious Game. Future Internet, 16(12), 447. https://doi.org/10.3390/fi16120447

Vancouver

Ulrich L, Carmassi G, Garelli P, Presti GL, Ramondetti G, Marullo G, et al. SIGNIFY: Leveraging Machine Learning and Gesture Recognition for Sign Language Teaching Through a Serious Game. Future Internet. 2024;16(12):447. doi:10.3390/fi16120447

BibTeX
@article{Ulrich2024,
  author = {Ulrich, Luca and Carmassi, Giulio and Garelli, Paolo and Presti, Gianluca Lo and Ramondetti, Gioele and Marullo, Giorgia and Innocente, Chiara and Vezzetti, Enrico},
  title = {SIGNIFY: Leveraging Machine Learning and Gesture Recognition for Sign Language Teaching Through a Serious Game},
  journal = {Future Internet},
  year = {2024},
  volume = {16},
  number = {12},
  pages = {447},
  doi = {10.3390/fi16120447}
}
Kostenlos starten →