EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Interactive learning of natural sign language with radar

Emre Kurtoğlu, Kenneth DeHaan, Caroline Kobek Pezzarossi, Darrin J. Griffin, Chris Crawford, Sevgi Z. Gurbuz

IET Radar, Sonar & Navigation · 2024-08 · 18 (8) · 1203-1216

DOI: 10.1049/rsn2.12565 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Radar, Sonar & Navigation, online erschienen am 10.05.2024. Verfasst von Emre Kurtoğlu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Science Foundation, American Association of University Women.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Over the past decade, there have been great advancements in radio frequency sensor technology for human–computer interaction applications, such as gesture recognition, and human activity recognition more broadly. While there is a significant amount of study on these topics, in most cases, experimental data are acquired in controlled settings by directing participants what motion to articulate. However, especially for communicative motions, such as sign language, such directed data sets do not accurately capture natural, in situ articulations. This results in a difference in the distribution of directed American Sign Language (ASL) versus natural ASL, which severely degrades natural sign language recognition in real‐world scenarios. To overcome these challenges and acquire more representative data for training deep models, the authors develop an interactive gaming environment, ChessSIGN, which records video and radar data of participants as they play the game without any external direction . The authors investigate various ways of generating synthetic samples from directed ASL data, but show that ultimately such data does not offer much improvement over just initialising using imagery from ImageNet. In contrast, an interactive learning paradigm is proposed by the authors in which model training is shown to improve as more and more natural ASL samples are acquired and augmented via synthetic samples generated from a physics‐aware generative adversarial network. The authors show that the proposed approach enables the recognition of natural ASL in a real‐world setting, achieving an accuracy of 69% for 29 ASL signs—a 60% improvement over conventional training with directed ASL data.

Förderung

  • National Science Foundation
  • American Association of University Women

Zitieren

APA

Kurtoğlu, E., DeHaan, K., Pezzarossi, C. K., Griffin, D. J., Crawford, C., & Gurbuz, S. Z. (2024). Interactive learning of natural sign language with radar. IET Radar, Sonar & Navigation, 18(8), 1203-1216. https://doi.org/10.1049/rsn2.12565

Vancouver

Kurtoğlu E, DeHaan K, Pezzarossi CK, Griffin DJ, Crawford C, Gurbuz SZ. Interactive learning of natural sign language with radar. IET Radar, Sonar & Navigation. 2024;18(8):1203-1216. doi:10.1049/rsn2.12565

BibTeX
@article{Kurtolu2024,
  author = {Kurtoğlu, Emre and DeHaan, Kenneth and Pezzarossi, Caroline Kobek and Griffin, Darrin J. and Crawford, Chris and Gurbuz, Sevgi Z.},
  title = {Interactive learning of natural sign language with radar},
  journal = {IET Radar, Sonar & Navigation},
  year = {2024},
  volume = {18},
  number = {8},
  pages = {1203--1216},
  doi = {10.1049/rsn2.12565}
}
Kostenlos starten →