EUVIMEDEUVIMED entdecken
Crossref

Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition

Qi Rao, Ke Sun, Xiaohan Wang, Qi Wang, Bang Zhang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4650-4658

DOI: 10.1609/aaai.v38i5.28265 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Qi Rao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Continuous sign language recognition (CSLR) aims to recognize gloss sequences from continuous sign videos. Recent works enhance the gloss representation consistency by mining correlations between visual and contextual modules within individual sentences. However, there still remain much richer correlations among glosses across different sentences. In this paper, we present a simple yet effective Cross-Sentence Gloss Consistency (CSGC), which enforces glosses belonging to a same category to be more consistent in representation than those belonging to different categories, across all training sentences. Specifically, in CSGC, a prototype is maintained for each gloss category and benefits the gloss discrimination in a contrastive way. Thanks to the well-distinguished gloss prototype, an auxiliary similarity classifier is devised to enhance the recognition clues, thus yielding more accurate results. Extensive experiments conducted on three CSLR datasets show that our proposed CSGC significantly boosts the performance of CSLR, surpassing existing state-of-the-art works by large margins (i.e., 1.6% on PHOENIX14, 2.4% on PHOENIX14-T, and 5.7% on CSL-Daily).

Zitieren

APA

Rao, Q., Sun, K., Wang, X., Wang, Q., & Zhang, B. (2024). Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4650-4658. https://doi.org/10.1609/aaai.v38i5.28265

Vancouver

Rao Q, Sun K, Wang X, Wang Q, Zhang B. Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4650-4658. doi:10.1609/aaai.v38i5.28265

BibTeX
@article{Rao2024,
  author = {Rao, Qi and Sun, Ke and Wang, Xiaohan and Wang, Qi and Zhang, Bang},
  title = {Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {5},
  pages = {4650--4658},
  doi = {10.1609/aaai.v38i5.28265}
}
Kostenlos starten →