Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4650-4658
DOI: 10.1609/aaai.v38i5.28265 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Qi Rao und 4 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref).
Abstract (Originalsprache)
Continuous sign language recognition (CSLR) aims to recognize gloss sequences from continuous sign videos. Recent works enhance the gloss representation consistency by mining correlations between visual and contextual modules within individual sentences. However, there still remain much richer correlations among glosses across different sentences. In this paper, we present a simple yet effective Cross-Sentence Gloss Consistency (CSGC), which enforces glosses belonging to a same category to be more consistent in representation than those belonging to different categories, across all training sentences. Specifically, in CSGC, a prototype is maintained for each gloss category and benefits the gloss discrimination in a contrastive way. Thanks to the well-distinguished gloss prototype, an auxiliary similarity classifier is devised to enhance the recognition clues, thus yielding more accurate results. Extensive experiments conducted on three CSLR datasets show that our proposed CSGC significantly boosts the performance of CSLR, surpassing existing state-of-the-art works by large margins (i.e., 1.6% on PHOENIX14, 2.4% on PHOENIX14-T, and 5.7% on CSL-Daily).
Zitieren
APA
Rao, Q., Sun, K., Wang, X., Wang, Q., & Zhang, B. (2024). Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4650-4658. https://doi.org/10.1609/aaai.v38i5.28265
Vancouver
Rao Q, Sun K, Wang X, Wang Q, Zhang B. Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4650-4658. doi:10.1609/aaai.v38i5.28265
BibTeX
@article{Rao2024,
author = {Rao, Qi and Sun, Ke and Wang, Xiaohan and Wang, Qi and Zhang, Bang},
title = {Cross-Sentence Gloss Consistency for Continuous Sign Language Recognition},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {5},
pages = {4650--4658},
doi = {10.1609/aaai.v38i5.28265}
}
EUVIMED