Generative Sign-Description Prompts with Multi-Positive Contrastive Learning for Sign Language Recognition
Sensors · 24.09.2025 · 25 (19) · 5957
DOI: 10.3390/s25195957 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 24.09.2025. Verfasst von Siyu Liang und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Technology, Xidian University, Xi'an 710071, China. Hauptthemen laut MEDLINE: Sign Language, Machine Learning. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Science and Technology Major Project, the National Natural Science Foundation of China, the Provincial Key Research and Development Program of Shaanxi.
Abstract (Originalsprache)
While sign language combines sequential hand motions with concurrent non-manual cues (e.g., mouth shapes and head tilts), current recognition systems lack multimodal annotation methods capable of capturing their hierarchical semantics. To bridge this gap, we propose GSP-MC, the first method integrating generative large language models into sign language recognition. It leverages retrieval-augmented generation with domain-specific large language models and expert-validated corpora to produce precise multipart descriptions. A dual-encoder architecture bidirectionally aligns hierarchical skeleton features with multi-level text descriptions (global, synonym, part) through probabilistic matching. The approach combines global and part-level losses with KL divergence optimization, ensuring robust alignment across relevant text-skeleton pairs while capturing sign semantics and detailed dynamics. Experiments demonstrate state-of-the-art performance, achieving 97.1% accuracy on the Chinese SLR500 (surpassing SSRL’s 96.9%) and 97.07% on the Turkish AUTSL (exceeding SML’s 96.85%), confirming cross-lingual potential for inclusive communication technologies.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Technology, Xidian University, Xi'an 710071, China.
- Xi'an Key Laboratory of Big Data and Intelligent Vision, Xi'an 710071, China.
- Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University, Xi'an 710071, China.
- College of Information Science and Technology, Dalian Martime University, Dalian 116026, China.
ORCID-Profile: Siyu Liang, Yunan Li, Wentian Xin, Huizhou Chen, Xujie Liu, Kang Liu, Qiguang Miao
Förderung
- the National Science and Technology Major Project
- the National Natural Science Foundation of China
- the Provincial Key Research and Development Program of Shaanxi
- the Research Project on Higher Education Teaching Reform of Shaanxi Province
- National Science and Technology Major Project
- National Natural Science Foundation of China
- Provincial Key Research and Development Program of Shaanxi
- Research Project on Higher Education Teaching Reform of Shaanxi Province
Zitieren
APA
Liang, S., Li, Y., Xin, W., Chen, H., Liu, X., Liu, K., & Miao, Q. (2025). Generative Sign-Description Prompts with Multi-Positive Contrastive Learning for Sign Language Recognition. Sensors, 25(19), 5957. https://doi.org/10.3390/s25195957
Vancouver
Liang S, Li Y, Xin W, Chen H, Liu X, Liu K, et al. Generative Sign-Description Prompts with Multi-Positive Contrastive Learning for Sign Language Recognition. Sensors. 2025;25(19):5957. doi:10.3390/s25195957
BibTeX
@article{Liang2025,
author = {Liang, Siyu and Li, Yunan and Xin, Wentian and Chen, Huizhou and Liu, Xujie and Liu, Kang and Miao, Qiguang},
title = {Generative Sign-Description Prompts with Multi-Positive Contrastive Learning for Sign Language Recognition},
journal = {Sensors},
year = {2025},
volume = {25},
number = {19},
pages = {5957},
doi = {10.3390/s25195957}
}
EUVIMED