EUVIMEDEUVIMED entdecken
Crossref

LinProVSR: Linguistics-Knowledge Guided Progressive Disambiguation Network for Visual Speech Recognition

Feng Xue, Baochao Zhu, Wei Jia, Shujie Li, Yu Li, Jinrui Zhang, Shengeng Tang, Dan Guo

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (14) · 11505-11513

DOI: 10.1609/aaai.v40i14.38133 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Feng Xue und 7 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual Speech Recognition (VSR), commonly known as lipreading, enables the recognition of spoken text by analyzing lip visual features. Due to the subtlety of lip movements, its recognition is much harder than other motion recognition tasks. Existing VSR models face the challenge of viseme ambiguity when processing phonemes with similar pronunciations—multiple phonemes share similar viseme features, leading to a notable drop in lipreading accuracy. To address this issue, this study proposes a Linguistics-Knowledge Guided Progressive Disambiguation Network for Visual Speech Recognition(LinProVSR) framework. First, an ambiguous sample set is constructed based on linguistic knowledge to provide supervisory signals for the model's training. Then, a Progressive Contrastive Disambiguation Network (PCDN) is designed, which progressively enhances the model's ability to capture the subtle viseme differences corresponding to similar phonemes through viseme-phoneme contrastive disambiguation in the encoding stage and text contrastive disambiguation in the decoding stage. Furthermore, we pioneer the Ambiguous Word Error Rate (AWER) metric specifically for evaluating recognition of phonetically ambiguous text, and verify the effectiveness of the proposed method on multiple public datasets, achieving a significant breakthrough especially in distinguishing visually similar phonemes.

Zitieren

APA

Xue, F., Zhu, B., Jia, W., Li, S., Li, Y., Zhang, J., Tang, S., & Guo, D. (2026). LinProVSR: Linguistics-Knowledge Guided Progressive Disambiguation Network for Visual Speech Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 40(14), 11505-11513. https://doi.org/10.1609/aaai.v40i14.38133

Vancouver

Xue F, Zhu B, Jia W, Li S, Li Y, Zhang J, et al. LinProVSR: Linguistics-Knowledge Guided Progressive Disambiguation Network for Visual Speech Recognition. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(14):11505-11513. doi:10.1609/aaai.v40i14.38133

BibTeX
@article{Xue2026,
  author = {Xue, Feng and Zhu, Baochao and Jia, Wei and Li, Shujie and Li, Yu and Zhang, Jinrui and Tang, Shengeng and Guo, Dan},
  title = {LinProVSR: Linguistics-Knowledge Guided Progressive Disambiguation Network for Visual Speech Recognition},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {14},
  pages = {11505--11513},
  doi = {10.1609/aaai.v40i14.38133}
}
Kostenlos starten →