Dynamic Local-Global Attention for Audio-Visual Speech Enhancement in Noisy Communication
INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES · 03.08.2026 · 6 (2) · 38-50
DOI: 10.66238/ijrss.v6i2.120 ↗Kurz gefasst
Fachartikel in der Zeitschrift INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES, online erschienen am 03.08.2026. Verfasst von Elizabeth C. Hughes. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Speech enhancement in noisy communication environments remains a formidable challenge, particularly when relying solely on acoustic signals corrupted by severe background interference. Recent advancements have demonstrated that incorporating visual cues, such as lip movements and facial expressions, can significantly improve speech intelligibility and quality. However, effectively fusing these multimodal streams while simultaneously capturing fine-grained phonetic details and long-range semantic dependencies remains an unresolved problem. This paper proposes a novel framework utilizing a dynamic local-global attention mechanism for audio-visual speech enhancement. The proposed architecture distinctively processes local temporal dynamics, crucial for aligning short-term phonetic and visemic features, while simultaneously modeling global sequence dependencies to maintain context and speaker characteristics across the entire utterance. A dynamic gating mechanism is introduced to adaptively weight the contributions of local and global attention representations based on the estimated instantaneous signal-to-noise ratio and the reliability of the visual stream. Comprehensive evaluations are conducted across multiple challenging acoustic environments, demonstrating that the proposed method substantially outperforms existing single-modality and multimodal baselines in terms of objective metrics such as perceptual evaluation of speech quality and short-time objective intelligibility. The findings underscore the critical role of multi-scale temporal modeling and adaptive multimodal fusion in advancing the robustness of communication systems in real-world scenarios.
Zitieren
APA
Hughes, E. C. (2026). Dynamic Local-Global Attention for Audio-Visual Speech Enhancement in Noisy Communication. INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES, 6(2), 38-50. https://doi.org/10.66238/ijrss.v6i2.120
Vancouver
Hughes EC. Dynamic Local-Global Attention for Audio-Visual Speech Enhancement in Noisy Communication. INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES. 2026;6(2):38-50. doi:10.66238/ijrss.v6i2.120
BibTeX
@article{Hughes2026,
author = {Hughes, Elizabeth C.},
title = {Dynamic Local-Global Attention for Audio-Visual Speech Enhancement in Noisy Communication},
journal = {INTERNATIONAL JOURNAL OF RESEARCH & SOCIAL SCIENCES},
year = {2026},
volume = {6},
number = {2},
pages = {38--50},
doi = {10.66238/ijrss.v6i2.120}
}
EUVIMED