A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality
Sensors · 2025
DOI: 10.3390/s25175484 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Alon Nemirovsky und 2 weiteren Autor:innen. Beteiligte Einrichtung: Andrew and Erna Viterbi Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel. Hauptthemen laut MEDLINE: Wearable Electronic Devices, Augmented Reality, Speech. Die Arbeit nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Israel Science Foundation, Pazy Research Foundation.
Abstract (Originalsprache)
This paper presents a bilinear framework for real-time speech source separation and dereverberation tailored to wearable augmented reality devices operating in dynamic acoustic environments. Using the Speech Enhancement for Augmented Reality (SPEAR) Challenge dataset, we perform extensive validation with real-world recordings and review key algorithmic parameters, including the forgetting factor and regularization. To enhance robustness against direction-of-arrival (DOA) estimation errors caused by head movements and localization uncertainty, we propose a region-of-interest (ROI) beamformer that replaces conventional point-source steering. Additionally, we introduce a multi-constraint beamforming design capable of simultaneously preserving multiple sources or suppressing known undesired sources. Experimental results demonstrate that ROI-based steering significantly improves robustness to localization errors while maintaining effective noise and reverberation suppression. However, this comes at the cost of increased high-frequency leakage from both desired and undesired sources. The multi-constraint formulation further enhances source separation with a modest trade-off in noise reduction. The proposed integration of ROI and LCMP within the low-complexity frameworks, validated comprehensively on the SPEAR dataset, offers a practical and efficient solution for real-time audio enhancement in wearable augmented reality systems.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Andrew and Erna Viterbi Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel.
ORCID-Profile: Gal Itzhak, Israel Cohen
Förderung
- Israel Science Foundation
- Pazy Research Foundation
Zitieren
APA
Nemirovsky, A., Itzhak, G., & Cohen, I. (2025). A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality. Sensors. https://doi.org/10.3390/s25175484
Vancouver
Nemirovsky A, Itzhak G, Cohen I. A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality. Sensors. 2025. doi:10.3390/s25175484
BibTeX
@article{Nemirovsky2025,
author = {Nemirovsky, Alon and Itzhak, Gal and Cohen, Israel},
title = {A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality},
journal = {Sensors},
year = {2025},
doi = {10.3390/s25175484}
}
EUVIMED