EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality

Alon Nemirovsky, Gal Itzhak iD, Israel Cohen iD

Sensors · 2025

DOI: 10.3390/s25175484 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2025. Verfasst von Alon Nemirovsky und 2 weiteren Autor:innen. Beteiligte Einrichtung: Andrew and Erna Viterbi Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel. Hauptthemen laut MEDLINE: Wearable Electronic Devices, Augmented Reality, Speech. Die Arbeit nennt 56 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Israel Science Foundation, Pazy Research Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper presents a bilinear framework for real-time speech source separation and dereverberation tailored to wearable augmented reality devices operating in dynamic acoustic environments. Using the Speech Enhancement for Augmented Reality (SPEAR) Challenge dataset, we perform extensive validation with real-world recordings and review key algorithmic parameters, including the forgetting factor and regularization. To enhance robustness against direction-of-arrival (DOA) estimation errors caused by head movements and localization uncertainty, we propose a region-of-interest (ROI) beamformer that replaces conventional point-source steering. Additionally, we introduce a multi-constraint beamforming design capable of simultaneously preserving multiple sources or suppressing known undesired sources. Experimental results demonstrate that ROI-based steering significantly improves robustness to localization errors while maintaining effective noise and reverberation suppression. However, this comes at the cost of increased high-frequency leakage from both desired and undesired sources. The multi-constraint formulation further enhances source separation with a modest trade-off in noise reduction. The proposed integration of ROI and LCMP within the low-complexity frameworks, validated comprehensively on the SPEAR dataset, offers a practical and efficient solution for real-time audio enhancement in wearable augmented reality systems.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Wearable Electronic DevicesAugmented RealitySpeechHumansAlgorithmsAcoustics

Stichwörter der Autor:innen

Kronecker product filteringadaptive beamformingaugmented realitydereverberationregion-of-interest beamformingsource separation

Autor:innen und Institutionen

  • Andrew and Erna Viterbi Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel.

ORCID-Profile: Gal Itzhak, Israel Cohen

Förderung

  • Israel Science Foundation
  • Pazy Research Foundation

Zitieren

APA

Nemirovsky, A., Itzhak, G., & Cohen, I. (2025). A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality. Sensors. https://doi.org/10.3390/s25175484

Vancouver

Nemirovsky A, Itzhak G, Cohen I. A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality. Sensors. 2025. doi:10.3390/s25175484

BibTeX
@article{Nemirovsky2025,
  author = {Nemirovsky, Alon and Itzhak, Gal and Cohen, Israel},
  title = {A Robust Bilinear Framework for Real-Time Speech Separation and Dereverberation in Wearable Augmented Reality},
  journal = {Sensors},
  year = {2025},
  doi = {10.3390/s25175484}
}
Kostenlos starten →