EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Brain‐Controlled Augmented Hearing for Spatially Moving Conversations in Multi‐Talker Environments

Vishal Choudhari iD, Cong Han iD, Stephan Bickel iD, Ashesh D Mehta iD, Catherine Schevon iD, Guy M McKhann iD, Nima Mesgarani iD

Advanced Science · 2024

DOI: 10.1002/advs.202401379 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advanced Science, 2024. Verfasst von Vishal Choudhari und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical Engineering, Columbia University, New York, NY, 10027, USA. Hauptthemen laut MEDLINE: Electroencephalography, Speech Perception. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NIDCD NIH HHS, The Marie-Josee and Henry R. Kravis Foundation, National Institute of Health's National Institute on Deafness and Other Communication Disorders.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Focusing on a specific conversation amidst multiple interfering talkers is challenging, especially for those with hearing loss. Brain‐controlled assistive hearing devices aim to alleviate this problem by enhancing the attended speech based on the listener's neural signals using auditory attention decoding (AAD). Departing from conventional AAD studies that relied on oversimplified scenarios with stationary talkers, a realistic AAD task that involves multiple talkers taking turns as they continuously move in space in background noise is presented. Invasive electroencephalography (iEEG) data are collected from three neurosurgical patients as they focused on one of the two moving conversations. An enhanced brain‐controlled assistive hearing system that combines AAD and a binaural speaker‐independent speech separation model is presented. The separation model unmixes talkers while preserving their spatial location and provides talker trajectories to the neural decoder to improve AAD accuracy. Subjective and objective evaluations show that the proposed system enhances speech intelligibility and facilitates conversation tracking while maintaining spatial cues and voice quality in challenging acoustic environments. This research demonstrates the potential of this approach in real‐world scenarios and marks a significant step toward developing assistive hearing technologies that adapt to the intricate dynamics of everyday auditory experiences.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ElectroencephalographySpeech PerceptionHumansMaleHearing AidsFemaleHearing LossMiddle AgedBrain-Computer InterfacesSpeech IntelligibilityAdultAttention

Stichwörter der Autor:innen

auditory attention decodingbrain‐computer interfacesdeep learninghearing aidsintracranial electroencephalographypsychophysicsspeech separation

Autor:innen und Institutionen

  • Department of Electrical Engineering, Columbia University, New York, NY, 10027, USA.
  • Mortimer B. Zuckerman Mind Brain Behavior Institute, New York, NY, 10027, USA.
  • Hofstra Northwell School of Medicine, Uniondale, NY, 11549, USA.
  • The Feinstein Institutes for Medical Research, Manhasset, NY, 11030, USA.
  • Department of Neurology, Columbia University, New York, NY, 10027, USA.
  • Department of Neurological Surgery, Vagelos College of Physicians and Surgeons, Columbia University, New York, New York, NY, 10027, USA.

ORCID-Profile: Vishal Choudhari, Cong Han, Stephan Bickel, Ashesh D Mehta, Catherine Schevon, Guy M McKhann, Nima Mesgarani

Förderung

  • NIDCD NIH HHS
  • The Marie-Josee and Henry R. Kravis Foundation
  • National Institute of Health's National Institute on Deafness and Other Communication Disorders

Zitieren

APA

Choudhari, V., Han, C., Bickel, S., Mehta, A. D., Schevon, C., McKhann, G. M., & Mesgarani, N. (2024). Brain‐Controlled Augmented Hearing for Spatially Moving Conversations in Multi‐Talker Environments. Advanced Science. https://doi.org/10.1002/advs.202401379

Vancouver

Choudhari V, Han C, Bickel S, Mehta AD, Schevon C, McKhann GM, et al. Brain‐Controlled Augmented Hearing for Spatially Moving Conversations in Multi‐Talker Environments. Advanced Science. 2024. doi:10.1002/advs.202401379

BibTeX
@article{Choudhari2024,
  author = {Choudhari, Vishal and Han, Cong and Bickel, Stephan and Mehta, Ashesh D and Schevon, Catherine and McKhann, Guy M and Mesgarani, Nima},
  title = {Brain‐Controlled Augmented Hearing for Spatially Moving Conversations in Multi‐Talker Environments},
  journal = {Advanced Science},
  year = {2024},
  doi = {10.1002/advs.202401379}
}
Kostenlos starten →