EUVIMEDEUVIMED entdecken
Crossref

AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception

Enrico Varano, Pierre Guilleminot, Tobias Reichenbach

The Journal of the Acoustical Society of America · 2023

DOI: 10.1121/10.0019460 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Journal of the Acoustical Society of America, 2023. Verfasst von Enrico Varano und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Bioengineering and Centre for Neurotechnology, Imperial College London, London, United Kingdom. Hauptthemen laut MEDLINE: Speech Perception. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: Engineering and Physical Sciences Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Seeing a speaker's face can help substantially with understanding their speech, particularly in challenging listening conditions. Research into the neurobiological mechanisms behind audiovisual integration has recently begun to employ continuous natural speech. However, these efforts are impeded by a lack of high-quality audiovisual recordings of a speaker narrating a longer text. Here, we seek to close this gap by developing AVbook, an audiovisual speech corpus designed for cognitive neuroscience studies and audiovisual speech recognition. The corpus consists of 3.6 h of audiovisual recordings of two speakers, one male and one female, each reading 59 passages from a narrative English text. The recordings were acquired at a high frame rate of 119.88 frames/s. The corpus includes phone-level alignment files and a set of multiple-choice questions to test attention to the different passages. We verified the efficacy of these questions in a pilot study. A short written summary is also provided for each recording. To enable audiovisual synchronization when presenting the stimuli, four videos of an electronic clapperboard were recorded with the corpus. The corpus is publicly available to support research into the neurobiology of audiovisual speech processing as well as the development of computer algorithms for audiovisual speech recognition.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Speech PerceptionMaleHumansFemaleSpeechPilot ProjectsVisual PerceptionAuditory Perception

Autor:innen und Institutionen

  • Department of Bioengineering and Centre for Neurotechnology, Imperial College London, London, United Kingdom.
  • Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Germany.

Förderung

  • Engineering and Physical Sciences Research Council

Zitieren

APA

Varano, E., Guilleminot, P., & Reichenbach, T. (2023). AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception. The Journal of the Acoustical Society of America. https://doi.org/10.1121/10.0019460

Vancouver

Varano E, Guilleminot P, Reichenbach T. AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception. The Journal of the Acoustical Society of America. 2023. doi:10.1121/10.0019460

BibTeX
@article{Varano2023,
  author = {Varano, Enrico and Guilleminot, Pierre and Reichenbach, Tobias},
  title = {AVbook, a high-frame-rate corpus of narrative audiovisual speech for investigating multimodal speech perception},
  journal = {The Journal of the Acoustical Society of America},
  year = {2023},
  doi = {10.1121/10.0019460}
}
Kostenlos starten →