EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Generative Spoken Dialogue Language Modeling

Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoît Sagot, Abdelrahman Mohamed, Emmanuel Dupoux

Transactions of the Association for Computational Linguistics · 14.03.2023 · 11 · 250-266

DOI: 10.1162/tacl_a_00545 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 14.03.2023. Verfasst von Tu Anh Nguyen und 10 weiteren Autor:innen. Die Arbeit wurde bisher 50-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We introduce dGSLM, the first “textless” model able to generate audio samples of naturalistic spoken dialogues. It uses recent work on unsupervised spoken unit discovery coupled with a dual-tower transformer architecture with cross-attention trained on 2000 hours of two-channel raw conversational audio (Fisher dataset) without any text or labels. We show that our model is able to generate speech, laughter, and other paralinguistic signals in the two channels simultaneously and reproduces more naturalistic and fluid turn taking compared to a text-based cascaded model.1,2

Zitieren

APA

Nguyen, T. A., Kharitonov, E., Copet, J., Adi, Y., Hsu, W., Elkahky, A., Tomasello, P., Algayres, R., Sagot, B., Mohamed, A., & Dupoux, E. (2023). Generative Spoken Dialogue Language Modeling. Transactions of the Association for Computational Linguistics, 11, 250-266. https://doi.org/10.1162/tacl_a_00545

Vancouver

Nguyen TA, Kharitonov E, Copet J, Adi Y, Hsu W, Elkahky A, et al. Generative Spoken Dialogue Language Modeling. Transactions of the Association for Computational Linguistics. 2023;11:250-266. doi:10.1162/tacl_a_00545

BibTeX
@article{Nguyen2023,
  author = {Nguyen, Tu Anh and Kharitonov, Eugene and Copet, Jade and Adi, Yossi and Hsu, Wei-Ning and Elkahky, Ali and Tomasello, Paden and Algayres, Robin and Sagot, Benoît and Mohamed, Abdelrahman and Dupoux, Emmanuel},
  title = {Generative Spoken Dialogue Language Modeling},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2023},
  volume = {11},
  pages = {250--266},
  doi = {10.1162/tacl_a_00545}
}
Kostenlos starten →