Emotion Intensity and its Control for Emotional Voice Conversion
IEEE Transactions on Affective Computing · 01.01.2023 · 14 (1) · 31-48
DOI: 10.1109/taffc.2022.3175578 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, erschienen am 01.01.2023. Verfasst von Kun Zhou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 66-mal zitiert (Crossref) und nennt 137 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Science and Engineering Research Council, Agency of Science, Technology and Research, National Robotics Program, Human Robot Collaborative AI.
Abstract (Originalsprache)
Emotional voice conversion (EVC) seeks to convert the emotional state of an utterance while preserving the linguistic content and speaker identity. In EVC, emotions are usually treated as discrete categories overlooking the fact that speech also conveys emotions with various intensity levels that the listener can perceive. In this paper, we aim to explicitly characterize and control the intensity of emotion. We propose to disentangle the speaker style from linguistic content and encode the speaker style into a style embedding in a continuous space that forms the prototype of emotion embedding. We further learn the actual emotion encoder from an emotion-labelled database and study the use of relative attributes to represent fine-grained emotion intensity. To ensure emotional intelligibility, we incorporate emotion classification loss and emotion embedding similarity loss into the training of the EVC network. As desired, the proposed network controls the fine-grained emotion intensity in the output speech. Through both objective and subjective evaluations, we validate the effectiveness of the proposed network for emotional expressiveness and emotion intensity control.
Förderung
- Science and Engineering Research Council, Agency of Science, Technology and Research
- National Robotics Program
- Human Robot Collaborative AI
- National Research Foundation Singapore
- Agency for Science, Technology and Research
- RIE2020 Advanced Manufacturing and Engineering Domain
- Ministry of Education - Singapore
- SUTD Start-up Grant Artificial Intelligence for Human Voice Conversion
Zitieren
APA
Zhou, K., Sisman, B., Rana, R., Schuller, B. W., & Li, H. (2023). Emotion Intensity and its Control for Emotional Voice Conversion. IEEE Transactions on Affective Computing, 14(1), 31-48. https://doi.org/10.1109/taffc.2022.3175578
Vancouver
Zhou K, Sisman B, Rana R, Schuller BW, Li H. Emotion Intensity and its Control for Emotional Voice Conversion. IEEE Transactions on Affective Computing. 2023;14(1):31-48. doi:10.1109/taffc.2022.3175578
BibTeX
@article{Zhou2023,
author = {Zhou, Kun and Sisman, Berrak and Rana, Rajib and Schuller, Björn W. and Li, Haizhou},
title = {Emotion Intensity and its Control for Emotional Voice Conversion},
journal = {IEEE Transactions on Affective Computing},
year = {2023},
volume = {14},
number = {1},
pages = {31--48},
doi = {10.1109/taffc.2022.3175578}
}
EUVIMED