Using Hybrid HMM/DNN Embedding Extractor Models in Computational Paralinguistic Tasks
Sensors · 2023
DOI: 10.3390/s23115208 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von Mercedes Vetráb und 1 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Informatics, University of Szeged, H-6720 Szeged, Hungary. Hauptthemen laut MEDLINE: Speech, Speech Perception. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NRDI Office of the Hungarian Ministry of Innovation and Technology, Artificial Intelligence National Laboratory Program.
Abstract (Originalsprache)
The field of computational paralinguistics emerged from automatic speech processing, and it covers a wide range of tasks involving different phenomena present in human speech. It focuses on the non-verbal content of human speech, including tasks such as spoken emotion recognition, conflict intensity estimation and sleepiness detection from speech, showing straightforward application possibilities for remote monitoring with acoustic sensors. The two main technical issues present in computational paralinguistics are (1) handling varying-length utterances with traditional classifiers and (2) training models on relatively small corpora. In this study, we present a method that combines automatic speech recognition and paralinguistic approaches, which is able to handle both of these technical issues. That is, we trained a HMM/DNN hybrid acoustic model on a general ASR corpus, which was then used as a source of embeddings employed as features for several paralinguistic tasks. To convert the local embeddings into utterance-level features, we experimented with five different aggregation methods, namely mean, standard deviation, skewness, kurtosis and the ratio of non-zero activations. Our results show that the proposed feature extraction technique consistently outperforms the widely used x-vector method used as the baseline, independently of the actual paralinguistic task investigated. Furthermore, the aggregation techniques could be combined effectively as well, leading to further improvements depending on the task and the layer of the neural network serving as the source of the local embeddings. Overall, based on our experimental results, the proposed method can be considered as a competitive and resource-efficient approach for a wide range of computational paralinguistic tasks.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute of Informatics, University of Szeged, H-6720 Szeged, Hungary.
- ELKH-SZTE Research Group on Artificial Intelligence, H-6720 Szeged, Hungary.
ORCID-Profile: Mercedes Vetráb, Gábor Gosztolya
Förderung
- NRDI Office of the Hungarian Ministry of Innovation and Technology
- Artificial Intelligence National Laboratory Program
Zitieren
APA
Vetráb, M., & Gosztolya, G. (2023). Using Hybrid HMM/DNN Embedding Extractor Models in Computational Paralinguistic Tasks. Sensors. https://doi.org/10.3390/s23115208
Vancouver
Vetráb M, Gosztolya G. Using Hybrid HMM/DNN Embedding Extractor Models in Computational Paralinguistic Tasks. Sensors. 2023. doi:10.3390/s23115208
BibTeX
@article{Vetrb2023,
author = {Vetráb, Mercedes and Gosztolya, Gábor},
title = {Using Hybrid HMM/DNN Embedding Extractor Models in Computational Paralinguistic Tasks},
journal = {Sensors},
year = {2023},
doi = {10.3390/s23115208}
}
EUVIMED