FedEmoNet: Privacy-preserving federated learning with TCN-Transformer fusion for cross-corpus speech emotion recognition
PLOS One · 07.05.2026 · 21 (5) · e0342953
DOI: 10.1371/journal.pone.0342953 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 07.05.2026. Verfasst von Mohammed Tawfik und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Computer and Information Technology, Sana'a University, Sana'a, Yemen. Hauptthemen laut MEDLINE: Speech, Emotions. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Federated learning offers a promising path toward privacy-preserving speech emotion recognition, yet existing approaches remain confined to single-corpus evaluation, lack formal differential privacy guarantees, and provide no mechanism for model interpretability. Meanwhile, cross-corpus generalization continues to challenge even centralized systems, with typical accuracy drops of 20–40% on unseen datasets due to domain shift in recording conditions, speaker demographics, and cultural expression norms. This paper introduces FedEmoNet, a unified framework that jointly addresses these open problems by combining FedProx-based distributed optimization, a hybrid Temporal Convolutional Network–Transformer (TCN-Transformer) architecture, Particle Swarm Optimization (PSO) feature selection, and calibrated ( ϵ = 1.0 , δ = 10 − 5 )-differential privacy. Five heterogeneous clients—two German-speech (EmoDB), two English-speech (RAVDESS), and one mixed—collaborate under non-IID conditions (Dirichlet α = 0.5 ) without exchanging raw audio. Each client extracts multi-scale phase space reconstructions at micro (25 ms), meso (250 ms), and macro (2.5 s) temporal resolutions alongside spectral and handcrafted features, which are fused through multi-head attention across the TCN-Transformer branches. On held-out, speaker-independent test sets the framework achieves 99.07% ± 0.35% accuracy on EmoDB (107 samples) and 98.96% ± 0.42% on RAVDESS (288 samples). Zero-shot cross-corpus evaluation on CREMA-D (1,488 samples) yields 68.15% ± 1.23% overall, with a clear arousal-dependent pattern: high-arousal emotions (angry, happy, sad) transfer at 71.9% versus 62.1% for low-arousal categories (neutral, disgust, fear). Ablation experiments confirm that PSO selection (+2.80%), Transformer blocks (+2.10%), and the FedProx protocol (+2.62%) each contribute significantly, and a monotonic reduced-data curve rules out memorization. Membership inference attack resistance drops to near-chance levels (AUC = 0.52) under differential privacy while retaining 98.5% accuracy. A dual SHAP–LIME explainability analysis reveals high inter-method agreement ( r = 0.997) and confirms that prosodic features—particularly fundamental frequency statistics—serve as language-invariant emotion indicators across all three corpora ( r = 0.94 cross-corpus consistency).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Computer and Information Technology, Sana'a University, Sana'a, Yemen.
- Department of Computer Science, Faculty of Information Technology, Ajloun National University, Ajloun, Jordan.
ORCID-Profile: Mohammed Tawfik
Zitieren
APA
Tawfik, M., Obeidat, R. A., Kamel, S., Aljarrah, N. A., Shehadeh, H. H., & Dalalah, A. (2026). FedEmoNet: Privacy-preserving federated learning with TCN-Transformer fusion for cross-corpus speech emotion recognition. PLOS One, 21(5), e0342953. https://doi.org/10.1371/journal.pone.0342953
Vancouver
Tawfik M, Obeidat RA, Kamel S, Aljarrah NA, Shehadeh HH, Dalalah A. FedEmoNet: Privacy-preserving federated learning with TCN-Transformer fusion for cross-corpus speech emotion recognition. PLOS One. 2026;21(5):e0342953. doi:10.1371/journal.pone.0342953
BibTeX
@article{Tawfik2026,
author = {Tawfik, Mohammed and Obeidat, Razan Ali and Kamel, Saddam and Aljarrah, Njood Anwer and Shehadeh, Haneen Hussein and Dalalah, Ahmad},
title = {FedEmoNet: Privacy-preserving federated learning with TCN-Transformer fusion for cross-corpus speech emotion recognition},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {5},
pages = {e0342953},
doi = {10.1371/journal.pone.0342953}
}
EUVIMED