Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition
Applied AI Letters · 2026-06 · 7 (2)
DOI: 10.1002/ail2.70023 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied AI Letters, online erschienen am 03.03.2026. Verfasst von Ussen Kimanuka und 2 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: African Union, Pan African University.
Abstract (Originalsprache)
Substantial improvements in automatic speech recognition performance have been realized through supervised fine‐tuning after self‐supervised pre‐training of a speech foundation model. The large size of foundation models, along with their varied losses and objective functions, makes it impractical to obtain optimum results with these models, and fine‐tuning each model independently for several downstream tasks is prohibitively expensive. The proposed methodology consists of three phases: feature extraction, feature fusion, and prediction. During feature extraction, several pre‐trained models, each with varying losses and objective functions, are used to derive representations. Then, a designed co‐attentional fusion mechanism is applied during feature fusion, enabling the network to adaptively weight different fusion operations to acquire common representations across models. Finally, a connectionist temporal classification (CTC) layer is used as a framework to generate transcription predictions. Moreover, the proposed self‐supervised feature‐fusion transformer block (SSF‐FT), incorporating inter‐model techniques, effectively captures both shared and distinctive information across all fused representations. We conducted an interpretability study in high‐resource (English) and low‐resource (Congolese) scenarios. In both settings, we observe that features performing well with shallow ensemble methods also perform well with attention‐weighted soft mixing. Experimental results demonstrate that our approach offers complementary strengths to existing ensemble techniques, with particular improvements in acoustically challenging and low‐resource scenarios.
Förderung
- African Union
- Pan African University
Zitieren
APA
Kimanuka, U., Maina, C. W., & Büyük, O. (2026). Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition. Applied AI Letters, 7(2). https://doi.org/10.1002/ail2.70023
Vancouver
Kimanuka U, Maina CW, Büyük O. Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition. Applied AI Letters. 2026;7(2). doi:10.1002/ail2.70023
BibTeX
@article{Kimanuka2026,
author = {Kimanuka, Ussen and Maina, Ciira wa and Büyük, Osman},
title = {Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition},
journal = {Applied AI Letters},
year = {2026},
volume = {7},
number = {2},
doi = {10.1002/ail2.70023}
}
EUVIMED