EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition

Ussen Kimanuka, Ciira wa Maina, Osman Büyük

Applied AI Letters · 2026-06 · 7 (2)

DOI: 10.1002/ail2.70023 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied AI Letters, online erschienen am 03.03.2026. Verfasst von Ussen Kimanuka und 2 weiteren Autor:innen. Die Arbeit nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: African Union, Pan African University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Substantial improvements in automatic speech recognition performance have been realized through supervised fine‐tuning after self‐supervised pre‐training of a speech foundation model. The large size of foundation models, along with their varied losses and objective functions, makes it impractical to obtain optimum results with these models, and fine‐tuning each model independently for several downstream tasks is prohibitively expensive. The proposed methodology consists of three phases: feature extraction, feature fusion, and prediction. During feature extraction, several pre‐trained models, each with varying losses and objective functions, are used to derive representations. Then, a designed co‐attentional fusion mechanism is applied during feature fusion, enabling the network to adaptively weight different fusion operations to acquire common representations across models. Finally, a connectionist temporal classification (CTC) layer is used as a framework to generate transcription predictions. Moreover, the proposed self‐supervised feature‐fusion transformer block (SSF‐FT), incorporating inter‐model techniques, effectively captures both shared and distinctive information across all fused representations. We conducted an interpretability study in high‐resource (English) and low‐resource (Congolese) scenarios. In both settings, we observe that features performing well with shallow ensemble methods also perform well with attention‐weighted soft mixing. Experimental results demonstrate that our approach offers complementary strengths to existing ensemble techniques, with particular improvements in acoustically challenging and low‐resource scenarios.

Förderung

  • African Union
  • Pan African University

Zitieren

APA

Kimanuka, U., Maina, C. W., & Büyük, O. (2026). Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition. Applied AI Letters, 7(2). https://doi.org/10.1002/ail2.70023

Vancouver

Kimanuka U, Maina CW, Büyük O. Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition. Applied AI Letters. 2026;7(2). doi:10.1002/ail2.70023

BibTeX
@article{Kimanuka2026,
  author = {Kimanuka, Ussen and Maina, Ciira wa and Büyük, Osman},
  title = {Inter‐Model Feature Fusion for Robust Low‐Resource Speech Recognition},
  journal = {Applied AI Letters},
  year = {2026},
  volume = {7},
  number = {2},
  doi = {10.1002/ail2.70023}
}
Kostenlos starten →