Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization
Electronics · 28.10.2026 · 15 (21) · 5004
DOI: 10.3390/electronics15215004 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 28.10.2026. Verfasst von Horia-Sebastian Ioniță und 2 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The automatic detection of musical stems within polyphonic mixtures is a vital preliminary step in contemporary Music Information Retrieval (MIR) tasks, notably stem-based source separation applications. While prior research has largely focused on isolated vocal activity detection (VAD) or frame-level instrument tagging, modern engineering requirements necessitate robust Stem Activity Detection (SAD) systems capable of real-time gating. In this study, we propose a hybrid framework combining a Transfer-Learning Audio Spectrogram Transformer (AST) with a strictly causal Digital Signal Processing (DSP) post-processor. To overcome the data-starvation limitations of training Transformers from scratch, we leverage an existing HuggingFace pre-trained backbone, dynamically padded to process 5.0-s high-resolution log-mel spectrograms. To explicitly model the absence of instruments and to mitigate significant segment-level class imbalance, we adapt a physics-aware stochastic source-dropping augmentation combined with an optimized Asymmetric Loss (ASL) function. To eliminate transient classification chatter and bridge micro-silences for live deployment, we introduce a non-linear, zero-look-ahead causal median filter. To ensure statistical rigor and eliminate overlapping-window data leakage, the models are evaluated across three random seeds utilizing track-level bootstrapping (1000 iterations) to generate 95% Confidence Intervals (CIs). Experimental results on the MUSDB18 test set demonstrate that our hybrid AST-DSP framework surpasses an official audio-pretrained PANN CNN14 baseline, achieving a global Macro F1-score of 0.914 and a Macro MCC of 0.462, highlighted by a robust AUPRC of 0.997 for Drums and 0.978 for Vocals. Operating at an end-to-end neural inference and DSP speed of approximately 11.62 ms per segment via Graphics Processing Unit (GPU) acceleration, the proposed system provides a highly stable, real-time streaming-capable gating mechanism for multi-stem presence detection in dense polyphonic mixtures.
Zitieren
APA
Ioniță, H., Popa, V., & Paleologu, C. (2026). Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization. Electronics, 15(21), 5004. https://doi.org/10.3390/electronics15215004
Vancouver
Ioniță H, Popa V, Paleologu C. Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization. Electronics. 2026;15(21):5004. doi:10.3390/electronics15215004
BibTeX
@article{Ioni2026,
author = {Ioniță, Horia-Sebastian and Popa, Victor and Paleologu, Constantin},
title = {Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization},
journal = {Electronics},
year = {2026},
volume = {15},
number = {21},
pages = {5004},
doi = {10.3390/electronics15215004}
}
EUVIMED