EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization

Horia-Sebastian Ioniță, Victor Popa, Constantin Paleologu

Electronics · 28.10.2026 · 15 (21) · 5004

DOI: 10.3390/electronics15215004 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 28.10.2026. Verfasst von Horia-Sebastian Ioniță und 2 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The automatic detection of musical stems within polyphonic mixtures is a vital preliminary step in contemporary Music Information Retrieval (MIR) tasks, notably stem-based source separation applications. While prior research has largely focused on isolated vocal activity detection (VAD) or frame-level instrument tagging, modern engineering requirements necessitate robust Stem Activity Detection (SAD) systems capable of real-time gating. In this study, we propose a hybrid framework combining a Transfer-Learning Audio Spectrogram Transformer (AST) with a strictly causal Digital Signal Processing (DSP) post-processor. To overcome the data-starvation limitations of training Transformers from scratch, we leverage an existing HuggingFace pre-trained backbone, dynamically padded to process 5.0-s high-resolution log-mel spectrograms. To explicitly model the absence of instruments and to mitigate significant segment-level class imbalance, we adapt a physics-aware stochastic source-dropping augmentation combined with an optimized Asymmetric Loss (ASL) function. To eliminate transient classification chatter and bridge micro-silences for live deployment, we introduce a non-linear, zero-look-ahead causal median filter. To ensure statistical rigor and eliminate overlapping-window data leakage, the models are evaluated across three random seeds utilizing track-level bootstrapping (1000 iterations) to generate 95% Confidence Intervals (CIs). Experimental results on the MUSDB18 test set demonstrate that our hybrid AST-DSP framework surpasses an official audio-pretrained PANN CNN14 baseline, achieving a global Macro F1-score of 0.914 and a Macro MCC of 0.462, highlighted by a robust AUPRC of 0.997 for Drums and 0.978 for Vocals. Operating at an end-to-end neural inference and DSP speed of approximately 11.62 ms per segment via Graphics Processing Unit (GPU) acceleration, the proposed system provides a highly stable, real-time streaming-capable gating mechanism for multi-stem presence detection in dense polyphonic mixtures.

Zitieren

APA

Ioniță, H., Popa, V., & Paleologu, C. (2026). Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization. Electronics, 15(21), 5004. https://doi.org/10.3390/electronics15215004

Vancouver

Ioniță H, Popa V, Paleologu C. Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization. Electronics. 2026;15(21):5004. doi:10.3390/electronics15215004

BibTeX
@article{Ioni2026,
  author = {Ioniță, Horia-Sebastian and Popa, Victor and Paleologu, Constantin},
  title = {Transfer-Learning Audio Spectrogram Transformer for Multi-Label Stem Detection with DSP Stabilization},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {5004},
  doi = {10.3390/electronics15215004}
}
Kostenlos starten →