Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction
Electronics · 01.08.2026 · 15 (15) · 3391
DOI: 10.3390/electronics15153391 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 01.08.2026. Verfasst von Ju-Yeol Oh und 1 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Sangmyung University.
Abstract (Originalsprache)
Target speech extraction (TSE) aims to recover a target speaker’s speech from mixed speech using auxiliary speaker information. Recent diffusion-based TSE models have shown promising performance, but conventional training mainly optimizes prediction accuracy at individual timesteps without explicitly using adjacent diffusion predictions as supervision. This may limit stable recovery under severe speaker overlap or strong interference. In this paper, we propose a trajectory-aware teacher–student fine-tuning framework based on a pretrained SoloSpeech extractor. The proposed framework constructs a current-centered local diffusion prediction trajectory from the current and adjacent timesteps. A teacher mixer uses this trajectory to generate a refined latent-space teacher target for additional student supervision. Experiments on Libri2Mix show that the proposed method improves PESQ from 1.92 to 1.97, ESTOI from 0.77 to 0.79, and SI-SNR from 10.45 dB to 11.42 dB compared with the reproduced SoloSpeech baseline under the same evaluation pipeline. It also reduces input-relative degradation cases by 45.8%, 36.7%, and 43.2% for SI-SNRi, PESQi, and ESTOIi, respectively. Additional comparisons with normal fine-tuning and VCTK + DEMAND support the proposed supervision under different fine-tuning conditions. Since the teacher mixer is used only during fine-tuning and removed during inference, the proposed method improves the extractor while preserving the SoloSpeech inference structure.
Förderung
- Sangmyung University
Zitieren
APA
Oh, J., & Lee, S. (2026). Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction. Electronics, 15(15), 3391. https://doi.org/10.3390/electronics15153391
Vancouver
Oh J, Lee S. Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction. Electronics. 2026;15(15):3391. doi:10.3390/electronics15153391
BibTeX
@article{Oh2026,
author = {Oh, Ju-Yeol and Lee, Seok-Pil},
title = {Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction},
journal = {Electronics},
year = {2026},
volume = {15},
number = {15},
pages = {3391},
doi = {10.3390/electronics15153391}
}
EUVIMED