EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction

Ju-Yeol Oh, Seok-Pil Lee

Electronics · 2026-08-01 · 15 (15) · 3391

DOI: 10.3390/electronics15153391 ↗

At a glance

Journal article in Electronics, published online on 2026-08-01. Written by Ju-Yeol Oh and 1 co-authors. 38 references. The full text is freely available (open access, CC BY 4.0). Funded by: Sangmyung University.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Target speech extraction (TSE) aims to recover a target speaker’s speech from mixed speech using auxiliary speaker information. Recent diffusion-based TSE models have shown promising performance, but conventional training mainly optimizes prediction accuracy at individual timesteps without explicitly using adjacent diffusion predictions as supervision. This may limit stable recovery under severe speaker overlap or strong interference. In this paper, we propose a trajectory-aware teacher–student fine-tuning framework based on a pretrained SoloSpeech extractor. The proposed framework constructs a current-centered local diffusion prediction trajectory from the current and adjacent timesteps. A teacher mixer uses this trajectory to generate a refined latent-space teacher target for additional student supervision. Experiments on Libri2Mix show that the proposed method improves PESQ from 1.92 to 1.97, ESTOI from 0.77 to 0.79, and SI-SNR from 10.45 dB to 11.42 dB compared with the reproduced SoloSpeech baseline under the same evaluation pipeline. It also reduces input-relative degradation cases by 45.8%, 36.7%, and 43.2% for SI-SNRi, PESQi, and ESTOIi, respectively. Additional comparisons with normal fine-tuning and VCTK + DEMAND support the proposed supervision under different fine-tuning conditions. Since the teacher mixer is used only during fine-tuning and removed during inference, the proposed method improves the extractor while preserving the SoloSpeech inference structure.

Funding

  • Sangmyung University

Cite

APA

Oh, J., & Lee, S. (2026). Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction. Electronics, 15(15), 3391. https://doi.org/10.3390/electronics15153391

Vancouver

Oh J, Lee S. Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction. Electronics. 2026;15(15):3391. doi:10.3390/electronics15153391

BibTeX
@article{Oh2026,
  author = {Oh, Ju-Yeol and Lee, Seok-Pil},
  title = {Trajectory-Aware Teacher–Student Fine-Tuning for Diffusion-Based Target Speech Extraction},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {15},
  pages = {3391},
  doi = {10.3390/electronics15153391}
}
Start for free →