Multi-Metric Preference Alignment for Generative Speech Restoration
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 34737-34745
DOI: 10.1609/aaai.v40i41.40775 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Junan Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Recent generative models have significantly advanced speech restoration tasks, yet their training objectives often misalign with human perceptual preferences, resulting in suboptimal quality. While post-training alignment has proven effective in other generative domains like text and image generation, its application to generative speech restoration remains largely under-explored. This work investigates the challenges of applying preference-based post-training to this task, focusing on how to define a robust preference signal and curate high-quality data to avoid reward hacking. To address these challenges, we propose a multi-metric preference alignment strategy. We construct a new dataset, GenSR-Pref, comprising 80K preference pairs, where each chosen sample is unanimously favored by a complementary suite of metrics covering perceptual quality, signal fidelity, content consistency, and timbre preservation. This principled approach ensures a holistic preference signal. Applying Direct Preference Optimization (DPO) with our dataset, we observe consistent and significant performance gains across three diverse generative paradigms: autoregressive models (AR), masked generative models (MGM), and flow-matching models (FM) on various restoration benchmarks, in both objective and subjective evaluations. Ablation studies confirm the superiority of our multi-metric strategy over single-metric approaches in mitigating reward hacking. Furthermore, we demonstrate that our aligned models can serve as powerful ''data annotators'', generating high-quality pseudo-labels to serve as a supervision signal for traditional discriminative models in data-scarce scenarios like singing voice restoration.
Zitieren
APA
Zhang, J., Zhang, X., Yang, J., Wang, Y., Fan, F., & Wu, Z. (2026). Multi-Metric Preference Alignment for Generative Speech Restoration. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 34737-34745. https://doi.org/10.1609/aaai.v40i41.40775
Vancouver
Zhang J, Zhang X, Yang J, Wang Y, Fan F, Wu Z. Multi-Metric Preference Alignment for Generative Speech Restoration. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):34737-34745. doi:10.1609/aaai.v40i41.40775
BibTeX
@article{Zhang2026,
author = {Zhang, Junan and Zhang, Xueyao and Yang, Jing and Wang, Yuancheng and Fan, Fan and Wu, Zhizheng},
title = {Multi-Metric Preference Alignment for Generative Speech Restoration},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {41},
pages = {34737--34745},
doi = {10.1609/aaai.v40i41.40775}
}
EUVIMED