EUVIMEDEUVIMED entdecken
Crossref

Interpreting Speaker Characteristics in the Dimensions of Self-Supervised Speech Features

Kyle Janse van Rensburg, Benjamin van Niekerk, Herman Kamper

IEEE Signal Processing Letters · 2026 · 33 · 3132-3136

DOI: 10.1109/lsp.2026.3716838 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2026. Verfasst von Kyle Janse van Rensburg und 2 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

How do speech models trained through self-supervised learning structure their representations? Previous studies have looked at how information is encoded in feature vectors across different layers. But few studies have considered whether speech characteristics are captured within individual dimensions of SSL features. In this paper, we specifically look at speaker information using PCA on utterance-averaged representations. For a range of SSL models, we find that the principal dimension that explains most variance encodes pitch and associated characteristics like gender. Other individual principal dimensions correlate with intensity, noise levels, the second formant, and higher frequency characteristics. We then use synthesis analyses to show that the dimensions for most characteristics are isolated from each other’s influence. We further show that characteristics can be changed by manipulating the corresponding dimensions.

Zitieren

APA

Rensburg, K. J. V., Niekerk, B. V., & Kamper, H. (2026). Interpreting Speaker Characteristics in the Dimensions of Self-Supervised Speech Features. IEEE Signal Processing Letters, 33, 3132-3136. https://doi.org/10.1109/lsp.2026.3716838

Vancouver

Rensburg KJV, Niekerk BV, Kamper H. Interpreting Speaker Characteristics in the Dimensions of Self-Supervised Speech Features. IEEE Signal Processing Letters. 2026;33:3132-3136. doi:10.1109/lsp.2026.3716838

BibTeX
@article{Rensburg2026,
  author = {Rensburg, Kyle Janse van and Niekerk, Benjamin van and Kamper, Herman},
  title = {Interpreting Speaker Characteristics in the Dimensions of Self-Supervised Speech Features},
  journal = {IEEE Signal Processing Letters},
  year = {2026},
  volume = {33},
  pages = {3132--3136},
  doi = {10.1109/lsp.2026.3716838}
}
Kostenlos starten →