Evolution of Performance Metrics for Accurate Evaluation of Speech-to-Speech Translation Models: A Literature Review
London Journal Of Engineering Research · 2025
DOI: 10.34257/ljervol25is4pg65 ↗Kurz gefasst
Fachartikel in der Zeitschrift London Journal Of Engineering Research, 2025. Verfasst von Gabriel O. Sobola und 7 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The translation of speech from a source to speech in a target language with generative artificial intelligence is an area of research that is presently being actively explored. This is aimed at solving global language barriers thereby ensuring seamless communication between the individuals involved. It has been well developed for high-resourced languages like English, Spanish, French and Chinese. Currently, objective evaluation metrics such as Bilingual Evaluation Understudy Scores (BLEUS), and subjective metrics such as Mean Opinion Score Naturalness (MOSN) and Mean Opinion Score Similarity (MOSS) are being used to evaluate the performance of the output of speech-to-speech models. However, low resourced languages are still undeveloped in the area of speech processing applications, especially the African indigenous languages. The output speech in the target language needs to be evaluated to determine the closeness to the ground truth, as well as how natural and intelligible it is to the intended listeners. This paper presents a review of trends from the current metrics to emerging ones such as Recall Oriented Understudy for Gisting Evaluation-L (ROUGE-L) and BLASER. The applications of speech models’ metrics on various leaderboards and modern AI platforms were also discussed. The outcome shows that while BLEU score and MOSN metrics are prevalent for speech models, there is a need to explore metrics such as ROUGE-L, and BERTScore which are machine translation metric due to their benefits.
Zitieren
APA
Sobola, G. O., Adetiba, E., Idowu-Bismark, O., Ajayi, O. O., Kala, R. J., Abayomi, A. T., Oshin, O., & Olaitan, O. (2025). Evolution of Performance Metrics for Accurate Evaluation of Speech-to-Speech Translation Models: A Literature Review. London Journal Of Engineering Research. https://doi.org/10.34257/ljervol25is4pg65
Vancouver
Sobola GO, Adetiba E, Idowu-Bismark O, Ajayi OO, Kala RJ, Abayomi AT, et al. Evolution of Performance Metrics for Accurate Evaluation of Speech-to-Speech Translation Models: A Literature Review. London Journal Of Engineering Research. 2025. doi:10.34257/ljervol25is4pg65
BibTeX
@article{Sobola2025,
author = {Sobola, Gabriel O. and Adetiba, Emmanuel and Idowu-Bismark, Olabode and Ajayi, Oluseyi O. and Kala, Raymond Jules and Abayomi, Abdul taofeek and Oshin, Oluwadamilola and Olaitan, Olutoyin},
title = {Evolution of Performance Metrics for Accurate Evaluation of Speech-to-Speech Translation Models: A Literature Review},
journal = {London Journal Of Engineering Research},
year = {2025},
doi = {10.34257/ljervol25is4pg65}
}
EUVIMED