Improving the efficiency of Whisper-based audio stream processing with CTranslate2 and FFMpeg tools
Information Technology and Computer Engineering · 20.04.2026 · (23) · 110-124
DOI: 10.31649/vitce/1.2026.110 ↗Kurz gefasst
Fachartikel in der Zeitschrift Information Technology and Computer Engineering, online erschienen am 20.04.2026. Verfasst von Vladyslav Radin und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The relevance of the study lies in the need to increase the performance and scalability of automatic speech recognition systems on devices with limited resources, which determines the goal of the work – to optimise Whisper by integrating CTranslate2 to accelerate calculations and FFmpeg for unified preparation of audio data. Experimental studies were conducted using the Whisper Turbo model on a graphics processor unit with support for the Compute Unified Device Architecture (CUDA) platform. The basic pipeline in the Python programming language, the optimised inference execution mechanism via CTranslate2 and the configuration with hybrid quantisation in the int8_float16 format were compared. The efficiency was evaluated using the indicators of prediction (inference) execution time, video memory use, and automatic speech recognition accuracy (Word Error Rate). Experimental results showed that the basic Whisper Turbo configuration provided the highest recognition accuracy (Word Error Rate = 0), but was characterised by high inference latency (8.5 s per audio file) and significant video memory consumption (4.9 GB). CTranslate2 integration reduced the processing time to 4.9 s (1.7 × speedup) and reduced Video Random Access Memory usage to 1.8 GB (-63%) without loss of quality. Further application of hybrid quantisation int8_float16 provided a reduction of inference time to 3.8 s and a reduction of memory consumption to 1 GB, which corresponds to an overall speedup of about 2.2× and an almost fivefold (4.9×) reduction in Video Random Access Memory requirements compared to the standard implementation, with unchanged Word Error Rate = 0. The obtained results confirmed the effectiveness of the combination of CTranslate2 and hybrid quantisation for building high-performance real-time Automatic Speech Recognition systems without compromising accuracy. The conclusions confirmed the practical suitability of the proposed configuration for multiuser services and edge scenarios without compromising speed and accuracy. The results of the study can be used by developers of automatic speech recognition systems to optimise models on memory-limited GPUs, and by companies providing streaming audio and multi-user services
Zitieren
APA
Radin, V., & Riabyi, M. (2026). Improving the efficiency of Whisper-based audio stream processing with CTranslate2 and FFMpeg tools. Information Technology and Computer Engineering, 110-124. https://doi.org/10.31649/vitce/1.2026.110
Vancouver
Radin V, Riabyi M. Improving the efficiency of Whisper-based audio stream processing with CTranslate2 and FFMpeg tools. Information Technology and Computer Engineering. 2026:110-124. doi:10.31649/vitce/1.2026.110
BibTeX
@article{Radin2026,
author = {Radin, Vladyslav and Riabyi, Myroslav},
title = {Improving the efficiency of Whisper-based audio stream processing with CTranslate2 and FFMpeg tools},
journal = {Information Technology and Computer Engineering},
year = {2026},
number = {23},
pages = {110--124},
doi = {10.31649/vitce/1.2026.110}
}
EUVIMED