Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation
JMIR Biomedical Engineering · 2024-03-21 · 9 · e56245
DOI: 10.2196/56245 ↗At a glance
Journal article in JMIR Biomedical Engineering, published online on 2024-03-21. Written by Nikhil Valsan Kulangareth and 3 co-authors. Institution: Klick Labs, Toronto, ON, Canada. Cited 24 times so far (Crossref); 20 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
The digital era has witnessed an escalating dependence on digital platforms for news and information, coupled with the advent of “deepfake” technology. Deepfakes, leveraging deep learning models on extensive data sets of voice recordings and images, pose substantial threats to media authenticity, potentially leading to unethical misuse such as impersonation and the dissemination of false information.
Objective
To counteract this challenge, this study aims to introduce the concept of innate biological processes to discern between authentic human voices and cloned voices. We propose that the presence or absence of certain perceptual features, such as pauses in speech, can effectively distinguish between cloned and authentic audio.
Methods
A total of 49 adult participants representing diverse ethnic backgrounds and accents were recruited. Each participant contributed voice samples for the training of up to 3 distinct voice cloning text-to-speech models and 3 control paragraphs. Subsequently, the cloning models generated synthetic versions of the control paragraphs, resulting in a data set consisting of up to 9 cloned audio samples and 3 control samples per participant. We analyzed the speech pauses caused by biological actions such as respiration, swallowing, and cognitive processes. Five audio features corresponding to speech pause profiles were calculated. Differences between authentic and cloned audio for these features were assessed, and 5 classical machine learning algorithms were implemented using these features to create a prediction model. The generalization capability of the optimal model was evaluated through testing on unseen data, incorporating a model-naive generator, a model-naive paragraph, and model-naive participants.
Results
Cloned audio exhibited significantly increased time between pauses (P
Subjects
Author keywords
Authors and institutions
- Klick Labs, Toronto, ON, Canada.
ORCID profiles: Nikhil Valsan Kulangareth, Jaycee Kaufman, Jessica Oreskovic, Yan Fossat
Cite
APA
Kulangareth, N. V., Kaufman, J., Oreskovic, J., & Fossat, Y. (2024). Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation. JMIR Biomedical Engineering, 9, e56245. https://doi.org/10.2196/56245
Vancouver
Kulangareth NV, Kaufman J, Oreskovic J, Fossat Y. Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation. JMIR Biomedical Engineering. 2024;9:e56245. doi:10.2196/56245
BibTeX
@article{Kulangareth2024,
author = {Kulangareth, Nikhil Valsan and Kaufman, Jaycee and Oreskovic, Jessica and Fossat, Yan},
title = {Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation},
journal = {JMIR Biomedical Engineering},
year = {2024},
volume = {9},
pages = {e56245},
doi = {10.2196/56245}
}
EUVIMED