EUVIMEDOdkrijte EUVIMED
Open access

Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation

Nikhil Valsan Kulangareth iD, Jaycee Kaufman iD, Jessica Oreskovic iD, Yan Fossat iD

JMIR Biomedical Engineering · 2024-03-21 · 9 · e56245

DOI: 10.2196/56245 ↗

At a glance

Journal article in JMIR Biomedical Engineering, published online on 2024-03-21. Written by Nikhil Valsan Kulangareth and 3 co-authors. Institution: Klick Labs, Toronto, ON, Canada. Cited 24 times so far (Crossref); 20 references. The full text is freely available in PubMed Central.

PovzetekSubjectsBibliografski podatkiCiteRelated studies

Abstract (original language)

Background

The digital era has witnessed an escalating dependence on digital platforms for news and information, coupled with the advent of “deepfake” technology. Deepfakes, leveraging deep learning models on extensive data sets of voice recordings and images, pose substantial threats to media authenticity, potentially leading to unethical misuse such as impersonation and the dissemination of false information.

Objective

To counteract this challenge, this study aims to introduce the concept of innate biological processes to discern between authentic human voices and cloned voices. We propose that the presence or absence of certain perceptual features, such as pauses in speech, can effectively distinguish between cloned and authentic audio.

Methods

A total of 49 adult participants representing diverse ethnic backgrounds and accents were recruited. Each participant contributed voice samples for the training of up to 3 distinct voice cloning text-to-speech models and 3 control paragraphs. Subsequently, the cloning models generated synthetic versions of the control paragraphs, resulting in a data set consisting of up to 9 cloned audio samples and 3 control samples per participant. We analyzed the speech pauses caused by biological actions such as respiration, swallowing, and cognitive processes. Five audio features corresponding to speech pause profiles were calculated. Differences between authentic and cloned audio for these features were assessed, and 5 classical machine learning algorithms were implemented using these features to create a prediction model. The generalization capability of the optimal model was evaluated through testing on unseen data, incorporating a model-naive generator, a model-naive paragraph, and model-naive participants.

Results

Cloned audio exhibited significantly increased time between pauses (P

Subjects

Author keywords

artificial intelligenceaudioclonedcloningdeep learningdeepfakedeepfakesmachine learningmodel-naivesoundsoundsspeechtext to speechvocalvocal biomarkers

Authors and institutions

  • Klick Labs, Toronto, ON, Canada.

ORCID profiles: Nikhil Valsan Kulangareth, Jaycee Kaufman, Jessica Oreskovic, Yan Fossat

Cite

APA

Kulangareth, N. V., Kaufman, J., Oreskovic, J., & Fossat, Y. (2024). Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation. JMIR Biomedical Engineering, 9, e56245. https://doi.org/10.2196/56245

Vancouver

Kulangareth NV, Kaufman J, Oreskovic J, Fossat Y. Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation. JMIR Biomedical Engineering. 2024;9:e56245. doi:10.2196/56245

BibTeX
@article{Kulangareth2024,
  author = {Kulangareth, Nikhil Valsan and Kaufman, Jaycee and Oreskovic, Jessica and Fossat, Yan},
  title = {Investigation of Deepfake Voice Detection Using Speech Pause Patterns: Algorithm Development and Validation},
  journal = {JMIR Biomedical Engineering},
  year = {2024},
  volume = {9},
  pages = {e56245},
  doi = {10.2196/56245}
}
Start for free →