EUVIMEDEUVIMED entdecken
Systematische ÜbersichtMetaanalyseOpen Access · CC BY-NC-ND 4.0

Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers

Alexander Diel, Tania Lalgi, Isabel Carolin Schröter, Karl F. MacDorman, Martin Teufel, Alexander Bäuerle

Computers in Human Behavior Reports · 2024-12 · 16 · 100538

DOI: 10.1016/j.chbr.2024.100538 ↗

Kurz gefasst

Systematische Übersicht in der Zeitschrift Computers in Human Behavior Reports, 2024-12. Verfasst von Alexander Diel und 5 weiteren Autor:innen. Die Arbeit wurde bisher 75-mal zitiert (Crossref) und nennt 122 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deepfakes are AI-generated media designed to look real, often with the intent to deceive. Deepfakes threaten public and personal safety by facilitating disinformation, propaganda, and identity theft. Though research has been conducted on human performance in deepfake detection, the results have not yet been synthesized. This systematic review and meta-analysis investigates human deepfake detection accuracy. Searches in PubMed, ScienceGov, JSTOR, Google Scholar, and paper references, conducted in June and October 2024, identified empirical studies measuring human detection of high-quality deepfakes. After pooling accuracy, odds-ratio, and sensitivity (d') effect sizes (k = 137 effects) from 56 papers involving 86,155 participants, we analyzed 1) overall deepfake detection performance, 2) performance across stimulus types (audio, image, text, and video), and 3) the effects of detection-improvement strategies. Overall deepfake detection rates (sensitivity) were not significantly above chance because 95% confidence intervals crossed 50%. Total deepfake detection accuracy was 55.54% (95% CI [48.87, 62.10], k = 67). For audio, accuracy was 62.08% [38.23, 83.18], k = 8; for images, 53.16% [42.12, 64.64], k = 18; for text, 52.00% [37.42, 65.88], k = 15; and for video, 57.31% [47.80, 66.57], k = 26. Odds ratios were 0.64 [0.52, 0.79], k = 62, indicating 39% detection accuracy, below chance (audio 45%, image 35%, text 40%, video 40%). Moreover, d' values show no significant difference from chance. However, strategies like feedback training, AI support, and deepfake caricaturization improved detection performance above chance levels (65.14% [55.21, 74.46], k = 15), especially for video stimuli.

Zitieren

APA

Diel, A., Lalgi, T., Schröter, I. C., MacDorman, K. F., Teufel, M., & Bäuerle, A. (2024). Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers. Computers in Human Behavior Reports, 16, 100538. https://doi.org/10.1016/j.chbr.2024.100538

Vancouver

Diel A, Lalgi T, Schröter IC, MacDorman KF, Teufel M, Bäuerle A. Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers. Computers in Human Behavior Reports. 2024;16:100538. doi:10.1016/j.chbr.2024.100538

BibTeX
@article{Diel2024,
  author = {Diel, Alexander and Lalgi, Tania and Schröter, Isabel Carolin and MacDorman, Karl F. and Teufel, Martin and Bäuerle, Alexander},
  title = {Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers},
  journal = {Computers in Human Behavior Reports},
  year = {2024},
  volume = {16},
  pages = {100538},
  doi = {10.1016/j.chbr.2024.100538}
}
Kostenlos starten →