Detecting Deepfake-Enabled Financial Fraud Through Cross-Modal Consistency and Reliability-Aware Fusion
Natural Resources for Human Health · 03.11.2026 · 6 (21s)
DOI: 10.53365/nrfhh.4412 ↗Kurz gefasst
Fachartikel in der Zeitschrift Natural Resources for Human Health, online erschienen am 03.11.2026. Verfasst von Akshay N. Shetye und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Remote onboarding, video-based Know-Your-Customer (Video-KYC) checks and voice-authorized payments now let a fraudster win a customer’s or an executive’s identity without ever being physically present. Synthetic faces and cloned voices are cheap to produce, yet most defenses still look at a single signal: deepfake detectors judge the media in isolation, and fraud engines judge only the transaction. This paper presents DeepShield-Fin, a cross-modal framework that reasons jointly over video, audio, interaction behavior, session context and transaction attributes during a live financial session. A Cross-Modal Consistency Analysis (CMCA) module turns five explicit agreement tests (lip–speech synchrony, face–voice identity, facial–vocal affect, behavior–context plausibility and transaction deviation) into an interpretable evidence vector. An Adaptive Gated Cross-Attention Fusion (AGCAF) module then weights each modality by an estimated reliability, so that a noisy microphone or a dropped video frame does not dominate the decision. A calibrated risk head drives a three-tier allow/step-up/block policy, and an Evidence Explanation Engine produces analyst-readable justifications. We specify DeepShield-FinDB, a 5,000-session benchmark with an explicit label-leakage audit, and a five-protocol validation plan. Preliminary validation uses the implemented audio branch and prototype: on 22,622 benchmark clips the audio branch reaches 95.2% accuracy (95% CI 94.9–95.5%), an MCC of 0.904 and a 4.8% EER, against 7.9–8.5% EER for HuBERT, wav2vec 2.0 and RawNet2, and the prototype completes an analysis window in 1.84 s. An analytical deployment study shows why single-channel detection is not enough: at a 1% fraud rate even this detector would raise about five false alarms per detected fraud, which motivates the cross-modal evidence that DeepShield-Fin adds.
Zitieren
APA
Shetye, A. N., & Jain, N. (2026). Detecting Deepfake-Enabled Financial Fraud Through Cross-Modal Consistency and Reliability-Aware Fusion. Natural Resources for Human Health, 6(21s). https://doi.org/10.53365/nrfhh.4412
Vancouver
Shetye AN, Jain N. Detecting Deepfake-Enabled Financial Fraud Through Cross-Modal Consistency and Reliability-Aware Fusion. Natural Resources for Human Health. 2026;6(21s). doi:10.53365/nrfhh.4412
BibTeX
@article{Shetye2026,
author = {Shetye, Akshay N. and Jain, Nilakshi},
title = {Detecting Deepfake-Enabled Financial Fraud Through Cross-Modal Consistency and Reliability-Aware Fusion},
journal = {Natural Resources for Human Health},
year = {2026},
volume = {6},
number = {21s},
doi = {10.53365/nrfhh.4412}
}
EUVIMED