EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Imbalance-Aware Cross-Modal Focal Modulation for Cross-Dataset Audio-Visual Deepfake Detection

Shahad Mohammad Bn Dokiey, Tariq M. Khan, Qazi Emad Ul Haq

Future Internet · 20.07.2026 · 18 (7) · 379

DOI: 10.3390/fi18070379 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Future Internet, online erschienen am 20.07.2026. Verfasst von Shahad Mohammad Bn Dokiey und 2 weiteren Autor:innen. Die Arbeit nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Naif Arab University for Security Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Audio-visual deepfake detection remains challenging under cross-dataset distribution shift, especially when the source-domain training data are severely imbalanced. Existing middle-fusion detectors often rely on softmax-based cross-attention, which can learn sharp source-domain token interactions and may transfer poorly to unseen datasets. This study proposes FocalNet, an audio-visual detector that replaces the cross-attention block of the 2D3MF framework with cross-modal focal modulation. The proposed module aggregates multi-scale temporal context before audio-visual interaction, enabling softmax-free contextual modulation between visual MARLIN features and audio EAT features. We evaluate the method under a strict FakeAVCeleb-to-DFDC protocol, where all training and validation is performed on FakeAVCeleb and the DFDC is used only as an unseen target-domain test set. Compared with the reproduced 2D3MF baseline, which collapses to a single-class prediction pattern on the DFDC, FocalNet achieves substantially stronger zero-shot score separation, with a DFDC ROC-AUC of 0.9324. Thresholded analysis further shows an improved balanced accuracy, macro F1 score, and MCC when the frozen source-domain operating point is applied. The model also preserves practical efficiency, requiring comparable FLOPs and lower per-sample inference time than the reproduced baseline. These findings suggest that cross-modal focal modulation is a promising alternative to attention-based middle fusion for audio-visual deepfake detection under dataset shifts, while broader validation across additional unseen datasets, multi-seed training, and deployment-oriented calibration remain important for future work.

Förderung

  • Naif Arab University for Security Sciences

Zitieren

APA

Dokiey, S. M. B., Khan, T. M., & Haq, Q. E. U. (2026). Imbalance-Aware Cross-Modal Focal Modulation for Cross-Dataset Audio-Visual Deepfake Detection. Future Internet, 18(7), 379. https://doi.org/10.3390/fi18070379

Vancouver

Dokiey SMB, Khan TM, Haq QEU. Imbalance-Aware Cross-Modal Focal Modulation for Cross-Dataset Audio-Visual Deepfake Detection. Future Internet. 2026;18(7):379. doi:10.3390/fi18070379

BibTeX
@article{Dokiey2026,
  author = {Dokiey, Shahad Mohammad Bn and Khan, Tariq M. and Haq, Qazi Emad Ul},
  title = {Imbalance-Aware Cross-Modal Focal Modulation for Cross-Dataset Audio-Visual Deepfake Detection},
  journal = {Future Internet},
  year = {2026},
  volume = {18},
  number = {7},
  pages = {379},
  doi = {10.3390/fi18070379}
}
Kostenlos starten →