Data Security Identification Based on Full-Dimensional Dynamic Convolution and Multi-Modal CLIP
Journal of Information, Technology and Policy · 2023
DOI: 10.62836/jitp.2023.430 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Information, Technology and Policy, 2023. Verfasst von Qinyi Zhu und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This paper addresses key challenges in data security and privacy protection in multimodal recognition within the current field of artificial intelligence. We propose a data security recognition method that integrates Omni-Dimensional Dynamic Convolution (ODConv) with a multimodal CLIP model. The method targets three biometric modalities—face, voiceprint, and behavior—by constructing a unified multimodal recognition framework. To effectively mask and protect users’ sensitive information, a Variational Autoencoder (VAE) is introduced to perturb and compress the raw modality data. In the feature extraction and fusion stage, ODConv replaces traditional convolutional structures, enhancing the model’s adaptive capability to semantic heterogeneity across different modalities. Meanwhile, leveraging CLIP’s cross-modal alignment mechanism, semantic-level fusion of face, voice, and behavior is achieved, improving the model’s understanding and recognition of identity information in complex scenarios. Experiments conducted on multiple public multimodal datasets systematically evaluate reconstruction error, recognition accuracy, and robustness against adversarial attacks. Results demonstrate that the proposed method maintains recognition performance while effectively reducing sensitive information leakage risks during model inversion and reconstruction attacks, validating its practicality and robustness in data security scenarios. This study provides a feasible pathway and technical reference for the trustworthy deployment of multimodal biometric recognition systems under privacy protection constraints.
Zitieren
APA
Zhu, Q., & Shao, D. (2023). Data Security Identification Based on Full-Dimensional Dynamic Convolution and Multi-Modal CLIP. Journal of Information, Technology and Policy. https://doi.org/10.62836/jitp.2023.430
Vancouver
Zhu Q, Shao D. Data Security Identification Based on Full-Dimensional Dynamic Convolution and Multi-Modal CLIP. Journal of Information, Technology and Policy. 2023. doi:10.62836/jitp.2023.430
BibTeX
@article{Zhu2023,
author = {Zhu, Qinyi and Shao, Dan},
title = {Data Security Identification Based on Full-Dimensional Dynamic Convolution and Multi-Modal CLIP},
journal = {Journal of Information, Technology and Policy},
year = {2023},
doi = {10.62836/jitp.2023.430}
}
EUVIMED