A Review on Methods and Applications in Multimodal Deep Learning
ACM Transactions on Multimedia Computing, Communications, and Applications · 30.06.2023 · 19 (2s) · 1-41
DOI: 10.1145/3545572 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 17.02.2023. Verfasst von Summaira Jabeen und 5 weiteren Autor:innen. Die Arbeit wurde bisher 204-mal zitiert (Crossref) und nennt 172 Literaturangaben. Gefördert von: Zhejiang Provincial Natural Science Foundation of China, National Key Research and Development Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Deep Learning has implemented a wide range of applications and has become increasingly popular in recent years. The goal of multimodal deep learning (MMDL) is to create models that can process and link information using various modalities. Despite the extensive development made for unimodal learning, it still cannot cover all the aspects of human learning. Multimodal learning helps to understand and analyze better when various senses are engaged in the processing of information. This article focuses on multiple types of modalities, i.e., image, video, text, audio, body gestures, facial expressions, physiological signals, flow, RGB, pose, depth, mesh, and point cloud. Detailed analysis of the baseline approaches and an in-depth study of recent advancements during the past five years (2017 to 2021) in multimodal deep learning applications has been provided. A fine-grained taxonomy of various multimodal deep learning methods is proposed, elaborating on different applications in more depth. Last, main issues are highlighted separately for each domain, along with their possible future research directions.
Förderung
- Zhejiang Provincial Natural Science Foundation of China
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- National Science Foundation for Distinguished Young Scholars
- Ant Group
- CAAI-HUAWEI MindSpore Open Fund
Zitieren
APA
Jabeen, S., Li, X., Amin, M. S., Bourahla, O., Li, S., & Jabbar, A. (2023). A Review on Methods and Applications in Multimodal Deep Learning. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(2s), 1-41. https://doi.org/10.1145/3545572
Vancouver
Jabeen S, Li X, Amin MS, Bourahla O, Li S, Jabbar A. A Review on Methods and Applications in Multimodal Deep Learning. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(2s):1-41. doi:10.1145/3545572
BibTeX
@article{Jabeen2023,
author = {Jabeen, Summaira and Li, Xi and Amin, Muhammad Shoib and Bourahla, Omar and Li, Songyuan and Jabbar, Abdul},
title = {A Review on Methods and Applications in Multimodal Deep Learning},
journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
year = {2023},
volume = {19},
number = {2s},
pages = {1--41},
doi = {10.1145/3545572}
}
EUVIMED