Effective Techniques for Multimodal Data Fusion: A Comparative Analysis
Sensors · 21.02.2023 · 23 (5) · 2381
DOI: 10.3390/s23052381 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 21.02.2023. Verfasst von Maciej Pawłowski und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa Street 75, 00-662 Warsaw, Poland. Die Arbeit wurde bisher 193-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Centre for Priority Research Area Artificial Intelligence and Robotics of Warsaw University of Technology within the Excellence Initiative: Research University…, The National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020, National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020, Digital Innovations.
Abstract (Originalsprache)
Data processing in robotics is currently challenged by the effective building of multimodal and common representations. Tremendous volumes of raw data are available and their smart management is the core concept of multimodal learning in a new paradigm for data fusion. Although several techniques for building multimodal representations have been proven successful, they have not yet been analyzed and compared in a given production setting. This paper explored three of the most common techniques, (1) the late fusion, (2) the early fusion, and (3) the sketch, and compared them in classification tasks. Our paper explored different types of data (modalities) that could be gathered by sensors serving a wide range of sensor applications. Our experiments were conducted on Amazon Reviews, MovieLens25M, and Movie-Lens1M datasets. Their outcomes allowed us to confirm that the choice of fusion technique for building multimodal representation is crucial to obtain the highest possible model performance resulting from the proper modality combination. Consequently, we designed criteria for choosing this optimal data fusion technique.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa Street 75, 00-662 Warsaw, Poland.
- WeSub, Adama Branickiego Street 17, 02-972 Warsaw, Poland.
- Faculty of Management, Warsaw University of Technology, Narbutta Street 85, 02-524 Warsaw, Poland.
ORCID-Profile: Maciej Pawłowski, Anna Wróblewska, Sylwia Sysko-Romańczuk
Förderung
- Centre for Priority Research Area Artificial Intelligence and Robotics of Warsaw University of Technology within the Excellence Initiative: Research University…
- The National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020
- National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020, Digital Innovations
Zitieren
APA
Pawłowski, M., Wróblewska, A., & Sysko-Romańczuk, S. (2023). Effective Techniques for Multimodal Data Fusion: A Comparative Analysis. Sensors, 23(5), 2381. https://doi.org/10.3390/s23052381
Vancouver
Pawłowski M, Wróblewska A, Sysko-Romańczuk S. Effective Techniques for Multimodal Data Fusion: A Comparative Analysis. Sensors. 2023;23(5):2381. doi:10.3390/s23052381
BibTeX
@article{Pawowski2023,
author = {Pawłowski, Maciej and Wróblewska, Anna and Sysko-Romańczuk, Sylwia},
title = {Effective Techniques for Multimodal Data Fusion: A Comparative Analysis},
journal = {Sensors},
year = {2023},
volume = {23},
number = {5},
pages = {2381},
doi = {10.3390/s23052381}
}
EUVIMED