EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Effective Techniques for Multimodal Data Fusion: A Comparative Analysis

Maciej Pawłowski iD, Anna Wróblewska iD, Sylwia Sysko-Romańczuk iD

Sensors · 21.02.2023 · 23 (5) · 2381

DOI: 10.3390/s23052381 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 21.02.2023. Verfasst von Maciej Pawłowski und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa Street 75, 00-662 Warsaw, Poland. Die Arbeit wurde bisher 193-mal zitiert (Crossref) und nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Centre for Priority Research Area Artificial Intelligence and Robotics of Warsaw University of Technology within the Excellence Initiative: Research University…, The National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020, National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020, Digital Innovations.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Data processing in robotics is currently challenged by the effective building of multimodal and common representations. Tremendous volumes of raw data are available and their smart management is the core concept of multimodal learning in a new paradigm for data fusion. Although several techniques for building multimodal representations have been proven successful, they have not yet been analyzed and compared in a given production setting. This paper explored three of the most common techniques, (1) the late fusion, (2) the early fusion, and (3) the sketch, and compared them in classification tasks. Our paper explored different types of data (modalities) that could be gathered by sensors serving a wide range of sensor applications. Our experiments were conducted on Amazon Reviews, MovieLens25M, and Movie-Lens1M datasets. Their outcomes allowed us to confirm that the choice of fusion technique for building multimodal representation is crucial to obtain the highest possible model performance resulting from the proper modality combination. Consequently, we designed criteria for choosing this optimal data fusion technique.

Schlagwörter

Stichwörter der Autor:innen

comparative analysisdata fusiondeep learning in sensor systemsmultimodal learningmultimodal representationneural networks

Autor:innen und Institutionen

  • Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa Street 75, 00-662 Warsaw, Poland.
  • WeSub, Adama Branickiego Street 17, 02-972 Warsaw, Poland.
  • Faculty of Management, Warsaw University of Technology, Narbutta Street 85, 02-524 Warsaw, Poland.

ORCID-Profile: Maciej Pawłowski, Anna Wróblewska, Sylwia Sysko-Romańczuk

Förderung

  • Centre for Priority Research Area Artificial Intelligence and Robotics of Warsaw University of Technology within the Excellence Initiative: Research University…
  • The National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020
  • National Centre for Research and Development in Poland, Smart Growth Operational Program for 2014-2020, Digital Innovations

Zitieren

APA

Pawłowski, M., Wróblewska, A., & Sysko-Romańczuk, S. (2023). Effective Techniques for Multimodal Data Fusion: A Comparative Analysis. Sensors, 23(5), 2381. https://doi.org/10.3390/s23052381

Vancouver

Pawłowski M, Wróblewska A, Sysko-Romańczuk S. Effective Techniques for Multimodal Data Fusion: A Comparative Analysis. Sensors. 2023;23(5):2381. doi:10.3390/s23052381

BibTeX
@article{Pawowski2023,
  author = {Pawłowski, Maciej and Wróblewska, Anna and Sysko-Romańczuk, Sylwia},
  title = {Effective Techniques for Multimodal Data Fusion: A Comparative Analysis},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {5},
  pages = {2381},
  doi = {10.3390/s23052381}
}
Kostenlos starten →