EUVIMEDEUVIMED entdecken
Crossref

Image-text crisis tweet categorization:a caption-based approach

Badreddine Farah, Guillaume Cleuziou, Cécile Gracianne, Adel Hafiane, Anaïs Halftermeyer, Raphaël Canals

Proceedings of the International ISCRAM Conference · 17.05.2024 · 21

DOI: 10.59297/9j4kjp22 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the International ISCRAM Conference, online erschienen am 17.05.2024. Verfasst von Badreddine Farah und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The growth of social media usage this last decade has made available a massive and valuable volume of multimedia data. However, the lack of large multimodal annotated datasets, along with the inherent noise and the diversity of multimodal relations in this type of data presents challenges for machine learning methods. Unlike classic multimodal data, social media data comes with a large diversity of relations between image and text making the interaction between the two modalities more difficult.Previous research concentrated on fusion strategies with separate encoders for each modality. This paper introduces CMB (Caption-based Multimodal BERT), a method of classifying crisis-related social media posts by utilizing information from both images and texts. CMB translates the image modality into a text-compatible space, facilitating intermodal interaction. Furthermore, CMB presents training opportunities to enhance the model's robustness to missing modalities. Experimental results show that CMB is competitive with well-established, costly, and manually crafted multimodal models.

Zitieren

APA

Farah, B., Cleuziou, G., Gracianne, C., Hafiane, A., Halftermeyer, A., & Canals, R. (2024). Image-text crisis tweet categorization:a caption-based approach. Proceedings of the International ISCRAM Conference, 21. https://doi.org/10.59297/9j4kjp22

Vancouver

Farah B, Cleuziou G, Gracianne C, Hafiane A, Halftermeyer A, Canals R. Image-text crisis tweet categorization:a caption-based approach. Proceedings of the International ISCRAM Conference. 2024;21. doi:10.59297/9j4kjp22

BibTeX
@article{Farah2024,
  author = {Farah, Badreddine and Cleuziou, Guillaume and Gracianne, Cécile and Hafiane, Adel and Halftermeyer, Anaïs and Canals, Raphaël},
  title = {Image-text crisis tweet categorization:a caption-based approach},
  journal = {Proceedings of the International ISCRAM Conference},
  year = {2024},
  volume = {21},
  doi = {10.59297/9j4kjp22}
}
Kostenlos starten →