EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Multimodal Transfer Learning for Crisis Response: A Unified DeBERTa–ResNet Framework on CrisisMMD

Haffiz Ud Din, Musharraf Ali, Anzal Hussain, Akmal Hussain, Syed Qamar Abbas Kazmi, Hannan Hussain, Syed Hassan Abbas, Sohail Ahmed

International Journal of Emerging Engineering and Technology · 13.11.2026 · 5 (1-1) · 60-71

DOI: 10.57041/8fpqd649 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Emerging Engineering and Technology, online erschienen am 13.11.2026. Verfasst von Haffiz Ud Din und 7 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Social media platforms generate large volumes of multimodal content during natural disasters that can support situational awareness and humanitarian decision-making. However, effectively analyzing both textual and visual information remains challenging due to weak cross-modal alignment, severe class imbalance, noisy user-generated data, and limited labeled samples. This paper presents a unified multimodal transfer-learning framework that combines a frozen DeBERTa-v3-base text encoder with a frozen ResNet-50 image encoder through a simple late-fusion strategy for crisis-related classification. The architecture projects the unimodal features into a common 256-dimensional space, concatenates them, and feeds the resulting representation into a lightweight multi-layer perceptron classifier. The framework is evaluated on all three official tasks of the CrisisMMD v2.0 dataset. Experimental results show an accuracy of 0.76 (macro-F1 = 0.74) on the informativeness task, 0.43 (macro-F1 = 0.39) on the humanitarian categories task, and 0.69 (macro-F1 = 0.64) on the damage severity task. The model performs relatively well on classes with distinctive multimodal signatures, while residual confusion remains on semantically overlapping or visually subtle categories. For the damage severity task, which is primarily image-based, the multimodal model achieves essentially the same performance as an image-only configuration, indicating limited contribution from text. The study also acknowledges that formal text-only and image-only ablation experiments under the same protocol were not conducted, so the precise benefit of fusion cannot be rigorously quantified. Overall, the work demonstrates that a computationally efficient frozen late-fusion architecture can serve as a practical and reproducible reference point for multimodal crisis informatics under the resource constraints typical of real-world disaster response.

Zitieren

APA

Din, H. U., Ali, M., Hussain, A., Hussain, A., Kazmi, S. Q. A., Hussain, H., Abbas, S. H., & Ahmed, S. (2026). Multimodal Transfer Learning for Crisis Response: A Unified DeBERTa–ResNet Framework on CrisisMMD. International Journal of Emerging Engineering and Technology, 5(1-1), 60-71. https://doi.org/10.57041/8fpqd649

Vancouver

Din HU, Ali M, Hussain A, Hussain A, Kazmi SQA, Hussain H, et al. Multimodal Transfer Learning for Crisis Response: A Unified DeBERTa–ResNet Framework on CrisisMMD. International Journal of Emerging Engineering and Technology. 2026;5(1-1):60-71. doi:10.57041/8fpqd649

BibTeX
@article{Din2026,
  author = {Din, Haffiz Ud and Ali, Musharraf and Hussain, Anzal and Hussain, Akmal and Kazmi, Syed Qamar Abbas and Hussain, Hannan and Abbas, Syed Hassan and Ahmed, Sohail},
  title = {Multimodal Transfer Learning for Crisis Response: A Unified DeBERTa–ResNet Framework on CrisisMMD},
  journal = {International Journal of Emerging Engineering and Technology},
  year = {2026},
  volume = {5},
  number = {1-1},
  pages = {60--71},
  doi = {10.57041/8fpqd649}
}
Kostenlos starten →