A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval
Journal of Organizational and End User Computing · 24.07.2026 · 38 (1) · 1-22
DOI: 10.4018/joeuc.416655 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Organizational and End User Computing, erschienen am 24.07.2026. Verfasst von Saeed Alzahrani und 2 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 3.0).
Abstract (Originalsprache)
Visual search systems often rely on image-only embeddings, limiting semantic understanding—especially with visually similar but semantically distinct items. To overcome this, the authors propose a unified multimodal search framework that integrates generative AI-based captioning and image understanding for improved retrieval. Using Qwen2-VL-7B, the system generates rich captions from images, then fuses visual and textual embeddings via cross-attention. A re-ranking module refines the results. Evaluated on COCO Captions and Fashion-Gen, the model achieves BLEU scores of 0.78 and 0.75, CIDEr scores of 1.12 and 1.08, and retrieval mAP of 0.75 and 0.73. It also records NDCG@10 of 0.91 and 0.89, outperforming baselines like FineCaption, SuperCap, and MM-Transformer by up to 9%. These results validate the approach in bridging the semantic gap between images and text, enabling more accurate, context-aware search in applications such as e-commerce, multimedia, and large-scale retrieval.
Zitieren
APA
Alzahrani, S., Mohammad, F., & Hussain, N. (2026). A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval. Journal of Organizational and End User Computing, 38(1), 1-22. https://doi.org/10.4018/joeuc.416655
Vancouver
Alzahrani S, Mohammad F, Hussain N. A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval. Journal of Organizational and End User Computing. 2026;38(1):1-22. doi:10.4018/joeuc.416655
BibTeX
@article{Alzahrani2026,
author = {Alzahrani, Saeed and Mohammad, Farah and Hussain, Nazar},
title = {A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval},
journal = {Journal of Organizational and End User Computing},
year = {2026},
volume = {38},
number = {1},
pages = {1--22},
doi = {10.4018/joeuc.416655}
}
EUVIMED