EUVIMEDEUVIMED entdecken
Open Access · CC BY 3.0

A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval

Saeed Alzahrani, Farah Mohammad, Nazar Hussain

Journal of Organizational and End User Computing · 24.07.2026 · 38 (1) · 1-22

DOI: 10.4018/joeuc.416655 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Organizational and End User Computing, erschienen am 24.07.2026. Verfasst von Saeed Alzahrani und 2 weiteren Autor:innen. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 3.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual search systems often rely on image-only embeddings, limiting semantic understanding—especially with visually similar but semantically distinct items. To overcome this, the authors propose a unified multimodal search framework that integrates generative AI-based captioning and image understanding for improved retrieval. Using Qwen2-VL-7B, the system generates rich captions from images, then fuses visual and textual embeddings via cross-attention. A re-ranking module refines the results. Evaluated on COCO Captions and Fashion-Gen, the model achieves BLEU scores of 0.78 and 0.75, CIDEr scores of 1.12 and 1.08, and retrieval mAP of 0.75 and 0.73. It also records NDCG@10 of 0.91 and 0.89, outperforming baselines like FineCaption, SuperCap, and MM-Transformer by up to 9%. These results validate the approach in bridging the semantic gap between images and text, enabling more accurate, context-aware search in applications such as e-commerce, multimedia, and large-scale retrieval.

Zitieren

APA

Alzahrani, S., Mohammad, F., & Hussain, N. (2026). A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval. Journal of Organizational and End User Computing, 38(1), 1-22. https://doi.org/10.4018/joeuc.416655

Vancouver

Alzahrani S, Mohammad F, Hussain N. A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval. Journal of Organizational and End User Computing. 2026;38(1):1-22. doi:10.4018/joeuc.416655

BibTeX
@article{Alzahrani2026,
  author = {Alzahrani, Saeed and Mohammad, Farah and Hussain, Nazar},
  title = {A Unified Multimodal Search Framework Using Generative AI and Image Understanding for Enhanced Information Retrieval},
  journal = {Journal of Organizational and End User Computing},
  year = {2026},
  volume = {38},
  number = {1},
  pages = {1--22},
  doi = {10.4018/joeuc.416655}
}
Kostenlos starten →