AugMMRev: An LLM-Augmented Multimodal Ranking Model for Personalized Image Material Retrieval
IEEE Transactions on Consumer Electronics · 2026-02 · 72 (1) · 2347-2359
DOI: 10.1109/tce.2026.3652186 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Consumer Electronics, 2026-02. Verfasst von Yu Li und 8 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: National Key Research and Development Project of China, Pioneer and Leading Goose Research and Development Program of Zhejiang, Natural Science Foundation of Zhejiang University of Science and Technology.
Abstract (Originalsprache)
Consumer electronics devices—including smartphones and smart cameras—generate massive volumes of image data. Image retrieval serves as a critical enabling technology for diverse image-centric applications in consumer electronic applications (like AI-powered photo retrieval in smartphone, efficient media asset and creative template retrieval in smart camera, streaming recommendation systems for smart TVs). Nevertheless, text-query-based image retrieval encounters unique challenges within consumer electronics environments. First, in consumer electronics applications, text-to-image retrieving queries are typically concise, frequently leading to ambiguity in intent. Second, numerous images lack textual descriptions and metadata tags, while others contain inaccuracies or unreliable annotations—particularly in consumer-generated content where such semantic gaps critically undermine retrieval accuracy. Third, identical text queries elicit divergent expected results from distinct consumers across varying contexts. To address those challenges and improve image retrieval accuracy in consumer electronic applications, we propose $\textsf {AugMMRev}$ . $\textsf {AugMMRev}$ is primarily designed as a re-ranking model within a multimodal retrieval pipeline to evaluate the similarity between the query and a given candidate image. To achieve stronger re-ranking performance, we used a variety of processing methods. $\textsf {AugMMRev}$ employs text augmentation to enrich abbreviated queries, utilizes image augmentation to enhance visually sparse content, and integrates user profiling to deliver personalized retrieval results. Comprehensive experiments are conducted to verify the effectiveness of $\textsf {AugMMRev}$ . $\textsf {AugMMRev}$ achieves 90.7% and 88.3% retrieval accuracy over real-business dataset and public dataset, which surpass the compared methods.
Förderung
- National Key Research and Development Project of China
- Pioneer and Leading Goose Research and Development Program of Zhejiang
- Natural Science Foundation of Zhejiang University of Science and Technology
- Graduate Course Development Project of Zhejiang University of Science and Technology
- Ideological and Political Education Teaching Research Project of Zhejiang University of Science and Technology
- Yangtze River Delta Science and Technology Innovation Community Joint Research Project
Zitieren
APA
Li, Y., He, J., Chen, X., Wang, Z., Chen, Z., Niu, Q., Chen, Z., Xu, W., & Yin, Y. (2026). AugMMRev: An LLM-Augmented Multimodal Ranking Model for Personalized Image Material Retrieval. IEEE Transactions on Consumer Electronics, 72(1), 2347-2359. https://doi.org/10.1109/tce.2026.3652186
Vancouver
Li Y, He J, Chen X, Wang Z, Chen Z, Niu Q, et al. AugMMRev: An LLM-Augmented Multimodal Ranking Model for Personalized Image Material Retrieval. IEEE Transactions on Consumer Electronics. 2026;72(1):2347-2359. doi:10.1109/tce.2026.3652186
BibTeX
@article{Li2026,
author = {Li, Yu and He, Jiaxuan and Chen, Xiaoxiao and Wang, Zhonglin and Chen, Zhengyu and Niu, Qi and Chen, Zulong and Xu, Wenjian and Yin, Yuyu},
title = {AugMMRev: An LLM-Augmented Multimodal Ranking Model for Personalized Image Material Retrieval},
journal = {IEEE Transactions on Consumer Electronics},
year = {2026},
volume = {72},
number = {1},
pages = {2347--2359},
doi = {10.1109/tce.2026.3652186}
}
EUVIMED