EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

LLMs Are Also Effective Embedding Models: An In-depth Overview

Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Kai Hua, Wenpeng Hu, Zhengwei Tao, Shuai Ma

ACM Transactions on Information Systems · 30.09.2026 · 44 (7) · 1-41

DOI: 10.1145/3831684 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Information Systems, online erschienen am 23.09.2026. Verfasst von Chongyang Tao und 8 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 216 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, State Key Laboratory of Complex & Critical Software Environment, CIE–Tianyi Cloud Research Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have revolutionized natural language processing by achieving state-of-the-art performance across various tasks. Recently, their effectiveness as embedding models has gained attention, marking a paradigm shift from traditional encoder-only models like ELMo and BERT to decoder-only, large-scale LLMs such as GPT, LLaMA, and Mistral. This survey provides an in-depth overview of this transition, beginning with foundational techniques before the LLM era, followed by LLM-based embedding models through two main strategies to derive embeddings from LLMs. (1) Direct prompting: We mainly discuss the prompt designs and the underlying rationale for deriving competitive embeddings. (2) Data-centric tuning: We cover extensive aspects that affect tuning an embedding model, including model architecture, training objectives, data construction, and so on. Upon the above, we also cover advanced methods for producing embeddings from longer texts, multilingual, code, cross-modal data, as well as reasoning-aware and other domain-specific scenarios. Furthermore, we discuss factors affecting choices of embedding models, such as performance comparisons, dense vs. sparse embeddings, pooling strategies, and scaling laws. Lastly, the survey highlights the limitations and challenges in adapting LLMs for embeddings, including cross-task embedding quality, tradeoffs between efficiency and accuracy, low-resource, long-context, data bias, robustness, and so on. This survey serves as a valuable resource for researchers and practitioners by synthesizing current advancements, highlighting key challenges, and offering a comprehensive framework for future work aimed at enhancing the effectiveness and efficiency of LLMs as embedding models.

Förderung

  • National Natural Science Foundation of China
  • State Key Laboratory of Complex & Critical Software Environment
  • CIE–Tianyi Cloud Research Program

Zitieren

APA

Tao, C., Shen, T., Gao, S., Zhang, J., Li, Z., Hua, K., Hu, W., Tao, Z., & Ma, S. (2026). LLMs Are Also Effective Embedding Models: An In-depth Overview. ACM Transactions on Information Systems, 44(7), 1-41. https://doi.org/10.1145/3831684

Vancouver

Tao C, Shen T, Gao S, Zhang J, Li Z, Hua K, et al. LLMs Are Also Effective Embedding Models: An In-depth Overview. ACM Transactions on Information Systems. 2026;44(7):1-41. doi:10.1145/3831684

BibTeX
@article{Tao2026,
  author = {Tao, Chongyang and Shen, Tao and Gao, Shen and Zhang, Junshuo and Li, Zhen and Hua, Kai and Hu, Wenpeng and Tao, Zhengwei and Ma, Shuai},
  title = {LLMs Are Also Effective Embedding Models: An In-depth Overview},
  journal = {ACM Transactions on Information Systems},
  year = {2026},
  volume = {44},
  number = {7},
  pages = {1--41},
  doi = {10.1145/3831684}
}
Kostenlos starten →