EUVIMEDEUVIMED entdecken
Crossref

Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models

Wenqi Jiang, Marco Zeller, Roger Waleffe, Torsten Hoefler, Gustavo Alonso

Proceedings of the VLDB Endowment · 2024-09 · 18 (1) · 42-52

DOI: 10.14778/3696435.3696439 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 11.02.2025. Verfasst von Wenqi Jiang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 93 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A Retrieval-Augmented Language Model (RALM) combines a large language model (LLM) with a vector database to retrieve context-specific knowledge during text generation. This strategy facilitates impressive generation quality even with smaller models, thus reducing computational demands by orders of magnitude. To serve RALMs efficiently and flexibly, we propose Chameleon , a heterogeneous accelerator system integrating both LLM and vector search accelerators in a disaggregated architecture. The heterogeneity ensures efficient serving for both inference and retrieval, while the disaggregation allows independent scaling of LLM and vector search accelerators to fulfill diverse RALM requirements. Our Chameleon prototype implements vector search accelerators on FPGAs and assigns LLM inference to GPUs, with CPUs as cluster coordinators. Evaluated on various RALMs, Chameleon exhibits up to 2.16× reduction in latency and 3.18× speedup in throughput compared to the hybrid CPU-GPU architecture. The promising results pave the way for adopting heterogeneous accelerators for not only LLM inference but also vector search in future RALM systems.

Zitieren

APA

Jiang, W., Zeller, M., Waleffe, R., Hoefler, T., & Alonso, G. (2024). Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models. Proceedings of the VLDB Endowment, 18(1), 42-52. https://doi.org/10.14778/3696435.3696439

Vancouver

Jiang W, Zeller M, Waleffe R, Hoefler T, Alonso G. Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models. Proceedings of the VLDB Endowment. 2024;18(1):42-52. doi:10.14778/3696435.3696439

BibTeX
@article{Jiang2024,
  author = {Jiang, Wenqi and Zeller, Marco and Waleffe, Roger and Hoefler, Torsten and Alonso, Gustavo},
  title = {Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models},
  journal = {Proceedings of the VLDB Endowment},
  year = {2024},
  volume = {18},
  number = {1},
  pages = {42--52},
  doi = {10.14778/3696435.3696439}
}
Kostenlos starten →