Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models
Proceedings of the VLDB Endowment · 2024-09 · 18 (1) · 42-52
DOI: 10.14778/3696435.3696439 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 11.02.2025. Verfasst von Wenqi Jiang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 93 Literaturangaben.
Abstract (Originalsprache)
A Retrieval-Augmented Language Model (RALM) combines a large language model (LLM) with a vector database to retrieve context-specific knowledge during text generation. This strategy facilitates impressive generation quality even with smaller models, thus reducing computational demands by orders of magnitude. To serve RALMs efficiently and flexibly, we propose Chameleon , a heterogeneous accelerator system integrating both LLM and vector search accelerators in a disaggregated architecture. The heterogeneity ensures efficient serving for both inference and retrieval, while the disaggregation allows independent scaling of LLM and vector search accelerators to fulfill diverse RALM requirements. Our Chameleon prototype implements vector search accelerators on FPGAs and assigns LLM inference to GPUs, with CPUs as cluster coordinators. Evaluated on various RALMs, Chameleon exhibits up to 2.16× reduction in latency and 3.18× speedup in throughput compared to the hybrid CPU-GPU architecture. The promising results pave the way for adopting heterogeneous accelerators for not only LLM inference but also vector search in future RALM systems.
Zitieren
APA
Jiang, W., Zeller, M., Waleffe, R., Hoefler, T., & Alonso, G. (2024). Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models. Proceedings of the VLDB Endowment, 18(1), 42-52. https://doi.org/10.14778/3696435.3696439
Vancouver
Jiang W, Zeller M, Waleffe R, Hoefler T, Alonso G. Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models. Proceedings of the VLDB Endowment. 2024;18(1):42-52. doi:10.14778/3696435.3696439
BibTeX
@article{Jiang2024,
author = {Jiang, Wenqi and Zeller, Marco and Waleffe, Roger and Hoefler, Torsten and Alonso, Gustavo},
title = {Chameleon: A Heterogeneous and Disaggregated Accelerator System for Retrieval-Augmented Language Models},
journal = {Proceedings of the VLDB Endowment},
year = {2024},
volume = {18},
number = {1},
pages = {42--52},
doi = {10.14778/3696435.3696439}
}
EUVIMED