EUVIMEDEUVIMED entdecken
Crossref

Enhancing LLM QoS Through Cloud-Edge Collaboration: A Diffusion-Based Multi-Agent Reinforcement Learning Approach

Zhi Yao, Zhiqing Tang, Wenmian Yang, Weijia Jia

IEEE Transactions on Services Computing · 2025-05 · 18 (3) · 1412-1427

DOI: 10.1109/tsc.2025.3562362 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Services Computing, 2025-05. Verfasst von Zhi Yao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Guangdong Key Lab of AI and Multi-modal Data Processing, Beijing Normal-Hong Kong Baptist University, Zhuhai.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) are widely used across various domains, but deploying them in cloud data centers often leads to significant response delays and high costs, undermining Quality of Service (QoS) at the network edge. Although caching LLM request results at the edge using vector databases can greatly reduce response times and costs for similar requests, this approach has been overlooked in prior research. To address this, we propose a novel Vector database-assisted cloud-Edge collaborative LLM QoS Optimization (VELO) framework that caches LLM request results at the edge using vector databases, thereby reducing response times for subsequent similar requests. Unlike methods that modify LLMs directly, VELO leaves the LLM's internal structure intact and is applicable to various LLMs. Building on VELO, we formulate the QoS optimization problem as a Markov Decision Process (MDP) and design an algorithm based on Multi-Agent Reinforcement Learning (MARL). Our algorithm employs a diffusion-based policy network to extract the LLM request features, determining whether to request the LLM in the cloud or retrieve results from the edge's vector database. Implemented in a real edge system, our experimental results demonstrate that VELO significantly enhances user satisfaction by simultaneously reducing delays and resource consumption for edge users of LLMs. Our DLRS algorithm improves performance by 15.0% on average for similar requests and by 14.6% for new requests compared to the baselines.

Förderung

  • National Natural Science Foundation of China
  • Guangdong Key Lab of AI and Multi-modal Data Processing
  • Beijing Normal-Hong Kong Baptist University, Zhuhai
  • Department of Education of Guangdong Province
  • Institute of Artificial Intelligence and Future Networks and Engineering Center of AI and Future Education
  • Guangdong Provincial Department of Science and Technology, China
  • Zhuhai Science-Tech Innovation Bureau
  • Interdisciplinary Intelligence Super Computer Center of Beijing Normal University at Zhuhai

Zitieren

APA

Yao, Z., Tang, Z., Yang, W., & Jia, W. (2025). Enhancing LLM QoS Through Cloud-Edge Collaboration: A Diffusion-Based Multi-Agent Reinforcement Learning Approach. IEEE Transactions on Services Computing, 18(3), 1412-1427. https://doi.org/10.1109/tsc.2025.3562362

Vancouver

Yao Z, Tang Z, Yang W, Jia W. Enhancing LLM QoS Through Cloud-Edge Collaboration: A Diffusion-Based Multi-Agent Reinforcement Learning Approach. IEEE Transactions on Services Computing. 2025;18(3):1412-1427. doi:10.1109/tsc.2025.3562362

BibTeX
@article{Yao2025,
  author = {Yao, Zhi and Tang, Zhiqing and Yang, Wenmian and Jia, Weijia},
  title = {Enhancing LLM QoS Through Cloud-Edge Collaboration: A Diffusion-Based Multi-Agent Reinforcement Learning Approach},
  journal = {IEEE Transactions on Services Computing},
  year = {2025},
  volume = {18},
  number = {3},
  pages = {1412--1427},
  doi = {10.1109/tsc.2025.3562362}
}
Kostenlos starten →