EUVIMEDEUVIMED entdecken
Crossref

A Federated Online Restless Bandit Framework for Cooperative Resource Allocation

Jingwen Tong, Xinran Li, Liqun Fu, Jun Zhang, Khaled B. Letaief

IEEE Transactions on Mobile Computing · 2024-12 · 23 (12) · 15274-15288

DOI: 10.1109/tmc.2024.3453250 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2024-12. Verfasst von Jingwen Tong und 4 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 37 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Universidade de Macau, Hong Kong Research Grants Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Restless multi-armed bandits (RMABs) have been widely utilized to address resource allocation problems with Markov reward processes (MRPs). Existing works often assume that the dynamics of MRPs are known prior, which makes the RMAB problem solvable from an optimization perspective. Nevertheless, an efficient learning-based solution for RMABs with unknown system dynamics remains an open problem. In this paper, we fill this gap by investigating a cooperative resource allocation problem with unknown system dynamics of MRPs. This problem can be modeled as a multi-agent online RMAB problem, where multiple agents collaboratively learn the system dynamics while maximizing their accumulated rewards. We devise a federated online RMAB framework to mitigate the communication overhead and data privacy issue by adopting the federated learning paradigm. Based on this framework, we put forth a Federated Thompson Sampling-enabled Whittle Index (FedTSWI) algorithm to solve this multi-agent online RMAB problem. The FedTSWI algorithm enjoys a high communication and computation efficiency, and a privacy guarantee. Moreover, we derive a regret upper bound for the FedTSWI algorithm. Finally, we demonstrate the effectiveness of the proposed algorithm on the case of online multi-user multi-channel access. Numerical results show that the proposed algorithm achieves a fast convergence rate of $\mathcal {O}(\sqrt{T\log (T)})$ O ( T log ( T ) ) and better performance compared with baselines. More importantly, its sample complexity reduces sublinearly with the number of agents.

Förderung

  • National Natural Science Foundation of China
  • Universidade de Macau
  • Hong Kong Research Grants Council
  • NSFC/RGC Collaborative Research Scheme

Zitieren

APA

Tong, J., Li, X., Fu, L., Zhang, J., & Letaief, K. B. (2024). A Federated Online Restless Bandit Framework for Cooperative Resource Allocation. IEEE Transactions on Mobile Computing, 23(12), 15274-15288. https://doi.org/10.1109/tmc.2024.3453250

Vancouver

Tong J, Li X, Fu L, Zhang J, Letaief KB. A Federated Online Restless Bandit Framework for Cooperative Resource Allocation. IEEE Transactions on Mobile Computing. 2024;23(12):15274-15288. doi:10.1109/tmc.2024.3453250

BibTeX
@article{Tong2024,
  author = {Tong, Jingwen and Li, Xinran and Fu, Liqun and Zhang, Jun and Letaief, Khaled B.},
  title = {A Federated Online Restless Bandit Framework for Cooperative Resource Allocation},
  journal = {IEEE Transactions on Mobile Computing},
  year = {2024},
  volume = {23},
  number = {12},
  pages = {15274--15288},
  doi = {10.1109/tmc.2024.3453250}
}
Kostenlos starten →