Multi-Agent Deep Reinforcement Learning Based Downlink Beamforming in Heterogeneous Networks
IEEE Transactions on Wireless Communications · 2023-06 · 22 (6) · 4247-4263
DOI: 10.1109/twc.2022.3224150 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Wireless Communications, 2023-06. Verfasst von Zitian Zhang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 42 Literaturangaben. Gefördert von: Zhejiang Province Key Research and Development Program, Zhejiang Provincial Key Laboratory of New Network Standards and Application Technology, National Natural Science Foundation of China.
Abstract (Originalsprache)
We consider a heterogeneous network (HetNet), where multiple access points (APs) of potentially different transmission capacities serve users simultaneously via beamforming in the same spectrum band. We propose a beamforming framework that exploits multi-agent deep reinforcement learning (DRL) for the HetNet to maximize the system downlink sum-rate. In our framework, each AP acts as an agent, which is equipped with an online policy deep neural network (DNN) and an online Q-function DNN. The former generates an AP’s beamforming vector based only on local observations in a time slot, while the latter evaluates the appropriateness of this beamforming vector. We present a distributed-updating-centralized-rewarding scheme to train the policy DNNs and Q-function DNNs of all the APs in an online trial-and-error way. Under this scheme, all the APs take the system downlink sum-rate in a recent time slot (informed by a central controller) as their identical one-step reward. Trained by the experience items with centralized rewards in every time slot, the weight vectors of each AP’s local DNNs will be updated in the direction to the global optimum. Simulation results demonstrate that the proposed framework converges fast and outperforms the benchmark beamforming methods in terms of the system downlink sum-rate performance.
Förderung
- Zhejiang Province Key Research and Development Program
- Zhejiang Provincial Key Laboratory of New Network Standards and Application Technology
- National Natural Science Foundation of China
Zitieren
APA
Zhang, Z., Hou, J., Chu, X., Zhou, H., Wei, G., & Zhang, J. (2023). Multi-Agent Deep Reinforcement Learning Based Downlink Beamforming in Heterogeneous Networks. IEEE Transactions on Wireless Communications, 22(6), 4247-4263. https://doi.org/10.1109/twc.2022.3224150
Vancouver
Zhang Z, Hou J, Chu X, Zhou H, Wei G, Zhang J. Multi-Agent Deep Reinforcement Learning Based Downlink Beamforming in Heterogeneous Networks. IEEE Transactions on Wireless Communications. 2023;22(6):4247-4263. doi:10.1109/twc.2022.3224150
BibTeX
@article{Zhang2023,
author = {Zhang, Zitian and Hou, Jinbo and Chu, Xiaoli and Zhou, Haibo and Wei, Guiyi and Zhang, Jie},
title = {Multi-Agent Deep Reinforcement Learning Based Downlink Beamforming in Heterogeneous Networks},
journal = {IEEE Transactions on Wireless Communications},
year = {2023},
volume = {22},
number = {6},
pages = {4247--4263},
doi = {10.1109/twc.2022.3224150}
}
EUVIMED