Energy-efficient joint resource allocation in 5G HetNet using Multi-Agent Parameterized Deep Reinforcement learning

Abstract

<p>Small cells are a promising technique to improve the capacity and throughput of future wireless networks. However, user association and <a href="https://www.sciencedirect.com/topics/computer-science/power-allocation" title="Learn more about power allocation from ScienceDirect's AI-generated Topic Pages">power allocation</a> in <a href="https://www.sciencedirect.com/topics/computer-science/heterogeneous-network" title="Learn more about heterogeneous networks from ScienceDirect's AI-generated Topic Pages">heterogeneous networks</a> is complicated by the dense deployment of small cells, resulting in non-convex and <a href="https://www.sciencedirect.com/topics/computer-science/combinatorial-problem" title="Learn more about combinatorial problems from ScienceDirect's AI-generated Topic Pages">combinatorial problems</a>. Conventionally, <a href="https://www.sciencedirect.com/topics/computer-science/machine-learning-technique" title="Learn more about machine learning techniques from ScienceDirect's AI-generated Topic Pages">machine learning techniques</a> are applied to the joint <a href="https://www.sciencedirect.com/topics/computer-science/optimization-problem" title="Learn more about optimization problem from ScienceDirect's AI-generated Topic Pages">optimization problem</a>, which has different action spaces. Gauging the continuous spaces to discrete spaces results in the loss of <a href="https://www.sciencedirect.com/topics/computer-science/granularity" title="Learn more about granularity from ScienceDirect's AI-generated Topic Pages">granularity</a> due to <a href="https://www.sciencedirect.com/topics/computer-science/discretization" title="Learn more about discretization from ScienceDirect's AI-generated Topic Pages">discretization</a> (e.g. potential power values in power allocation). Due to its hybrid action space, it is sub-optimal to solve joint user association (discrete spaces) and power allocation (continuous spaces) problems by applying traditional <a href="https://www.sciencedirect.com/topics/computer-science/machine-learning-approach" title="Learn more about machine learning approaches from ScienceDirect's AI-generated Topic Pages">machine learning approaches</a>. This work proposes a Multi-Agent Parameterized <a href="https://www.sciencedirect.com/topics/computer-science/deep-reinforcement-learning" title="Learn more about Deep Reinforcement Learning from ScienceDirect's AI-generated Topic Pages">Deep Reinforcement Learning</a> (MA-PDRL) approach to address the joint user association and power allocation problem efficiently. According to simulation results, the proposed multi-agent PDRL performs better in energy efficiency and QoS satisfaction than WMMSE, game theory, Q-learning, and DRL techniques.</p>