RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación
Artículo científico (JournalArticle)
Energy-Efficient Sleep Mode Optimization in 5G mmWave Networks via Multi-Agent Deep Reinforcement Learning
[EN] Dynamic sleep mode optimization (SMO) in millimeter-wave (mmWave) networks is essential for maximizing energy efficiency (EE) under stringent quality-of-service (QoS) constraints. However, existing optimization and reinforcement learning (RL)-based approaches rely on aggregated, static base station (BS) traffic models that fail to capture non-stationary traffic dynamics and suffer from prohibitively large state-action spaces, limiting their real-world deployment. To address these challenges, this paper proposes a Multi-Agent Deep Reinforcement Learning (MARL) framework employing a Double Deep Q-Network (DDQN), referred to as MARL-DDQN, for adaptive SMO in a 3D urban environment using a time-varying and community-based user equipment (UE) mobility model. Unlike conventional single-agent RL, the proposed MARL-DDQN enables scalable, distributed decision-making with minimal signaling overhead. A realistic BS power consumption model and beamforming are integrated to accurately quantify EE, while QoS is uniquely defined in terms of throughput. The proposed method adaptively learns SMO policies to maximize EE while mitigating inter-cell interference and ensuring throughput fairness. Extensive simulations demonstrate that MARL-DDQN consistently outperforms state-of-the-art SM strategies, including the All On, iterative QoS-aware load-based (IT-QoS-LB), MARL-DDPG, and MARL-PPO, achieving up to 0.60 Mbit/Joule EE, 8.5 Mbps 10th-percentile throughput, and satisfying QoS constraints 95% of the time under dynamic network scenarios., This work was supported in part by NSF under Award CNS-2332835, in part by the Generalitat Valenciana through the Convocatoria d'Incorporacio i Reincorporacio de Personal Investigador d'Excellencia (CIDEGENT) PlaGenT under Grant CIDEXG/2022/17, and in part by the Project iTENTE funded by Ministry of Science, Innovation and Universities (MCIN)/State Research Agency (AEI)/10.13039/501100011033 and European Union "NextGenerationEU"/PRTR" under Grant CNS2023-144333.