Federated Multi-Agent Deep Reinforcement Learning (F-MADRL) is gathering keen research interests, as it may offer efficient solutions towards meeting the extreme requirements of future wireless communication networks. By contrast to centralized Deep Reinforcement Learning (DRL) and Multi- Agent DRL (MADRL), F-MADRL enables edge devices to cooperate without sharing their private data, while reducing the delays and signaling costs inherent to centralized approaches. In this article, we explore the new opportunities brought by F-MADRL by conducting a holistic survey of its related recent works. Firstly, we categorize state-of-the-art F-MADRL approaches, based on some distinctive features such as aggregation frequency, signaling overhead and privacy level. To better illustrate the behavior and advantages of F-MADRL, it is numerically compared to its fully centralized and distributed independent DRL counterparts, through a Sub-6GHz/mmWave band association optimization problem for Internet of Things (IoT) wireless communications. Finally, we identify and discuss the open research directions and challenges, in order to spur further interest in this promising area.

The severe spectrum scarcity and the stringent requirements of Beyond 5G applications call for an integrated use of low frequency Sub-6 GHz and high frequency millimeter Wave bands. Focusing on future Internet of Things (IoT) Short-Packet Communications (SPC), this paper investigates the optimized usage of such diverse wireless interfaces. We propose an unifying framework devoted to SPC that jointly optimizes the user partitioning over each band, and the radio resource scheduling within each band. Leveraging Deep Reinforcement Learning (DRL) tools, the proposed method enables to better tackle the challenges imposed by dynamically varying mobile environments such as the Line-of-Sight situations of each link, and the heterogeneity of individual Quality of Service (QoS) requirements, such as rate, delay and reliability. Regarding the DRL-based user partitioning to each band, we have investigated three different types of partitioning actions to obtain a high network performance as well as a rapid convergence. Regarding the proposed sub-schedulers within each band, we designed two optimization methods, i.e., one that leverages Difference of Convex Programming (DCP) technique, and the second that accelerates convergence to a local optimum. Numerical evaluations show that the proposed methods outperform conventional approaches in terms of sum-rate and QoS outage probabilities.