Full duplex (FD) is an auspicious wireless technology that holds the potential to double data rates through simultaneous transmission and reception. This paper proposes two innovative designs of hybrid beamforming (HYBF) for a multicell massive multiple-input-multiple-output (mMIMO) millimeter wave (mmWave) FD system. Initially, we introduce a novel centralized HYBF (C-HYBF) scheme, which employs the minorization-maximization (MM) method. However, while centralized beamforming designs offer superior performance, they suffer from high computational complexity, substantial communication overhead, and demand expensive computational resources. To surmount these challenges, we present a framework that facilitates per-link parallel and distributed HYBF (P$\&$D-HYBF) in the mmWave frequency band. This cooperative approach enables each base station (BS) to independently solve its local, low-complexity sub-problems in parallel, resulting in a substantial reduction in communication overhead and computational complexity. Simulation results demonstrate that P$\&$D-HYBF achieves comparable performance to C-HYBF, and with only a few radio-frequency (RF) chains, both designs surpass the capabilities of fully digital half duplex (HD) systems.

Full-Duplex (FD) communication can revolutionize wireless communications as it doubles spectral efficiency and offers numerous other advantages over a half-duplex (HD) system. In this paper, we present a novel and practical joint hybrid beamforming (HYBF) and combining scheme for millimeter-wave (mmWave) massive MIMO FD system for weighted sum-rate (WSR) maximization with multi-antenna HD uplink and downlink users with non-ideal hardware. Moreover, we present a novel interference and self-interference (SI) aware optimal power allocation scheme for the optimal beamforming directions. The analog processing stage is assumed to be quantized, and both the unit-modulus and unconstrained cases are considered. Moreover, compared to the traditional sum-power constraints, the proposed algorithm is designed under the joint sum-power and the practical per-antenna power constraints. To model the non-ideal hardware of a hybrid FD transceiver, we extend the traditional limited dynamic range (LDR) noise model to mmWave. Our HYBF design relies on alternating optimization based on the minorization-maximization method. We investigate the maximum achievable gain of a hybrid FD system with different levels of the LDR noise variance and with different numbers of radio-frequency (RF) chains over a HD system. Simulation results show that the mmWave massive MIMO FD systems can significantly outperform the fully digital HD systems with only a few RF chains if the LDR noise generated from the limited number of RF chains available is low. If the LDR noise variance dominates, FD communication with HYBF results to be disadvantageous than a HD system.