Distributed Generative Adversarial Networks (D-GANs) are a recent addition to generative model literature, but they come with significant limitations and caveats. This paper categorizes and analyzes the existing D-GAN models and introduces a novel four-part categorization framework based on their mathematical properties. We provided generalized mathematical formulations for each D-GAN category, addressing convergence issues and potential improvements, which are essential for developing more robust and reliable D-GANs. Additionally, we conducted detailed mathematical analyses of existing D-GANs to identify their shortcomings in terms of expanse, convergence and output quality. This rigorous analysis is pivotal for advancing the field, as it highlights critical areas needing improvement and sets the stage for future innovations that will enhance the performance and reliability of distributed generative models. This work establishes a foundation for future research, and standardizes the evaluation of D-GAN models. Additionally, it guides the research community towards addressing unresolved issues and exploring new avenues, thereby enhancing the overall effectiveness and applicability of the distributed generative models in AI/ML.