Abstract 5 G is the fifth generation of cellular networks. It enables billions of connected devices to gather and share information in real time; a key facilitator in Industrial Internet of Things (IoT) applications. It has more capabilities in terms of bandwidth, latency/delay, processing powers and flexibility to utilize either edge or cloud resources. Furthermore, 6G is expected to be equipped with the new capability to converge ubiquitous communication, computation, sensing and controlling for a variety of sectors, which heightens the complexity in a more heterogeneous environment This increased complexity, combined with energy efficiency and Service Level Agreement (SLA) requirements makes application of Machine Learning (ML) and distributed ML necessary. A decentralized approach stemming from distributed learning is a very attractive option compared with a centralized architecture for model learning and inference. Distributed ML exploits recent Artificial Intelligence (AI) technology advancements to allow collaborated ML, whilst safeguarding private data, minimizing both communication and computation overhead along with addressing ultra-low latency requirements. In this paper, we review a number of distributed ML architectures and designs, that focus on optimizing communication, computation and resource distribution. Privacy, information security and compute frameworks, are also analyzed and compared with respect to different distributed ML approaches. We summarize the major contributions and trends in this area and highlight the potential of distributed ML to help researchers and practitioners make informed decisions on selecting the right ML approach for 5 G and Beyond related AI applications. To enable distributed ML for 5 G and Beyond, communication, security, and computing platform often counter balance each other, thus, consideration and optimization of these aspects at an overall system level is crucial to realize the full potential of AI for 5G and Beyond. These different aspects do not only pertain to 5 G, but will also enable careful design of distributed machine learning architectures to circumvent the same hurdles that will inevitably burden 5 G and Beyond network generations. This is the first survey paper that brings together all these aspects for distributed ML.