We propose an efficient protocol for decentralized training of deep neural networks from
distributed data sources. The proposed protocol allows to handle different phases of model …

Deep neural networks (DNNs) have gained tremendous attractions as compelling solutions
for applications such as image classification, object detection, speech recognition, and so …

The parameter server architecture is prevalently used for distributed deep learning. Each
worker machine in a such system trains the complete model, which leads to a large amount …

Effective parallelization strategies are crucial for the performance of distributed deep neural
network (DNN) training. Recently, several methods have been proposed to search …

Recent trends in high-performance computing and deep learning have led to the
proliferation of studies on large-scale deep neural network training. However, the frequent …

The rapid spread of the Internet of Things (IoT), with billions of connected devices, has
generated huge amounts of data and asks for decentralized solutions for machine learning …

For the pursuit of ubiquitous computing, distributed computing systems containing the cloud,
edge devices, and Internet-of-Things devices are highly demanded. However, existing …

Big deep neural network (DNN) models trained on large amounts of data have recently
achieved the best accuracy on hard tasks, such as image and speech recognition. Training …

To afford the huge computational cost, large-scale deep neural networks (DNN) are usually
trained on the distributed system, especially the widely-used parameter server architecture …

The demand for artificial intelligence has grown significantly over the past decade, and this
growth has been fueled by advances in machine learning techniques and the ability to …