Domain-specific hardware is becoming a promising topic in the backdrop of improvement
slow down for general-purpose processors due to the foreseeable end of Moore's Law …

Artificial intelligence (AI) has achieved remarkable breakthroughs in a wide range of fields,
ranging from speech processing, image classification to drug discovery. This is driven by the …

We study gradient compression methods to alleviate the communication bottleneck in data-
parallel distributed optimization. Despite the significant attention received, current …

Powerful computer clusters are used nowadays to train complex deep neural networks
(DNN) on large datasets. Distributed training increasingly becomes communication bound …

Graph Convolutional Networks (GCNs) is the state-of-the-art method for learning graph-
structured data, and training large-scale GCNs requires distributed training across multiple …

Machine learning can provide deep insights into data, allowing machines to make high-
quality predictions and having been widely used in real-world applications, such as text …

Reinforcement learning (RL) has attracted much attention recently, as new and emerging AI-
based applications are demanding the capabilities to intelligently react to environment …

Large-scale distributed training of Deep Neural Networks (DNNs) on state-of-the-art
platforms are expected to be severely communication constrained. To overcome this …

Powerful computer clusters are used nowadays to train complex deep neural networks
(DNN) on large datasets. Distributed training workloads increasingly become …

Checkpoints play an important role in training long running machine learning (ML) models.
Checkpoints take a snapshot of an ML model and store it in a non-volatile memory so that …