With the widespread use of distributed machine learning (DML), many IT companies have
established networks dedicated to DML. Different communication architectures of DML have …

Researchers have repurposed programmable network devices to place small amounts of
application computation in the network, sometimes yielding orders-of-magnitude …

Powerful computer clusters are used nowadays to train complex deep neural networks
(DNN) on large datasets. Distributed training workloads increasingly become …

In this talk, we introduce Merlin HugeCTR. Merlin HugeCTR is an open source, GPU-
accelerated integration framework for click-through rate estimation. It optimizes both training …

Distributed deep learning training usually adopts All-Reduce as the synchronization
mechanism for data parallel algorithms due to its high performance in homogeneous …

Increasingly rich data sets and complicated models make distributed machine learning more
and more important. However, the cost of extensive and frequent parameter …

The slowdown of single-chip performance scaling combined with the growing demands of
computing ever larger problems efficiently has led to a renewed interest in distributed …

As the scale of distributed training increases, it brings huge communication overhead in
clusters. Some works try to reduce the communication cost through gradient compression or …

It is important to scale out deep neural network (DNN) training for reducing model training
time. The high communication overhead is one of the major performance bottlenecks for …

Gradient compression is a promising approach to alleviating the communication bottleneck
in data parallel deep neural network (DNN) training by significantly reducing the data …