Training graph neural networks (GNNs) is extremely time consuming because sparse graph-
based operations are hard to be accelerated by community hardware. Prior art successfully …

With the rapid growth in the volume of data sets, models, and devices in the domain of deep
learning, there is increasing attention on large-scale distributed deep learning. In contrast to …

Data-parallel distributed training (DDT) is the de facto way to accelerate deep learning on
multiple GPUs. In DDT, communication for gradient synchronization is the major efficiency …

Large deep learning models have recently garnered substantial attention from both
academia and industry. Nonetheless, frequent failures are observed during large model …

Activation compressed training provides a solution towards reducing the memory cost of
training deep neural networks (DNNs). However, state-of-the-art work combines a search of …

Distributed training is the de facto standard to scale up the training of Deep Neural Networks
(DNNs) with multiple GPUs. The performance bottleneck of distributed training lies in …

Data-parallel distributed training (DDT) is the de facto way to accelerate deep learning on
multiple GPUs. In DDT, communication for gradient synchronization is the major efficiency …

Graph neural networks (GNNs) have gained considerable success in graph-based learning
tasks, yet training GNNs on large graphs is still inefficient. The root cause is the graph-based …

Deep neural networks (DNNs) have achieved unparalleled performance in numerous fields,
including computer vision, natural language processing, and recommendation systems …