Deep learning have been widely used in various fields and has worked very well as a major
role. While the gradual penetration into various fields, data quantity of each applications is …

As the popularity of deep learning in industry rapidly grows, efficient training of deep neural
networks (DNNs) becomes important. To train a DNN with a large amount of data, distributed …

With growing scale of the data volume and neural network size, we have come into the era
of distributed deep learning. High-performance training and inference on distributed …

Data parallel distributed deep learning has been used to accelerate the learning speed. The
communication is becoming a bottleneck as the computation time is accelerated. Therefore …

Deep learning with a large number of parametersrequires distributed training, where model
accuracy and runtimeare two important factors to be considered. However, there hasbeen …

Artificial intelligence has made remarkable progress in handling complex tasks, thanks to
advances in hardware acceleration and machine learning algorithms. However, to acquire …

Faster training of deep neural networks is desired to speed up the research and
development cycle in deep learning. Distributed deep learning and second-order …

Training real-world Deep Neural Networks (DNNs) can take an eon (ie, weeks or months)
without leveraging distributed systems. Even distributed training takes inordinate time, of …

Distributed training of Deep Neural Networks (DNNs) on High-Performance Computing
(HPC) systems is becoming increasingly common. HPC systems dedicated entirely or …

Demand is mounting in the industry for scalable GPU-based deep learning systems.
Unfortunately, existing training applications built atop popular deep learning frameworks …