As deep learning models and input data continue to scale at an unprecedented rate, it has
become inevitable to move towards distributed training platforms to fit the models and …

Reinforcement Learning (RL) is a powerful technology to solve decisionmaking problems
such as robotics control. Modern RL algorithms, ie, Deep Q-Learning, are based on costly …

Deep Learning (DL) training platforms are built by interconnecting multiple DL accelerators
(eg, GPU/TPU) via fast, customized interconnects with 100s of gigabytes (GBs) of bandwidth …

Distributed training is a solution to reduce DNN training time by splitting the task across
multiple NPUs (eg, GPU/TPU). However, distributed training adds communication overhead …

Transfer learning in natural language processing (NLP) uses increasingly large models that
tackle challenging problems. Consequently, these applications are driving the requirements …

Robotic systems, such as autonomous unmanned aerial vehicles (UAVs) and self-driving
cars, have been widely deployed in many scenarios and have the potential to revolutionize …

As Deep Learning (DL) models grow larger and more complex, training jobs are
increasingly distributed across multiple Computing Units (CU) such as GPUs and TPUs …

Large-scale distributed deep learning training has enabled developments of more complex
deep neural network models to learn from larger datasets for sophisticated tasks. In …

In this paper, we consider the parallel implementation of an already-trained deep model on
multiple processing nodes (aka workers). Specifically, we investigate as to how a deep …

Training is an important aspect of deep learning to enable network models to be deployed.
To scale training, multiple GPUs are commonly used with data parallelism to exploit the …