Scalable deep learning on distributed infrastructures: Challenges, techniques, and tools
R Mayer, HA Jacobsen - ACM Computing Surveys (CSUR), 2020 - dl.acm.org
Deep Learning (DL) has had an immense success in the recent past, leading to state-of-the-
art results in various domains, such as image recognition and natural language processing …
art results in various domains, such as image recognition and natural language processing …
A survey of techniques for optimizing deep learning on GPUs
The rise of deep-learning (DL) has been fuelled by the improvements in accelerators. Due to
its unique features, the GPU continues to remain the most widely used accelerator for DL …
its unique features, the GPU continues to remain the most widely used accelerator for DL …
Liquid: Intelligent resource estimation and network-efficient scheduling for deep learning jobs on distributed GPU clusters
Deep learning (DL) is becoming increasingly popular in many domains, including computer
vision, speech recognition, self-driving automobiles, etc. GPU can train DL models efficiently …
vision, speech recognition, self-driving automobiles, etc. GPU can train DL models efficiently …
A review on community detection in large complex networks from conventional to deep learning methods: A call for the use of parallel meta-heuristic algorithms
Complex networks (CNs) have gained much attention in recent years due to their
importance and popularity. The rapid growth in the size of CNs leads to more difficulties in …
importance and popularity. The rapid growth in the size of CNs leads to more difficulties in …
Optimizing distributed training deployment in heterogeneous GPU clusters
This paper proposes HeteroG, an automatic module to accelerate deep neural network
training in heterogeneous GPU clusters. To train a deep learning model with large amounts …
training in heterogeneous GPU clusters. To train a deep learning model with large amounts …
Fast training of deep learning models over multiple gpus
This paper proposes FastT, a transparent module to work with the TensorFlow framework for
automatically identifying a satisfying deployment and execution order of operations in DNN …
automatically identifying a satisfying deployment and execution order of operations in DNN …
Marble: A multi-gpu aware job scheduler for deep learning on hpc systems
Deep learning (DL) has become a key tool for solving complex scientific problems. However,
managing the multi-dimensional large-scale data associated with DL, especially atop extant …
managing the multi-dimensional large-scale data associated with DL, especially atop extant …
Garfield: System support for byzantine machine learning (regular paper)
We present GARFIELD, a library to transparently make machine learning (ML) applications,
initially built with popular (but fragile) frameworks, eg, TensorFlow and PyTorch, Byzantine …
initially built with popular (but fragile) frameworks, eg, TensorFlow and PyTorch, Byzantine …
PSNet: Reconfigurable network topology design for accelerating parameter server architecture based distributed machine learning
Abstract The bottleneck of Distributed Machine Learning (DML) has shifted from computation
to communication. Lots of works have focused on speeding up communication phase from …
to communication. Lots of works have focused on speeding up communication phase from …
Online job scheduling for distributed machine learning in optical circuit switch networks
Networking has become a well-known performance bottleneck for distributed machine
learning (DML). Although lots of works have focused on accelerating the communication …
learning (DML). Although lots of works have focused on accelerating the communication …