Dive into deep learning
This open-source book represents our attempt to make deep learning approachable,
teaching readers the concepts, the context, and the code. The entire book is drafted in …
teaching readers the concepts, the context, and the code. The entire book is drafted in …
Blink: Fast and generic collectives for distributed ml
G Wang, S Venkataraman… - Proceedings of …, 2020 - proceedings.mlsys.org
Abstract Model parameter synchronization across GPUs introduces high overheads for data-
parallel training at scale. Existing parameter synchronization protocols cannot effectively …
parallel training at scale. Existing parameter synchronization protocols cannot effectively …
Parameter hub: a rack-scale parameter server for distributed deep neural network training
Distributed deep neural network (DDNN) training constitutes an increasingly important
workload that frequently runs in the cloud. Larger DNN models and faster compute engines …
workload that frequently runs in the cloud. Larger DNN models and faster compute engines …
Communication algorithm-architecture co-design for distributed deep learning
Large-scale distributed deep learning training has enabled developments of more complex
deep neural network models to learn from larger datasets for sophisticated tasks. In …
deep neural network models to learn from larger datasets for sophisticated tasks. In …