The evolution of distributed systems for graph neural networks and their origin in graph processing and deep learning: A survey

J Vatter, R Mayer, HA Jacobsen - ACM Computing Surveys, 2023 - dl.acm.org
Graph neural networks (GNNs) are an emerging research field. This specialized deep
neural network architecture is capable of processing graph structured data and bridges the …

Demystifying parallel and distributed deep learning: An in-depth concurrency analysis

T Ben-Nun, T Hoefler - ACM Computing Surveys (CSUR), 2019 - dl.acm.org
Deep Neural Networks (DNNs) are becoming an important tool in modern computing
applications. Accelerating their training is a major challenge and techniques range from …

The rotten apples of Brazil's agribusiness

R Rajão, B Soares-Filho, F Nunes, J Börner… - Science, 2020 - science.org
In the increasingly polarized international political arena, it has become difficult to find
common ground to solve Brazil's ongoing environmental crisis, which has global as well as …

Learning scheduling algorithms for data processing clusters

H Mao, M Schwarzkopf, SB Venkatakrishnan… - Proceedings of the …, 2019 - dl.acm.org
Efficiently scheduling data processing jobs on distributed compute clusters requires complex
algorithms. Current systems use simple, generalized heuristics and ignore workload …

Ray: A distributed framework for emerging {AI} applications

P Moritz, R Nishihara, S Wang, A Tumanov… - … USENIX symposium on …, 2018 - usenix.org
The next generation of AI applications will continuously interact with the environment and
learn from these interactions. These applications impose new and demanding systems …

Augment your batch: Improving generalization through instance repetition

E Hoffer, T Ben-Nun, I Hubara… - Proceedings of the …, 2020 - openaccess.thecvf.com
Large-batch SGD is important for scaling training of deep neural networks. However, without
fine-tuning hyperparameter schedules, the generalization of the model may be hampered …

Shenango: Achieving high {CPU} efficiency for latency-sensitive datacenter workloads

A Ousterhout, J Fried, J Behrens, A Belay… - … USENIX Symposium on …, 2019 - usenix.org
Datacenter applications demand microsecond-scale tail latencies and high request rates
from operating systems, and most applications handle loads that have high variance over …

Neural acceleration for general-purpose approximate programs

H Esmaeilzadeh, A Sampson, L Ceze… - 2012 45th annual …, 2012 - ieeexplore.ieee.org
This paper describes a learning-based approach to the acceleration of approximate
programs. We describe the Parrot transformation, a program transformation that selects and …

[图书][B] Introduction to algorithms

TH Cormen, CE Leiserson, RL Rivest, C Stein - 2022 - books.google.com
A comprehensive update of the leading algorithms text, with new material on matchings in
bipartite graphs, online algorithms, machine learning, and other topics. Some books on …

A comprehensive survey on coded distributed computing: Fundamentals, challenges, and networking applications

JS Ng, WYB Lim, NC Luong, Z Xiong… - … Surveys & Tutorials, 2021 - ieeexplore.ieee.org
Distributed computing has become a common approach for large-scale computation tasks
due to benefits such as high reliability, scalability, computation speed, and cost …