The way that information propagates in neural networks is of great importance. In this paper,
we propose Path Aggregation Network (PANet) aiming at boosting information flow in …

We present Atos, a dynamic scheduling framework for multi-node-GPU systems that
supports PGAS-style lightweight one-sided memory operations within and between nodes …

Reaching Exascale will require leveraging massive parallelism while potentially leveraging
asynchronous communication to help achieve scalability at such large levels of concurrency …

We introduce Portus, an efficient checkpointing system for DNN models. The core of Portus
is a three-level index structure and a direct RDMA datapath that enables fast check-points …

In this chapter, metrics for measuring and improving data center efficiencies are explored.
Metrics at varying levels from the infrastructure components to the entire data center are …

One-sided communication (also known as remote memory access, or RMA) in the Message
Passing Interface (MPI) is a communication interface that has been introduced in MPI-2 …

We describe light-weight protocols for selective packet ordering in out-of-order networks that
carry memory traffic. The protocols are designed for heterogeneous high-performance …

MPI One-sided communications have the potential to increase applications performance by
reducing the noise on remote processors. They consist in Remote Memory Accesses …

The Complex neuronal network models require support from simulation environment for
efficient network simulations. To compute the models increasing complexity necessitated the …

We present methods for the effective application level reordering of non-blocking RDMA
operations. We supplement out-of-order hardware delivery mechanisms with heuristics to …