Training large deep learning models at scale is very challenging. This paper proposes
Chimera, a novel pipeline parallelism scheme which combines bidirectional pipelines for …

The scale of applications and computing systems is tremendously increasing and needs to
increase even more to realize exascale systems. As the number of nodes keeps growing …

Practical applications of quantum computers require millions of physical qubits and it will be
challenging for individual quantum processors to reach such qubit numbers. It is therefore …

In the generalized minimal residual method (GMRES), the global all-to-all communication
required in each iteration for orthogonalization and normalization of the Krylov base vectors …

Message progression schemes that enable communication and computation to be
overlapped have the potential to improve the performance of parallel applications. With …

MapReduce is an emerging programming paradigm for data-parallel applications. We
discuss common strategies to implement a MapReduce runtime and propose an optimized …

System noise can negatively impact the performance of HPC systems, and the
interconnection network is one of the main factors contributing to this problem. To mitigate …

Ever-increasing core counts create the need to develop parallel algorithms that avoid
closely coupled execution across all cores. We present performance analysis of several …

A major design objective of MPI is to enable support for the construction of safe parallel
libraries that can be used and mixed freely in complex applications. In this respect, MPI has …

With the growing popularity of big-data applications, Data Center Networks (DCN)
increasingly carry larger and longer traffic flows. As a result of this increased flow granularity …