MPI collective communication operations are used extensively in parallel applications. As
such, researchers have been investigating how to improve their performance and scalability …

As model sizes and training datasets continue to increase, large-scale model training
frameworks reduce memory consumption by various sharding techniques. However, the …

Abstract Message Passing Interface (MPI) is the de facto standard for parallel programming,
and collective operations in MPI are widely utilized by numerous scientific applications. The …

The Message Passing Interface (MPI) is a programming model for developing high-
performance applications on large-scale machines. A key component of MPI is its collective …

The paper presents an evaluation of all-reduce collective MPI algorithms for an environment
based on a geographically-distributed compute cluster. The testbed was split into two sites …

Imbalanced process arrival patterns (PAPs) are ubiquitous in many parallel and distributed
systems, especially in HPC ones. The collective operations, eg in MPI, are designed for …

Two novel algorithms for the all-gather operation resilient to imbalanced process arrival
patterns (PATs) are presented. The first one, Background Disseminated Ring (BDR), is …

Sequential Monte Carlo methods are a useful tool to tackle non-linear problems in a
Bayesian setting. A target posterior distribution is approximated by moving a set of weighted …

Abstract High-Performance Computing (HPC) is the key to tackle computationally intensive
problems such as Deep Learning (DL) and scientific applications. Message Passing …

The Clairvoyant algorithm proposed in “A novel MPI reduction algorithm resilient to
imbalances in process arrival times” was analyzed, commented and improved. The …