The Art of Multiprocessor Programming, Second Edition, provides users with an authoritative
guide to multicore programming. This updated edition introduces higher level software …

Busy-wait techniques are heavily used for mutual exclusion and barrier synchronization in
shared-memory parallel programs. Unfortunately, typical implementations of busy-waiting …

We describe our work on improving the performance of collective communication operations
in MPICH for clusters connected by switched networks. For each collective operation, we …

The memory coherence problem in designing and implementing a shared virtual memory on
loosely coupled multiprocessors is studied in depth. Two classes of algorithms, centralized …

The goal of software synthesis is to generate programs automatically from high-level
specifications. However, efficient implementations for challenging programs require a …

Near-Data-Processing (NDP) architectures present a promising way to alleviate data
movement costs and can provide significant performance and energy benefits to parallel …

Serverless computing greatly simplifies the use of cloud resources. In particular, Function-as-
a-Service (FaaS) platforms enable programmers to develop applications as individual …

Munin is a distributed shared memory (DSM) system that allows shared memory parallel
programs to be executed efficiently on distributed memory multiprocessors. Munin is unique …

Applying machine learning techniques to the quickly growing data in science and industry
requires highly-scalable algorithms. Large datasets are most commonly processed" data …

Communication overhead is one of the major obstacles to train large deep learning models
at scale. Gradient sparsification is a promising technique to reduce the communication …