BACKGROUND Improvements in computing power can claim a large share of the credit for
many of the things that we take for granted in our modern lives: cellphones that are more …

Data sets in large applications are often too massive to fit completely inside the computers
internal memory. The resulting input/output communication (or I/O) between fast internal …

The high computational and memory requirements of large language model (LLM) inference
make it feasible only with multiple high-end accelerators. Motivated by the emerging …

Transformers are one of the most important machine learning workloads today. Training one
is a very compute-intensive task, often taking days or weeks, and significant attention has …

This textbook covers both direct and iterative methods for the solution of linear systems, least
squares problems, eigenproblems, and the singular value decomposition. Earlier versions …

Roughly, the cycle for developing a special-purpose system can be divided into three
phases–task definition, design, and implementation. During task definition, some system …

We provide tight upper and lower bounds, up to a constant factor, for the number of inputs
and outputs (I/OS) between internal memory and secondary storage required for five sorting …

This paper presents asymptotically optimal algorithms for rectangular matrix transpose, FFT,
and sorting on computers with multiple levels of caching. Unlike previous optimal algorithms …

The present book provides an introduction to using space-filling curves (SFC) as tools in
scientific computing. Special focus is laid on the representation of SFC and on resulting …

Blocking is a well-known optimization technique for improving the effectiveness of memory
hierarchies. Instead of operating on entire rows or columns of an array, blocked algorithms …