Deep Learning training jobs process large amounts of training data using many GPU
devices, often running for weeks or months. When hardware or software failures happen …

Gradient checkpointing is an optimization that reduces the memory footprint by re-computing
some operations instead of saving their activations. Previous works on checkpointing have …

Fault tolerance is becoming increasingly important since the probability of permanent
hardware failures increases with machine size. A typical resilience approach to fail/stop …

With increasing scale and complexity of supercomputing and cloud computing architectures,
faults are becoming a frequent occurrence, which makes reliability a difficult challenge …

As we approach the era of exa-scale computing, fault tolerance is of growing importance.
The increasing number of cores as well as the increased complexity of modern …

Deep learning (DL) applications are increasingly being deployed on HPC systems, to
leverage the massive parallelism and computing power of those systems for DL model …

Deep learning is slowly, but steadily, hitting a memory bottleneck. While the tensor
computation in top-of-the-line GPUs increased by 32x over the last five years, the total …

In the age of big data, deep learning has emerged as a powerful tool to extract insight and
exploit its value, both in industry and scientific applications. One common pattern emerging …

We present a lightweight technique to minimize error recovery costs in approximate
computing environments. We take advantage of the key observation that if an application …

Larger deep learning models usually lead to higher model quality, however with an ever-
increasing GPU memory footprint. Although several tensor checkpointing techniques have …