Elevated temperatures limit the peak performance of systems because of frequent
interventions by thermal throttling. Non-uniform thermal states across system nodes also …

Scientists are increasingly exploring and utilizing the massive parallelism of general-
purpose accelerators such as GPUs for scientific breakthroughs. As a result, datacenters …

Machine learning (ML) models are increasingly trained in clusters with non-dedicated
workers possessing heterogeneous resources. In such scenarios, model training efficiency …

Machine learning (ML) models are increasingly trained with distributed workers possessing
heterogeneous resources. In such scenarios, model training efficiency may be negatively …

Large-scale computing systems are increasingly using accelerators such as GPUs to enable
peta-and exa-scale levels of compute to meet the needs of Machine Learning (ML) and …

Motivated by a wide range of applications from parallel computing to distributed learning, we
study distributed online load balancing among multiple workers. We aim to minimize the …

Slower workers, known as stragglers, can signifi-cantly prolong training time in Machine
Learning (ML) clusters. We present SMS, a proactive straggler mitigation system with four …

With the growing power consumption of heterogeneous clusters, energy‐aware resource
management has been a hot research topic recently. Among various candidate techniques …

Elastic cloud computing provides new opportunities for accelerating the process of scientific
discovery. However, unlike high-performance computing (HPC) systems that are built and …

Performance variation is a significant problem for large scale HPC systems and will increase
on future exascale systems. In this work, we show that performance variation impacts the …