Contemporary GPUs are widely used for throughput-oriented data-parallel workloads and
increasingly are being considered for latency-sensitive applications in datacenters …

Deep learning (DL) has demonstrated its remarkable success in a wide variety of fields. The
development of a DL model is a time-consuming and resource-intensive procedure. Hence …

Hardware accelerators such as GPUs are required for real-time, low-latency inference with
Deep Neural Networks (DNN). However, due to the inherent limits to the parallelism they …

With the development of deep learning, hardware accelerators represented by GPUs have
been used to accelerate the execution of deep learning applications. A key problem in GPU …

Hazelwood et al. observed that at Facebook data centers, variations in user activity (eg due
to diurnal load) resulted in low utilization periods with large pools of idle resources [4]. To …

Deep learning tasks (DLT) include training and inference tasks, where training DLTs have
requirements on minimizing average job completion time (JCT) and inference tasks need …

GPUs have been widely adopted to serve online deep learning-based services that have
stringent QoS requirements. However, emerging deep learning serving systems often result …

GPUs are known to benefit structured applications with ample parallelism, such as deep
learning in a datacenter. Recently, GPUs have shown promise for irregular streaming …

Modern hardware accelerators such as GP-GPUs and DSPs are commonly being used in
real-time settings such as high-performance multimedia systems and autonomous vehicles …

Modern cluster management systems, such as Kubernetes, support heterogeneous
workloads and resources. However, existing resource schedulers in these systems do not …