This paper presents a solution to the challenge of mitigating carbon emissions from hosting
large-scale machine learning (ML) inference services. ML inference is critical to modern …

Online inference is becoming a key service product for many businesses, deployed in cloud
platforms to meet customer demands. Despite their revenue-generation capability, these …

Transformer has dominated the field of natural language processing because of its strong
capability in learning from sequential input data. In recent years, various computing and …

As machine learning (ML) becomes more and more popular, datacenter operators use
hardware accelerators such as GPUs to tackle the high computation demand of ML …

In today's production machine learning (ML) systems, models are continuously trained,
improved, and deployed. ML design and training are becoming a continuous workflow of …

Object cloud storage systems are deployed with diverse applications that have varying
latency service level objectives (SLOs), posting challenges for supporting quality of service …

With the emerging of machine learning, many commercial companies increasingly utilize
machine learning inference systems as backend services to improve their products …

Two notable characteristics of modern GPU-accelerated HPC clusters are:(1) they
increasingly run deep learning (DL) model-training workloads, and (2) they consist of …

Recent years have witnessed increasing interest in machine learning inferences on
serverless computing for its auto-scaling and cost effective properties. Existing serverless …

The use of intelligent sensors and edge devices has grown exponentially for automation in
the industry to hyper-personalize applications, minimize cost, improve efficiency, and …