Learning Rate (LR) is an important hyper-parameter to tune for effective training of deep
neural networks (DNNs). Even for the baseline of a constant learning rate, it is non-trivial to …

Dataloaders, in charge of moving data from storage into GPUs while training machine
learning models, might hold the key to drastically improving the performance of training jobs …

Neural network ensembles are gaining popularity by harnessing the complementary wisdom
of multiple base models. Ensemble teams with high diversity promote high failure …

CPU is a powerful, pervasive, and indispensable platform for running deep learning (DL)
workloads in systems ranging from mobile to extreme-end servers. In this article, we present …

Deep learning has gained tremendous success in various fields while training deep neural
networks (DNNs) is very compute-intensive, which results in numerous deep learning …

The choice of learning rate (LR) functions and policies has evolved from a simple fixed LR to
the decaying LR and the cyclic LR, aiming to improve the accuracy and reduce the training …

Large Language Models (LLMs) represent the recent success of deep learning in achieving
remarkable human-like predictive performance. It has become a mainstream strategy to …

Tensors have drawn a growing attention in many applications, such as physics, engineering
science, social networks, recommended systems. Tensor decomposition is the key to …

Ensemble learning is gaining renewed interests in recent years. This paper presents
EnsembleBench, a holistic framework for evaluating and recommending high diversity and …

Deep learning frameworks are powerful tools to support model training. They dispatch
operators by mapping them into a series of kernel functions and launching these kernel …