Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

Although Transformer-based language representations achieve state-of-the-art accuracy on
various natural language processing (NLP) tasks, the large model size has been …

In recent years, Deep Neural Networks (DNNs) have emerged as a widely adopted
approach in many application domains. Training DNN models is also becoming a significant …

Existing general purpose frameworks for gigantic model training, ie, dense models with
billions of parameters, cannot scale efficiently on cloud environment with various networking …

Distributed training of large deep neural networks requires frequent exchange of massive
data between machines, thus communication efficiency is a major concern. Existing …

Due to the explosion in the size of the training datasets, distributed learning has received
growing interest in recent years. One of the major bottlenecks is the large communication …

The COVID-19 pandemic highlights the need for computational tools to automate and
accelerate drug design for novel protein targets. We leverage deep learning language …

In the Australian construction industry, effective supply chain risk management (SCRM) is
critical due to its complex networks and susceptibility to various risks. This study explores the …

Dense embedding-based semantic matching is widely used in e-commerce product search
to address the shortcomings of lexical matching such as sensitivity to spelling variants. The …

The widespread popularity of deep neural networks (DNNs) has made it an important
workload in modern datacenters. Training DNNs is both computation-intensive and memory …