Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial
intelligence (AI), including computer vision, natural language processing, and speech …

Data-parallel distributed training (DDT) has become the de-facto standard for accelerating
the training of most deep learning tasks on massively parallel hardware. In the DDT …

With the rapid development of deep learning, training Big Models (BMs) for multiple
downstream tasks becomes a popular paradigm. Researchers have achieved various …

In this paper, the use of Artificial Neural Networks (ANNs) in the form of Convolutional
Neural Networks (AlexNET) for the fast and energy-efficient fitting of the Dynamic Memdiode …

Sago palm tree, known as Metroxylon Sagu Rottb, is one of the priority commodities in
Indonesia. Based on our previous research, the potential habitat of the plant has been …

Communication overhead is the key challenge for distributed training. Gradient compression
is a widely used approach to reduce communication traffic. When combined with a parallel …

Variations of stochastic gradient decedent (SGD) methods are at the core of training deep
neural network models. However, in distributed deep learning, where multiple computing …

We propose VirtualFlow, a system leveraging a novel abstraction called virtual node
processing to decouple the model from the hardware. In each step of training or inference …

Distributed data-parallel training has been widely adopted for deep neural network (DNN)
models. Although current deep learning (DL) frameworks scale well for dense models like …

Gradient compression (GC) is a promising approach to addressing the communication
bottleneck in distributed deep learning (DDL). However, it is challenging to find the optimal …