Single-Program-Multiple-Data (SPMD) parallelism has recently been adopted to train large
deep neural networks (DNNs). Few studies have explored its applicability on heterogeneous …

Foundation models (FMs), known as pre-trained models, have garnered significant interest
in Industrial Internet due to their remarkable performance and robust generalization …

The challenges and resilience of vehicular ad hoc network (VANET) and deep neural
network (DNN) hybrid architectures in terms of reliability in smart cities have attracted much …

With the increasing proliferation of Internet-of-Things (IoT) devices, it is a growing trend
toward training a deep neural network (DNN) model in pipeline parallelism across resource …

Edge computing in federated learning based on centralized architecture often faces
communication constraints in large clusters. Although there have been some efforts like …

The recent breakthroughs in large-scale DNN attract significant attention from both
academia and industry toward distributed DNN training techniques. Due to the time …

Machine learning (ML) training and inference systems encounter constraints in current
computation environments due to increased ML model sizes, the fast-growing popularity of …