Aliasing black box adversarial attack with joint self-attention distribution and confidence probability
Deep neural networks (DNNs) are vulnerable to adversarial attacks, in which a small
perturbation to samples can cause misclassification. However, how to select important …
perturbation to samples can cause misclassification. However, how to select important …
Advances of pipeline model parallelism for deep learning training: an overview
Deep learning has become the cornerstone of artificial intelligence, playing an increasingly
important role in human production and lifestyle. However, as the complexity of problem …
important role in human production and lifestyle. However, as the complexity of problem …
{SOTER}: Guarding Black-box Inference for General Neural Networks at the Edge
The prosperity of AI and edge computing has pushed more and more well-trained DNN
models to be deployed on third-party edge devices to compose mission-critical applications …
models to be deployed on third-party edge devices to compose mission-critical applications …
Breaking the computation and communication abstraction barrier in distributed machine learning workloads
Recent trends towards large machine learning models require both training and inference
tasks to be distributed. Considering the huge cost of training these models, it is imperative to …
tasks to be distributed. Considering the huge cost of training these models, it is imperative to …
A survey on auto-parallelism of large-scale deep learning training
P Liang, Y Tang, X Zhang, Y Bai, T Su… - … on Parallel and …, 2023 - ieeexplore.ieee.org
Deep learning (DL) has gained great success in recent years, leading to state-of-the-art
performance in research community and industrial fields like computer vision and natural …
performance in research community and industrial fields like computer vision and natural …
{MGG}: Accelerating graph neural networks with {Fine-Grained}{Intra-Kernel}{Communication-Computation} pipelining on {Multi-GPU} platforms
The increasing size of input graphs for graph neural networks (GNNs) highlights the demand
for using multi-GPU platforms. However, existing multi-GPU GNN systems optimize the …
for using multi-GPU platforms. However, existing multi-GPU GNN systems optimize the …
Automatic pipeline parallelism: A parallel inference framework for deep learning applications in 6G mobile communication systems
With the rapid development of wireless communication, achieving the neXt generation Ultra-
Reliable and Low-Latency Communications (xURLLC) in 6G mobile communication …
Reliable and Low-Latency Communications (xURLLC) in 6G mobile communication …
Fold3D: Rethinking and Parallelizing Computational and Communicational Tasks in the Training of Large DNN Models
Training a large DNN (eg, GPT3) efficiently on commodity clouds is challenging even with
the latest 3D parallel training systems (eg, Megatron v3. 0). In particular, along the pipeline …
the latest 3D parallel training systems (eg, Megatron v3. 0). In particular, along the pipeline …
Explsched: Maximizing deep learning cluster efficiency for exploratory jobs
Resource management for Deep Learning (DL) clusters is essential for system efficiency
and model training quality. Existing schedulers provided by DL frameworks are mostly …
and model training quality. Existing schedulers provided by DL frameworks are mostly …
Distributed optimization of graph convolutional network using subgraph variance
In recent years, distributed graph convolutional networks (GCNs) training frameworks have
achieved great success in learning the representation of graph-structured data with large …
achieved great success in learning the representation of graph-structured data with large …