As their size increases, Large Languages Models (LLMs) are natural candidates for network
pruning methods: approaches that drop a subset of network weights while striving to …

By forcing N out of M consecutive weights to be non-zero, the recent N: M fine-grained
network sparsity has received increasing attention with its two attractive advantages over …

Though network pruning receives popularity in reducing the complexity of convolutional
neural networks (CNNs), it remains an open issue to concurrently maintain model accuracy …

Deep learning technology has found a promising application in lightweight model design, for
which pruning is an effective means of achieving a large reduction in both model parameters …

We focus on addressing the dense backward propagation issue for training efficiency of N:
M fine-grained sparsity that preserves at most N out of M consecutive weights and achieves …

Sparse training has received an upsurging interest in machine learning due to its tantalizing
saving potential for both the entire training process as well as the inference. Dynamic sparse …

Recent innovations on hardware (eg Nvidia A100) have motivated learning N: M structured
sparsity masks from scratch for fast model inference. However, state-of-the-art learning …

Traditional channel-wise pruning methods by reducing network channels struggle to
effectively prune efficient CNN models with depth-wise convolutional layers and certain …

State-of-the-art deep learning models have a parameter count that reaches into the billions.
Training, storing and transferring such models is energy and time consuming, thus costly. A …

Despite the remarkable success of Large Language Models (LLMs), the massive size poses
significant deployment challenges, particularly on resource-constrained hardware. While …