Neural network pruning typically removes connections or neurons from a pretrained
converged model; while a new pruning paradigm, pruning at initialization (PaI), attempts to …

The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

Pruning the parameters of deep neural networks has generated intense interest due to
potential savings in time, memory and energy both during training and at test time. Recent …

Overparameterization has been shown to benefit both the optimization and generalization of
neural networks, but large networks are resource hungry at both training and test time …

Second-order information, in the form of Hessian-or Inverse-Hessian-vector products, is a
fundamental tool for solving optimization problems. Recently, there has been significant …

Weight pruning is an effective technique to reduce the model size and inference time for
deep neural networks in real world deployments. However, since magnitudes and relative …

Reducing the test time resource requirements of a neural network while preserving test
accuracy is crucial for running inference on resource-constrained devices. To achieve this …

Sparse training is a natural idea to accelerate the training speed of deep neural networks
and save the memory usage, especially since large modern neural networks are …

The emergence of powerful deep learning architectures has resulted in breakthrough
innovations in several fields such as healthcare, precision farming, banking, education, and …

We present algorithms and an architectural methodology to enable zero skipping while
increasing frequency in per-layer customized data flow Convolutional Neural Network …