A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations
Modern deep neural networks, particularly recent large language models, come with
massive model sizes that require significant computational and storage resources. To …
massive model sizes that require significant computational and storage resources. To …
Efficient deep learning: A survey on making deep learning models smaller, faster, and better
G Menghani - ACM Computing Surveys, 2023 - dl.acm.org
Deep learning has revolutionized the fields of computer vision, natural language
understanding, speech recognition, information retrieval, and more. However, with the …
understanding, speech recognition, information retrieval, and more. However, with the …
Sparsegpt: Massive language models can be accurately pruned in one-shot
E Frantar, D Alistarh - International Conference on Machine …, 2023 - proceedings.mlr.press
We show for the first time that large-scale generative pretrained transformer (GPT) family
models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal …
models can be pruned to at least 50% sparsity in one-shot, without any retraining, at minimal …
A simple and effective pruning approach for large language models
As their size increases, Large Languages Models (LLMs) are natural candidates for network
pruning methods: approaches that drop a subset of network weights while striving to …
pruning methods: approaches that drop a subset of network weights while striving to …
Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives
Abstract We present Ego-Exo4D a diverse large-scale multimodal multiview video dataset
and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric …
and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric …
Optimal brain compression: A framework for accurate post-training quantization and pruning
E Frantar, D Alistarh - Advances in Neural Information …, 2022 - proceedings.neurips.cc
We consider the problem of model compression for deep neural networks (DNNs) in the
challenging one-shot/post-training setting, in which we are given an accurate trained model …
challenging one-shot/post-training setting, in which we are given an accurate trained model …
Language models are super mario: Absorbing abilities from homologous models as a free lunch
In this paper, we unveil that Language Models (LMs) can acquire new capabilities by
assimilating parameters from homologous models without retraining or GPUs. We first …
assimilating parameters from homologous models without retraining or GPUs. We first …
NTIRE 2023 challenge on efficient super-resolution: Methods and results
This paper reviews the NTIRE 2023 challenge on efficient single-image super-resolution
with a focus on the proposed solutions and results. The aim of this challenge is to devise a …
with a focus on the proposed solutions and results. The aim of this challenge is to devise a …
M6-rec: Generative pretrained language models are open-ended recommender systems
Industrial recommender systems have been growing increasingly complex, may
involve\emph {diverse domains} such as e-commerce products and user-generated …
involve\emph {diverse domains} such as e-commerce products and user-generated …
A survey on efficient convolutional neural networks and hardware acceleration
Over the past decade, deep-learning-based representations have demonstrated remarkable
performance in academia and industry. The learning capability of convolutional neural …
performance in academia and industry. The learning capability of convolutional neural …