Model compression and hardware acceleration for neural networks: A comprehensive survey

L Deng, G Li, S Han, L Shi, Y Xie - Proceedings of the IEEE, 2020 - ieeexplore.ieee.org
Domain-specific hardware is becoming a promising topic in the backdrop of improvement
slow down for general-purpose processors due to the foreseeable end of Moore's Law …

[HTML][HTML] Deep learning with spiking neurons: Opportunities and challenges

M Pfeiffer, T Pfeil - Frontiers in neuroscience, 2018 - frontiersin.org
Spiking neural networks (SNNs) are inspired by information processing in biology, where
sparse and asynchronous binary signals are communicated and processed in a massively …

Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery

Y Wen, N Jain, J Kirchenbauer… - Advances in …, 2024 - proceedings.neurips.cc
The strength of modern generative models lies in their ability to be controlled through
prompts. Hard prompts comprise interpretable words and tokens, and are typically hand …

Binary neural networks: A survey

H Qin, R Gong, X Liu, X Bai, J Song, N Sebe - Pattern Recognition, 2020 - Elsevier
The binary neural network, largely saving the storage and computation, serves as a
promising technique for deploying deep models on resource-limited devices. However, the …

Differentiable soft quantization: Bridging full-precision and low-bit neural networks

R Gong, X Liu, S Jiang, T Li, P Hu… - Proceedings of the …, 2019 - openaccess.thecvf.com
Hardware-friendly network quantization (eg, binary/uniform quantization) can efficiently
accelerate the inference and meanwhile reduce memory consumption of the deep neural …

Qsparse-local-SGD: Distributed SGD with quantization, sparsification and local computations

D Basu, D Data, C Karakus… - Advances in Neural …, 2019 - proceedings.neurips.cc
Communication bottleneck has been identified as a significant issue in distributed
optimization of large-scale learning models. Recently, several approaches to mitigate this …

Blockdrop: Dynamic inference paths in residual networks

Z Wu, T Nagarajan, A Kumar… - Proceedings of the …, 2018 - openaccess.thecvf.com
Very deep convolutional neural networks offer excellent recognition results, yet their
computational expense limits their impact for many real-world applications. We introduce …

Training and inference with integers in deep neural networks

S Wu, G Li, F Chen, L Shi - arXiv preprint arXiv:1802.04680, 2018 - arxiv.org
Researches on deep neural networks with discrete parameters and their deployment in
embedded systems have been active and promising topics. Although previous works have …

Ultra-low precision 4-bit training of deep neural networks

X Sun, N Wang, CY Chen, J Ni… - Advances in …, 2020 - proceedings.neurips.cc
In this paper, we propose a number of novel techniques and numerical representation
formats that enable, for the very first time, the precision of training systems to be aggressively …

A survey on methods and theories of quantized neural networks

Y Guo - arXiv preprint arXiv:1808.04752, 2018 - arxiv.org
Deep neural networks are the state-of-the-art methods for many real-world tasks, such as
computer vision, natural language processing and speech recognition. For all its popularity …