Survey of optimization algorithms in modern neural networks

R Abdulkadirov, P Lyakhov, N Nagornov - Mathematics, 2023 - mdpi.com
The main goal of machine learning is the creation of self-learning algorithms in many areas
of human activity. It allows a replacement of a person with artificial intelligence in seeking to …

A systematic literature review on binary neural networks

R Sayed, H Azmi, H Shawkey, AH Khalil… - IEEE Access, 2023 - ieeexplore.ieee.org
This paper presents an extensive literature review on Binary Neural Network (BNN). BNN
utilizes binary weights and activation function parameters to substitute the full-precision …

Bibench: Benchmarking and analyzing network binarization

H Qin, M Zhang, Y Ding, A Li, Z Cai… - International …, 2023 - proceedings.mlr.press
Network binarization emerges as one of the most promising compression approaches
offering extraordinary computation and memory savings by minimizing the bit-width …

Pokebnn: A binary pursuit of lightweight accuracy

Y Zhang, Z Zhang, L Lew - … of the IEEE/CVF Conference on …, 2022 - openaccess.thecvf.com
Abstract Optimization of Top-1 ImageNet promotes enormous networks that may be
impractical in inference settings. Binary neural networks (BNNs) have the potential to …

Pb-llm: Partially binarized large language models

Y Shang, Z Yuan, Q Wu, Z Dong - arXiv preprint arXiv:2310.00034, 2023 - arxiv.org
This paper explores network binarization, a radical form of quantization, compressing model
weights to a single bit, specifically for Large Language Models (LLMs) compression. Due to …

Automl in the age of large language models: Current challenges, future opportunities and risks

A Tornede, D Deng, T Eimer, J Giovanelli… - arXiv preprint arXiv …, 2023 - arxiv.org
The fields of both Natural Language Processing (NLP) and Automated Machine Learning
(AutoML) have achieved remarkable results over the past years. In NLP, especially Large …

Efar 2023: Efficient face recognition competition

JN Kolf, F Boutros, J Elliesen… - … Joint Conference on …, 2023 - ieeexplore.ieee.org
This paper presents the summary of the Efficient Face Recognition Competition (EFaR) held
at the 2023 International Joint Conference on Biometrics (IJCB 2023). The competition …

[HTML][HTML] S3NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks

K Suetake, S Ikegawa, R Saiin, Y Sawada - Neural Networks, 2023 - Elsevier
As the scales of neural networks increase, techniques that enable them to run with low
computational cost and energy efficiency are required. From such demands, various efficient …

Taming binarized neural networks and mixed-integer programs

J Aspman, G Korpas, J Marecek - … of the AAAI Conference on Artificial …, 2024 - ojs.aaai.org
There has been a great deal of recent interest in binarized neural networks, especially
because of their explainability. At the same time, automatic differentiation algorithms such as …

A comprehensive review and a taxonomy of edge machine learning: Requirements, paradigms, and techniques

W Li, H Hacid, E Almazrouei, M Debbah - AI, 2023 - mdpi.com
The union of Edge Computing (EC) and Artificial Intelligence (AI) has brought forward the
Edge AI concept to provide intelligent solutions close to the end-user environment, for …