Improving the performance of deep neural networks using two proposed activation functions

AA Alkhouly, A Mohammed, HA Hefny - IEEE Access, 2021 - ieeexplore.ieee.org
In artificial neural networks, activation functions play a significant role in the learning
process. Choosing the proper activation function is a major factor in achieving a successful …

HcLSH: a novel non-linear monotonic activation function for deep learning methods

H Abdel-Nabi, G Al-Naymat, MZ Ali, A Awajan - IEEE Access, 2023 - ieeexplore.ieee.org
Activation functions are essential components in any neural network model; they play a
crucial role in determining the network's expressive power through their introduced non …

A survey on recently proposed activation functions for Deep Learning

M Gustineli - arXiv preprint arXiv:2204.02921, 2022 - arxiv.org
Artificial neural networks (ANN), typically referred to as neural networks, are a class of
Machine Learning algorithms and have achieved widespread success, having been …

The quest for the golden activation function

M Basirat, PM Roth - arXiv preprint arXiv:1808.00783, 2018 - arxiv.org
Deep Neural Networks have been shown to be beneficial for a variety of tasks, in particular
allowing for end-to-end learning and reducing the requirement for manual design decisions …

An empirical study on generalizations of the ReLU activation function

C Banerjee, T Mukherjee, E Pasiliao Jr - Proceedings of the 2019 ACM …, 2019 - dl.acm.org
Deep Neural Networks have become the tool of choice for Machine Learning practitioners
today. They have been successfully applied for solving a large class of learning problems …

Nipuna: A novel optimizer activation function for deep neural networks

G Madhu, S Kautish, KA Alnowibet, HM Zawbaa… - Axioms, 2023 - mdpi.com
In recent years, various deep neural networks with different learning paradigms have been
widely employed in various applications, including medical diagnosis, image analysis, self …

Parametric deformable exponential linear units for deep neural networks

Q Cheng, HL Li, Q Wu, L Ma, KN Ngan - Neural Networks, 2020 - Elsevier
Rectified activation units make an important contribution to the success of deep neural
networks in many computer vision tasks. In this paper, we propose a Parametric Deformable …

Learning activation functions: A new paradigm for understanding neural networks

M Goyal, R Goyal, B Lall - arXiv preprint arXiv:1906.09529, 2019 - arxiv.org
The scope of research in the domain of activation functions remains limited and centered
around improving the ease of optimization or generalization quality of neural networks …

Activation functions: Comparison of trends in practice and research for deep learning

C Nwankpa, W Ijomah, A Gachagan… - arXiv preprint arXiv …, 2018 - arxiv.org
Deep neural networks have been successfully used in diverse emerging domains to solve
real world complex problems with may more deep learning (DL) architectures, being …

Activation functions in deep learning: A comprehensive survey and benchmark

SR Dubey, SK Singh, BB Chaudhuri - Neurocomputing, 2022 - Elsevier
Neural networks have shown tremendous growth in recent years to solve numerous
problems. Various types of neural networks have been introduced to deal with different types …