How important are activation functions in regression and classification? A survey, performance comparison, and future directions

AD Jagtap, GE Karniadakis - Journal of Machine Learning for …, 2023 - dl.begellhouse.com
Inspired by biological neurons, the activation functions play an essential part in the learning
process of any artificial neural network (ANN) commonly used in many real-world problems …

A survey on recently proposed activation functions for Deep Learning

M Gustineli - arXiv preprint arXiv:2204.02921, 2022 - arxiv.org
Artificial neural networks (ANN), typically referred to as neural networks, are a class of
Machine Learning algorithms and have achieved widespread success, having been …

Biologically inspired oscillating activation functions can bridge the performance gap between biological and artificial neurons

MM Noel, S Bharadwaj, V Muthiah-Nakarajan… - arXiv preprint arXiv …, 2021 - arxiv.org
The recent discovery of special human neocortical pyramidal neurons that can individually
learn the XOR function highlights the significant performance gap between biological and …

HcLSH: a novel non-linear monotonic activation function for deep learning methods

H Abdel-Nabi, G Al-Naymat, MZ Ali, A Awajan - IEEE Access, 2023 - ieeexplore.ieee.org
Activation functions are essential components in any neural network model; they play a
crucial role in determining the network's expressive power through their introduced non …

Gish: a novel activation function for image classification

M Kaytan, IB Aydilek, C Yeroğlu - Neural Computing and Applications, 2023 - Springer
Abstract In Convolutional Neural Networks (CNNs), the selection and use of appropriate
activation functions is of critical importance. It has been seen that the Rectified Linear Unit …

Amplifying Sine Unit: An Oscillatory Activation Function for Deep Neural Networks to Recover Nonlinear Oscillations Efficiently

JU Rahman, F Makhdoom, D Lu - arXiv preprint arXiv:2304.09759, 2023 - arxiv.org
Many industrial and real life problems exhibit highly nonlinear periodic behaviors and the
conventional methods may fall short of finding their analytical or closed form solutions. Such …

ASU-CNN: An Efficient Deep Architecture for Image Classification and Feature Visualizations

JU Rahman, F Makhdoom, D Lu - arXiv preprint arXiv:2305.19146, 2023 - arxiv.org
Activation functions play a decisive role in determining the capacity of Deep Neural
Networks as they enable neural networks to capture inherent nonlinearities present in data …

[HTML][HTML] Modeling non-linear effects with neural networks in Relational Event Models

E Filippi-Mazzola, EC Wit - Social Networks, 2024 - Elsevier
Dynamic networks offer an insight of how relational systems evolve. However, modeling
these networks efficiently remains a challenge, primarily due to computational constraints …

NFT artwork generation using oscillatory activation functions in GANs

P Sharma, AR Sahoo, S Sinha, S Bharadwaj - 2022 - engrxiv.org
The concept of digital ownership is not new, and has been widely used in gaming contexts
to allow players to customize their experiences via profile pictures, skins, upgrades and add …

Using convolutional neural networks to detect compression algorithms

S Bharadwaj - … of International Conference on Communication and …, 2022 - Springer
Abstract Machine learning is penetrating various domains virtually, thereby proliferating
excellent results. It has also found an outlet in digital forensics, wherein it is becoming the …