Parametric Activation Functions for Neural Networks: A Tutorial Survey

LS Pusztaházi, G Eigner, O Csiszár - IEEE Access, 2024 - ieeexplore.ieee.org
Activation functions are pivotal in neural networks, determining the output of each neuron.
Traditionally, functions like sigmoid and ReLU have been static and deterministic. However …

[HTML][HTML] Uninorm-like parametric activation functions for human-understandable neural models

O Csiszár, LS Pusztaházi, L Dénes-Fazakas… - Knowledge-Based …, 2023 - Elsevier
We present a deep learning model for finding human-understandable connections between
input features. Our approach uses a parameterized, differentiable activation function, based …

Esc-rules: Explainable, semantically constrained rule sets

M Glauer, R West, S Michie, J Hastings - arXiv preprint arXiv:2208.12523, 2022 - arxiv.org
We describe a novel approach to explainable prediction of a continuous variable based on
learning fuzzy weighted rules. Our model trains a set of weighted rules to maximise …

Short-Term Forecasting of Daily Reference Crop Evapotranspiration Based on Calibrated Hargreaves–Samani Equation at Regional Scale

S Baber, K Ullah - Earth Systems and Environment, 2024 - Springer
This study focuses on enhancing real-time irrigation decisions and stream flow forecasts
using short-term daily forecasts of reference evapotranspiration (ETo). While conventional …

[HTML][HTML] Laser simulator logic: A novel inference system for highly overlapping of linguistic variable in membership functions

MAH Ali, S Mekhilef, N Yusoff, B Abd Razak - Journal of King Saud …, 2022 - Elsevier
A new approach for Fuzzification and Defuzzification processes of a high degree of
overlapping between the linguistic variables through proportional and relative-dynamic …

Optimizing neural networks through activation function discovery and automatic weight initialization

G Bingham - arXiv preprint arXiv:2304.03374, 2023 - arxiv.org
Automated machine learning (AutoML) methods improve upon existing models by
optimizing various aspects of their design. While present methods focus on hyperparameters …

[图书][B] Parameterizing and aggregating activation functions in deep neural networks

LB Godfrey - 2018 - search.proquest.com
The nonlinear activation functions applied by each neuron in a neural network are essential
for making neural networks powerful representational models. If these are omitted, even …

Parametric activation functions modelling fuzzy connectives for better explainability of neural models

LS Pusztaházi, G Csiszár, MS Gashler… - 2022 IEEE 20th …, 2022 - ieeexplore.ieee.org
In this work, a neuro-fuzzy hybrid deep learning model is presented for finding human-
readable relationships between input features with the help of nilpotent fuzzy logic and multi …

Logical Activation Functions: Logit-space equivalents of Boolean Operators

SC Lowe, R Earle, J d'Eon, T Trappenberg, S Oore - 2021 - openreview.net
Neuronal representations within artificial neural networks are commonly understood as
logits, representing the log-odds score of presence (versus absence) of features within the …

Logical activation functions: logit-space equivalents of probabilistic boolean operators

S Lowe, R Earle, J d'Eon… - Advances in Neural …, 2022 - proceedings.neurips.cc
The choice of activation functions and their motivation is a long-standing issue within the
neural network community. Neuronal representations within artificial neural networks are …