Inspired by biological neurons, the activation functions play an essential part in the learning
process of any artificial neural network (ANN) commonly used in many real-world problems …

As machine learning (ML) becomes pervasive in high performance computing, ML has
found its way into safety-critical domains (eg, autonomous vehicles). Thus the reliability of …

Neural networks have proven to be a highly effective tool for solving complex problems in
many areas of life. Recently, their importance and practical usability have further been …

In artificial neural networks, activation functions play a significant role in the learning
process. Choosing the proper activation function is a major factor in achieving a successful …

We present a neural network technique for the analysis and extrapolation of time-series data
called neural decomposition (ND). Units with a sinusoidal activation function are used to …

Transformer-based language models utilize the attention mechanism for substantial
performance improvements in almost all natural language processing (NLP) tasks. Similar …

We present the soft exponential activation function for artificial neural networks that
continuously interpolates between logarithmic, linear, and exponential functions. This …

Diversity of environments is a key challenge that causes learned robotic controllers to fail
due to the discrepancies between the training and evaluation conditions. Training from …

The activation function is introduced to determine the output of neural networks by mapping
the resulting values of neurons into a specific range. The activation functions often suffer …

We propose a data-driven artificial viscosity model for shock capturing in discontinuous
Galerkin methods. The proposed model trains a multi-layer feedforward network to map from …