Deep neural networks are generally designed as a stack of differentiable layers, in which a
prediction is obtained only after running the full stack. Recently, some contributions have …

In recent years, larger and deeper models are springing up and continuously pushing state-
of-the-art (SOTA) results across various fields like natural language processing (NLP) and …

Shallow supervised 1-hidden layer neural networks have a number of favorable properties
that make them easier to interpret, analyze, and optimize than their deep counterparts, but …

Due to the need to store the intermediate activations for back-propagation, end-to-end (E2E)
training of deep networks usually suffers from high GPUs memory footprint. This paper aims …

In this paper, we propose a novel approach for efficient training of deep neural networks in a
bottom-up fashion using a layered structure. Our algorithm, which we refer to as deep …

For a long time, biology and neuroscience fields have been a great source of inspiration for
computer scientists, towards the development of Artificial Intelligence (AI) technologies. This …

Abstract Operational Neural Networks (ONNs) have recently been proposed to address the
well-known limitations and drawbacks of conventional Convolutional Neural Networks …

Feed-forward, fully connected artificial neural networks or the so-called multi-layer
perceptrons are well-known universal approximators. However, their learning performance …

Greedy layer-wise or module-wise training of neural networks is compelling in constrained
and on-device settings where memory is limited, as it circumvents a number of problems of …

The traditional multilayer perceptron (MLP) using a McCulloch-Pitts neuron model is
inherently limited to a set of neuronal activities, ie, linear weighted sum followed by …