Cortical neurons emit seemingly erratic trains of action potentials or “spikes,” and neural
network dynamics emerge from the coordinated spiking activity within neural circuits. These …

The vanishing gradient problem (VGP) is an important issue at training time on multilayer
neural networks using the backpropagation algorithm. This problem is worse when sigmoid …

Recurrent neural networks are widely used on time series data, yet such models often
ignore the underlying physical structures in such sequences. A new class of physics-based …

Circuits of biological neurons, such as in the functional parts of the brain can be modeled as
networks of coupled oscillators. Inspired by the ability of these systems to express a rich set …

Viewing recurrent neural networks (RNNs) as continuous-time dynamical systems, we
propose a recurrent unit that describes the hidden state's evolution with two parts: a well …

The design of recurrent neural networks (RNNs) to accurately process sequential inputs with
long-time dependencies is very challenging on account of the exploding and vanishing …

The exploding and vanishing gradient problem has been the major conceptual principle
behind most architecture and training improvements in recurrent neural networks (RNNs) …

Most recurrent neural networks (RNNs) do not include a fundamental constraint of real
neural circuits: Dale's Law, which implies that neurons must be excitatory (E) or inhibitory (I) …

Back-propagation through time (BPTT) has been widely used for training Recurrent Neural
Networks (RNNs). BPTT updates RNN parameters on an instance by back-propagating the …

Physics-informed machine learning (PIML) is a set of methods and tools that systematically
integrate machine learning (ML) algorithms with physical constraints and abstract …