Neural jump stochastic differential equations
Many time series are effectively generated by a combination of deterministic continuous
flows along with discrete jumps sparked by stochastic events. However, we usually do not …
flows along with discrete jumps sparked by stochastic events. However, we usually do not …
Infinitely deep bayesian neural networks with stochastic differential equations
We perform scalable approximate inference in continuous-depth Bayesian neural networks.
In this model class, uncertainty about separate weights in each layer gives hidden units that …
In this model class, uncertainty about separate weights in each layer gives hidden units that …
Width and depth limits commute in residual networks
We show that taking the width and depth to infinity in a deep neural network with skip
connections, when branches are scaled by $1/\sqrt {depth} $, result in the same covariance …
connections, when branches are scaled by $1/\sqrt {depth} $, result in the same covariance …
On the infinite-depth limit of finite-width neural networks
S Hayou - Transactions on Machine Learning Research, 2022 - openreview.net
In this paper, we study the infinite-depth limit of finite-width residual neural networks with
random Gaussian weights. With proper scaling, we show that by fixing the width and taking …
random Gaussian weights. With proper scaling, we show that by fixing the width and taking …
Wide neural networks as gaussian processes: Lessons from deep equilibrium models
Neural networks with wide layers have attracted significant attention due to their
equivalence to Gaussian processes, enabling perfect fitting of training data while …
equivalence to Gaussian processes, enabling perfect fitting of training data while …
Neural tangent kernel analysis of deep narrow neural networks
The tremendous recent progress in analyzing the training dynamics of overparameterized
neural networks has primarily focused on wide networks and therefore does not sufficiently …
neural networks has primarily focused on wide networks and therefore does not sufficiently …
Scaling properties of deep residual networks
Residual networks (ResNets) have displayed impressive results in pattern recognition and,
recently, have garnered considerable theoretical interest due to a perceived link with neural …
recently, have garnered considerable theoretical interest due to a perceived link with neural …
Stochastic normalizing flows
We introduce stochastic normalizing flows, an extension of continuous normalizing flows for
maximum likelihood estimation and variational inference (VI) using stochastic differential …
maximum likelihood estimation and variational inference (VI) using stochastic differential …
Commutative Scaling of Width and Depth in Deep Neural Networks
S Hayou - Journal of Machine Learning Research, 2024 - jmlr.org
In this paper, we study the commutativity of infinite width and depth limits in deep neural
networks. Our aim is to understand the behavior of neural functions (functions that depend …
networks. Our aim is to understand the behavior of neural functions (functions that depend …
Deep Learning-based Approaches for State Space Models: A Selective Review
J Lin, G Michailidis - arXiv preprint arXiv:2412.11211, 2024 - arxiv.org
State-space models (SSMs) offer a powerful framework for dynamical system analysis,
wherein the temporal dynamics of the system are assumed to be captured through the …
wherein the temporal dynamics of the system are assumed to be captured through the …