Generalization bounds for neural ordinary differential equations and deep residual networks

P Marion - Advances in Neural Information Processing …, 2024 - proceedings.neurips.cc
Neural ordinary differential equations (neural ODEs) are a popular family of continuous-
depth deep learning models. In this work, we consider a large family of parameterized ODEs …

Implicit regularization of deep residual networks towards neural ODEs

P Marion, YH Wu, ME Sander, G Biau - arXiv preprint arXiv:2309.01213, 2023 - arxiv.org
Residual neural networks are state-of-the-art deep learning models. Their continuous-depth
analog, neural ordinary differential equations (ODEs), are also widely used. Despite their …

Deep learning bulk spacetime from boundary optical conductivity

B Ahn, HS Jeong, KY Kim, K Yun - Journal of High Energy Physics, 2024 - Springer
A bstract We employ a deep learning method to deduce the bulk spacetime from boundary
optical conductivity. We apply the neural ordinary differential equation technique, tailored for …

Residual alignment: uncovering the mechanisms of residual networks

J Li, V Papyan - Advances in Neural Information Processing …, 2024 - proceedings.neurips.cc
The ResNet architecture has been widely adopted in deep learning due to its significant
boost to performance through the use of simple skip connections, yet the underlying …

A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation.

H Niu, Z Yi, T He - International Journal of Neural Systems, 2024 - europepmc.org
Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized
various image recognition tasks. In this paper, we propose a novel architecture called …

Approximating Langevin Monte Carlo with ResNet-like neural network architectures

M Eigel, C Miranda, J Schütte, D Sommer - arXiv preprint arXiv …, 2023 - arxiv.org
We sample from a given target distribution by constructing a neural network which maps
samples from a simple reference, eg the standard normal distribution, to samples from the …

Vanilla feedforward neural networks as a discretization of dynamic systems

Y Duan, L Li, G Ji, Y Cai - arXiv preprint arXiv:2209.10909, 2022 - arxiv.org
Deep learning has made significant applications in the field of data science and natural
science. Some studies have linked deep neural networks to dynamic systems, but the …

A Novel Convolutional Neural Network Architecture with a Continuous Symmetry

Y Liu, H Shao, B Bai - CAAI International Conference on Artificial …, 2023 - Springer
This paper introduces a new Convolutional Neural Network (ConvNet) architecture inspired
by a class of partial differential equations (PDEs) called quasi-linear hyperbolic systems …

Deep linear networks for regression are implicitly regularized towards flat minima

P Marion, L Chizat - arXiv preprint arXiv:2405.13456, 2024 - arxiv.org
The largest eigenvalue of the Hessian, or sharpness, of neural networks is a key quantity to
understand their optimization dynamics. In this paper, we study the sharpness of deep linear …

A multimetric evaluation method for comprehensively assessing the influence of the icosahedral diamond grid quality on SCNN performance

Y Duan, X Zhao, W Sun, Q Liu, M Qin - International Journal of …, 2024 - Taylor & Francis
The increasing availability of global observational data has sparked a demand for deep
learning algorithms on spherical grids to enable intelligent analysis at a global scale …