Neural ordinary differential equations

RTQ Chen, Y Rubanova… - Advances in neural …, 2018 - proceedings.neurips.cc
We introduce a new family of deep neural network models. Instead of specifying a discrete
sequence of hidden layers, we parameterize the derivative of the hidden state using a …

Pde-net: Learning pdes from data

Z Long, Y Lu, X Ma, B Dong - International conference on …, 2018 - proceedings.mlr.press
Partial differential equations (PDEs) play a prominent role in many disciplines of science
and engineering. PDEs are commonly derived based on empirical observations. However …

Multistep neural networks for data-driven discovery of nonlinear dynamical systems

M Raissi, P Perdikaris, GE Karniadakis - arXiv preprint arXiv:1801.01236, 2018 - arxiv.org
The process of transforming observed data into predictive mathematical models of the
physical world has always been paramount in science and engineering. Although data is …

Reversible architectures for arbitrarily deep residual neural networks

B Chang, L Meng, E Haber, L Ruthotto… - Proceedings of the …, 2018 - ojs.aaai.org
Recently, deep residual networks have been successfully applied in many computer vision
and natural language processing tasks, pushing the state-of-the-art performance with …

[PDF][PDF] A mean-field optimal control formulation of deep learning

E Weinan, J Han, Q Li - arXiv preprint arXiv:1807.01083, 2018 - researchgate.net
Recent work linking deep neural networks and dynamical systems opened up new avenues
to analyze deep learning. In particular, it is observed that new insights can be obtained by …

Deep limits of residual neural networks

M Thorpe, Y van Gennip - arXiv preprint arXiv:1810.11741, 2018 - arxiv.org
Neural networks have been very successful in many applications; we often, however, lack a
theoretical understanding of what the neural networks are actually learning. This problem …

Dynamically unfolding recurrent restorer: A moving endpoint control method for image restoration

X Zhang, Y Lu, J Liu, B Dong - arXiv preprint arXiv:1805.07709, 2018 - arxiv.org
In this paper, we propose a new control framework called the moving endpoint control to
restore images corrupted by different degradation levels in one model. The proposed control …

Monge-amp\ere flow for generative modeling

L Zhang, L Wang - arXiv preprint arXiv:1809.10188, 2018 - arxiv.org
We present a deep generative model, named Monge-Amp\ere flow, which builds on
continuous-time gradient flow arising from the Monge-Amp\ere equation in optimal transport …

Functional gradient boosting based on residual network perception

A Nitanda, T Suzuki - International Conference on Machine …, 2018 - proceedings.mlr.press
Abstract Residual Networks (ResNets) have become state-of-the-art models in deep
learning and several theoretical studies have been devoted to understanding why ResNet …

Stochastic training of residual networks: a differential equation viewpoint

Q Sun, Y Tao, Q Du - arXiv preprint arXiv:1812.00174, 2018 - arxiv.org
During the last few years, significant attention has been paid to the stochastic training of
artificial neural networks, which is known as an effective regularization approach that helps …