Machine learning and the physical sciences
Machine learning (ML) encompasses a broad range of algorithms and modeling tools used
for a vast array of data processing tasks, which has entered most scientific disciplines in …
for a vast array of data processing tasks, which has entered most scientific disciplines in …
Big data deep learning: challenges and perspectives
XW Chen, X Lin - IEEE access, 2014 - ieeexplore.ieee.org
Deep learning is currently an extremely active research area in machine learning and
pattern recognition society. It has gained huge successes in a broad area of applications …
pattern recognition society. It has gained huge successes in a broad area of applications …
[HTML][HTML] High-dimensional dynamics of generalization error in neural networks
We perform an analysis of the average generalization dynamics of large neural networks
trained using gradient descent. We study the practically-relevant “high-dimensional” regime …
trained using gradient descent. We study the practically-relevant “high-dimensional” regime …
The loss surfaces of multilayer networks
We study the connection between the highly non-convex loss function of a simple model of
the fully-connected feed-forward neural network and the Hamiltonian of the spherical spin …
the fully-connected feed-forward neural network and the Hamiltonian of the spherical spin …
Modeling the influence of data structure on learning in neural networks: The hidden manifold model
Understanding the reasons for the success of deep neural networks trained using stochastic
gradient-based methods is a key open problem for the nascent theory of deep learning. The …
gradient-based methods is a key open problem for the nascent theory of deep learning. The …
Efficient backprop
The convergence of back-propagation learning is analyzed so as to explain common
phenomenon observedb y practitioners. Many undesirable behaviors of backprop can be …
phenomenon observedb y practitioners. Many undesirable behaviors of backprop can be …
Nonlinear random matrix theory for deep learning
J Pennington, P Worah - Advances in neural information …, 2017 - proceedings.neurips.cc
Neural network configurations with random weights play an important role in the analysis of
deep learning. They define the initial loss landscape and are closely related to kernel and …
deep learning. They define the initial loss landscape and are closely related to kernel and …
The gaussian equivalence of generative models for learning with shallow neural networks
Understanding the impact of data structure on the computational tractability of learning is a
key challenge for the theory of neural networks. Many theoretical works do not explicitly …
key challenge for the theory of neural networks. Many theoretical works do not explicitly …
Continual learning in the teacher-student setup: Impact of task similarity
Continual learning {—} the ability to learn many tasks in sequence {—} is critical for artificial
learning systems. Yet standard training methods for deep networks often suffer from …
learning systems. Yet standard training methods for deep networks often suffer from …
Classifying high-dimensional gaussian mixtures: Where kernel methods fail and neural networks succeed
A recent series of theoretical works showed that the dynamics of neural networks with a
certain initialisation are well-captured by kernel methods. Concurrent empirical work …
certain initialisation are well-captured by kernel methods. Concurrent empirical work …