A random matrix approach to neural networks

C Louart, Z Liao, R Couillet - The Annals of Applied Probability, 2018 - JSTOR
… norm, W ∈ Rn×p is a matrix of independent zero-mean unit … lemma arising from nonasymptotic
random matrix arguments, … ple covariance matrix models, involving notably the moment …

Geometry of neural network loss surfaces via random matrix theory

J Pennington, Y Bahri - International conference on machine …, 2017 - proceedings.mlr.press
… as a structured random matrix and we study its eigenvalues in the context of random matrix
theory, using tools from free probability. We focus on singlehidden-layer networks, but in …

Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning

CH Martin, MW Mahoney - Journal of Machine Learning Research, 2021 - jmlr.org
… We will use advanced methods from Random Matrix Theory (RMT), developed in the theory
of self organizing systems, to characterize DNN layer weight matrices, Wl,2 during and after …

Nonlinear random matrix theory for deep learning

J Pennington, P Worah - Advances in neural information …, 2017 - proceedings.neurips.cc
… door for direct applications of random matrix theory to deep … neural networks can be
incorporated into a standard method of proof in random matrix theory known as the moments method

Randomness in neural networks: an overview

S Scardapane, D Wang - Wiley Interdisciplinary Reviews: Data …, 2017 - Wiley Online Library
… This class of neural networks attractive and valuable to the data mining … neural networks
and kernel functions. A clear exposition of the basic framework underlying all these approaches

Eigenvalue spectra of random matrices for neural networks

K Rajan, LF Abbott - Physical review letters, 2006 - APS
… spectra of random matrices do not apply to synaptic connectivity matrices because of the …
Therefore, we compute eigenvalue spectra of large random matrices with excitatory and …

Products of many large random matrices and gradients in deep neural networks

B Hanin, M Nica - Communications in Mathematical Physics, 2020 - Springer
… We study products of random matrices in the regime where the number of terms and the size
of the matrices simultaneously tend to infinity. Our main theorem is that the logarithm of the \(…

Path integral approach to random neural networks

A Crisanti, H Sompolinsky - Physical Review E, 2018 - APS
… Here we present a systematic method based on path integrals which overcomes … the method
to a large nonlinear rate-based neural network with random asymmetric connectivity matrix. …

More than a toy: Random matrix models predict how real-world neural representations generalize

A Wei, W Hu, J Steinhardt - International Conference on …, 2022 - proceedings.mlr.press
… , when applied to kernels derived from large-scale neural networks (eg, ResNet-50) and real
… whenever a local random matrix law holds. Finally, we apply this random matrix theory lens …

Random matrix theory and wireless communications

AM Tulino, S Verdú - Foundations and Trends® in …, 2004 - nowpublishers.com
… For example, in 1977 Grenander and Silverstein [101] rediscovered (1.10) motivated by a
neural network problem where the entries of H take only two values. Also unaware of the in-…