Provable guarantees for neural networks via gradient feature learning
Neural networks have achieved remarkable empirical performance, while the current
theoretical analysis is not adequate for understanding their success, eg, the Neural Tangent …
theoretical analysis is not adequate for understanding their success, eg, the Neural Tangent …
Provable guarantees for nonlinear feature learning in three-layer neural networks
One of the central questions in the theory of deep learning is to understand how neural
networks learn hierarchical features. The ability of deep networks to extract salient features …
networks learn hierarchical features. The ability of deep networks to extract salient features …
Learning hierarchical polynomials with three-layer neural networks
We study the problem of learning hierarchical polynomials over the standard Gaussian
distribution with three-layer neural networks. We specifically consider target functions of the …
distribution with three-layer neural networks. We specifically consider target functions of the …
Rethink depth separation with intra-layer links
FL Fan, ZY Li, H Xiong, T Zeng - arXiv preprint arXiv:2305.07037, 2023 - arxiv.org
The depth separation theory is nowadays widely accepted as an effective explanation for the
power of depth, which consists of two parts: i) there exists a function representable by a …
power of depth, which consists of two parts: i) there exists a function representable by a …
Depth Separations in Neural Networks: Separating the Dimension from the Accuracy
We prove an exponential separation between depth 2 and depth 3 neural networks, when
approximating an $\mathcal {O}(1) $-Lipschitz target function to constant accuracy, with …
approximating an $\mathcal {O}(1) $-Lipschitz target function to constant accuracy, with …