Understanding deep learning is increasingly emergent as it penetrates more and more into
industry and science. In recent years, a research line from Fourier analysis sheds light on …

The computing demand for massive applications has led to the ubiquitous deployment of
computing power. This trend results in the urgent need for higher-level computing resource …

The generalization mystery of overparametrized deep nets has motivated efforts to
understand how gradient descent (GD) converges to low-loss solutions that generalize well …

Neural networks have achieved remarkable empirical performance, while the current
theoretical analysis is not adequate for understanding their success, eg, the Neural Tangent …

The training process of ReLU neural networks often exhibits complicated nonlinear
phenomena. The nonlinearity of models and non-convexity of loss pose significant …

We analytically investigate how over-parameterization of models in randomized machine
learning algorithms impacts the information leakage about their training data. Specifically …

We study the average robustness notion in deep neural networks in (selected) wide and
narrow, deep and shallow, as well as lazy and non-lazy training settings. We prove that in …

Substantial work indicates that the dynamics of neural networks (NNs) is closely related to
their initialization of parameters. Inspired by the phase diagram for two-layer ReLU NNs with …

We prove a general Embedding Principle of loss landscape of deep neural networks (NNs)
that unravels a hierarchical structure of the loss landscape of NNs, ie, loss landscape of an …

Understanding the structure of loss landscape of deep neural networks (DNNs) is obviously
important. In this work, we prove an embedding principle that the loss landscape of a DNN" …