In this article, we review the literature on statistical theories of neural networks from three
perspectives: approximation, training dynamics, and generative models. In the first part …

Classically, data interpolation with a parametrized model class is possible as long as the
number of parameters is larger than the number of equations to be satisfied. A puzzling …

Classically, data interpolation with a parametrized model class is possible as long as the
number of parameters is larger than the number of equations to be satisfied. A puzzling …

In this paper, we propose a novel normalization method called gradient normalization (GN)
to tackle the training instability of Generative Adversarial Networks (GANs) caused by the …

Imitation learning trains a policy by mimicking expert demonstrations. Various imitation
methods were proposed and empirically evaluated, meanwhile, their theoretical …

Recent Transformer-based models such as Transformer-XL and BERT have achieved huge
success on various natural language processing tasks. However, contextualized …

Spectral normalization (SN) is a widely-used technique for improving the stability and
sample quality of Generative Adversarial Networks (GANs). However, current understanding …

The training process of generative adversarial networks (GANs) is unstable and does not
converge globally. In this paper, we examine the stability of GANs from the perspective of …

Generative adversarial networks (GANs) have been impactful on many problems and
applications but suffer from unstable training. The Wasserstein GAN (WGAN) leverages the …

As one of the representative models in the field of image generation, generative adversarial
networks (GANs) face a significant challenge: how to make the best trade-off between the …