A systematic review on overfitting control in shallow and deep neural networks

MM Bejani, M Ghatee - Artificial Intelligence Review, 2021 - Springer
Shallow neural networks process the features directly, while deep networks extract features
automatically along with the training. Both models suffer from overfitting or poor …

Normalization techniques in training dnns: Methodology, analysis and application

L Huang, J Qin, Y Zhou, F Zhu, L Liu… - IEEE transactions on …, 2023 - ieeexplore.ieee.org
Normalization techniques are essential for accelerating the training and improving the
generalization of deep neural networks (DNNs), and have successfully been used in various …

Gemnet: Universal directional graph neural networks for molecules

J Gasteiger, F Becker… - Advances in Neural …, 2021 - proceedings.neurips.cc
Effectively predicting molecular interactions has the potential to accelerate molecular
dynamics by multiple orders of magnitude and thus revolutionize chemical simulations …

Fedbn: Federated learning on non-iid features via local batch normalization

X Li, M Jiang, X Zhang, M Kamp, Q Dou - arXiv preprint arXiv:2102.07623, 2021 - arxiv.org
The emerging paradigm of federated learning (FL) strives to enable collaborative training of
deep models on the network edge without centrally aggregating raw data and hence …

Dive into deep learning

A Zhang, ZC Lipton, M Li, AJ Smola - arXiv preprint arXiv:2106.11342, 2021 - arxiv.org
This open-source book represents our attempt to make deep learning approachable,
teaching readers the concepts, the context, and the code. The entire book is drafted in …

Activation functions: Comparison of trends in practice and research for deep learning

C Nwankpa, W Ijomah, A Gachagan… - arXiv preprint arXiv …, 2018 - arxiv.org
Deep neural networks have been successfully used in diverse emerging domains to solve
real world complex problems with may more deep learning (DL) architectures, being …

Deep learning for generic object detection: A survey

L Liu, W Ouyang, X Wang, P Fieguth, J Chen… - International journal of …, 2020 - Springer
Object detection, one of the most fundamental and challenging problems in computer vision,
seeks to locate object instances from a large number of predefined categories in natural …

Quantifying generalization in reinforcement learning

K Cobbe, O Klimov, C Hesse, T Kim… - … on machine learning, 2019 - proceedings.mlr.press
In this paper, we investigate the problem of overfitting in deep reinforcement learning.
Among the most common benchmarks in RL, it is customary to use the same environments …

Sparse invariant risk minimization

X Zhou, Y Lin, W Zhang… - … Conference on Machine …, 2022 - proceedings.mlr.press
Abstract Invariant Risk Minimization (IRM) is an emerging invariant feature extracting
technique to help generalization with distributional shift. However, we find that there exists a …

Understanding the generalization benefit of normalization layers: Sharpness reduction

K Lyu, Z Li, S Arora - Advances in Neural Information …, 2022 - proceedings.neurips.cc
Abstract Normalization layers (eg, Batch Normalization, Layer Normalization) were
introduced to help with optimization difficulties in very deep nets, but they clearly also help …