[HTML][HTML] Landscape and training regimes in deep learning

M Geiger, L Petrini, M Wyart - Physics Reports, 2021 - Elsevier
Deep learning algorithms are responsible for a technological revolution in a variety of tasks
including image recognition or Go playing. Yet, why they work is not understood. Ultimately …

Exploring generalization in deep learning

B Neyshabur, S Bhojanapalli… - Advances in neural …, 2017 - proceedings.neurips.cc
With a goal of understanding what drives generalization in deep networks, we consider
several recently suggested explanations, including norm-based control, sharpness and …

A closer look at accuracy vs. robustness

YY Yang, C Rashtchian, H Zhang… - Advances in neural …, 2020 - proceedings.neurips.cc
Current methods for training robust networks lead to a drop in test accuracy, which has led
prior works to posit that a robustness-accuracy tradeoff may be inevitable in deep learning …

Lipschitz regularity of deep neural networks: analysis and efficient estimation

A Virmaux, K Scaman - Advances in Neural Information …, 2018 - proceedings.neurips.cc
Deep neural networks are notorious for being sensitive to small well-chosen perturbations,
and estimating the regularity of such architectures is of utmost importance for safe and …

Breaking the curse of dimensionality with convex neural networks

F Bach - Journal of Machine Learning Research, 2017 - jmlr.org
We consider neural networks with a single hidden layer and non-decreasing positively
homogeneous activation functions like the rectified linear units. By letting the number of …

Parallelized stochastic gradient descent

M Zinkevich, M Weimer, L Li… - Advances in neural …, 2010 - proceedings.neurips.cc
With the increase in available data parallel machine learning has become an increasingly
pressing problem. In this paper we present the first parallel stochastic gradient descent …

Semi-Supervised Learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]

O Chapelle, B Scholkopf, A Zien - IEEE Transactions on Neural …, 2009 - ieeexplore.ieee.org
This book addresses some theoretical aspects of semisupervised learning (SSL). The book
is organized as a collection of different contributions of authors who are experts on this topic …

Fast and robust earth mover's distances

O Pele, M Werman - 2009 IEEE 12th international conference …, 2009 - ieeexplore.ieee.org
We present a new algorithm for a robust family of Earth Mover's Distances-EMDs with
thresholded ground distances. The algorithm transforms the flow-network of the EMD so that …

On the empirical estimation of integral probability metrics

BK Sriperumbudur, K Fukumizu, A Gretton, B Schölkopf… - 2012 - projecteuclid.org
Given two probability measures, P and Q defined on a measurable space, S, the integral
probability metric (IPM) is defined as F (P, Q)=\sup\left {\left | S f\, d PS f\, d Q\right |\,:\, f ∈ …

Finite-sample guarantees for Wasserstein distributionally robust optimization: Breaking the curse of dimensionality

R Gao - Operations Research, 2023 - pubsonline.informs.org
Wasserstein distributionally robust optimization (DRO) aims to find robust and generalizable
solutions by hedging against data perturbations in Wasserstein distance. Despite its recent …