Optimal errors and phase transitions in high-dimensional generalized linear models

J Barbier, F Krzakala, N Macris… - Proceedings of the …, 2019 - National Acad Sciences
Generalized linear models (GLMs) are used in high-dimensional machine learning,
statistics, communications, and signal processing. In this paper we analyze GLMs when the …

A model of double descent for high-dimensional binary linear classification

Z Deng, A Kammoun… - Information and Inference …, 2022 - academic.oup.com
We consider a model for logistic regression where only a subset of features of size is used
for training a linear classifier over training samples. The classifier is obtained by running …

Transfer learning under high-dimensional generalized linear models

Y Tian, Y Feng - Journal of the American Statistical Association, 2023 - Taylor & Francis
In this work, we study the transfer learning problem under high-dimensional generalized
linear models (GLMs), which aim to improve the fit on target data by borrowing information …

Towards optimal one pass large scale learning with averaged stochastic gradient descent

W Xu - arXiv preprint arXiv:1107.2490, 2011 - arxiv.org
For large scale learning problems, it is desirable if we can obtain the optimal model
parameters by going through the data in only one pass. Polyak and Juditsky (1992) showed …

Learning gaussian mixtures with generalized linear models: Precise asymptotics in high-dimensions

B Loureiro, G Sicuro, C Gerbelot… - Advances in …, 2021 - proceedings.neurips.cc
Generalised linear models for multi-class classification problems are one of the fundamental
building blocks of modern machine learning tasks. In this manuscript, we characterise the …

The impact of regularization on high-dimensional logistic regression

F Salehi, E Abbasi, B Hassibi - Advances in Neural …, 2019 - proceedings.neurips.cc
Logistic regression is commonly used for modeling dichotomous outcomes. In the classical
setting, where the number of observations is much larger than the number of parameters …

A non-asymptotic moreau envelope theory for high-dimensional generalized linear models

L Zhou, F Koehler, P Sur… - Advances in Neural …, 2022 - proceedings.neurips.cc
We prove a new generalization bound that shows for any class of linear predictors in
Gaussian space, the Rademacher complexity of the class and the training error under any …

Generalization error of generalized linear models in high dimensions

M Emami, M Sahraee-Ardakan… - International …, 2020 - proceedings.mlr.press
At the heart of machine learning lies the question of generalizability of learned rules over
previously unseen data. While over-parameterized models based on neural networks are …

Stability and generalization of learning algorithms that converge to global optima

Z Charles, D Papailiopoulos - International conference on …, 2018 - proceedings.mlr.press
We establish novel generalization bounds for learning algorithms that converge to global
minima. We derive black-box stability results that only depend on the convergence of a …

Minimax Rates of Estimation for High-Dimensional Linear Regression Over -Balls

G Raskutti, MJ Wainwright, B Yu - IEEE transactions on …, 2011 - ieeexplore.ieee.org
Consider the high-dimensional linear regression model y= X β*+ w, where y∈\BBR n is an
observation vector, X∈\BBR n× d is a design matrix with d>; n, β*∈\BBR d is an unknown …