Universality of approximate message passing algorithms and tensor networks

T Wang, X Zhong, Z Fan - The Annals of Applied Probability, 2024 - projecteuclid.org
The supplementary appendix contains additional details about AMP algorithms for
rectangular matrices and the rectangular generalized invariant universality class of …

Learning gaussian mixtures with generalized linear models: Precise asymptotics in high-dimensions

B Loureiro, G Sicuro, C Gerbelot… - Advances in …, 2021 - proceedings.neurips.cc
Generalised linear models for multi-class classification problems are one of the fundamental
building blocks of modern machine learning tasks. In this manuscript, we characterise the …

Equivalence of approximate message passing and low-degree polynomials in rank-one matrix estimation

A Montanari, AS Wein - Probability Theory and Related Fields, 2024 - Springer
We consider the problem of estimating an unknown parameter vector θ∈ R n, given noisy
observations Y= θ θ T/n+ Z of the rank-one matrix θ θ T, where Z has independent Gaussian …

Rigorous dynamical mean-field theory for stochastic gradient descent methods

C Gerbelot, E Troiani, F Mignacco, F Krzakala… - SIAM Journal on …, 2024 - SIAM
We prove closed-form equations for the exact high-dimensional asymptotics of a family of
first-order gradient-based methods, learning an estimator (eg, M-estimator, shallow neural …

Spectral universality in regularized linear regression with nearly deterministic sensing matrices

R Dudeja, S Sen, YM Lu - IEEE Transactions on Information …, 2024 - ieeexplore.ieee.org
It has been observed that the performances of many high-dimensional estimation problems
are universal with respect to underlying sensing (or design) matrices. Specifically, matrices …

Optimal algorithms for the inhomogeneous spiked Wigner model

A Pak, J Ko, F Krzakala - Advances in Neural Information …, 2024 - proceedings.neurips.cc
We study a spiked Wigner problem with an inhomogeneous noise profile. Our aim in this
problem is to recover the signal passed through an inhomogeneous low-rank matrix …

Perturbative construction of mean-field equations in extensive-rank matrix factorization and denoising

A Maillard, F Krzakala, M Mézard… - Journal of Statistical …, 2022 - iopscience.iop.org
Factorization of matrices where the rank of the two factors diverges linearly with their sizes
has many applications in diverse areas such as unsupervised representation learning …

On double-descent in uncertainty quantification in overparametrized models

L Clarté, B Loureiro, F Krzakala… - International …, 2023 - proceedings.mlr.press
Uncertainty quantification is a central challenge in reliable and trustworthy machine
learning. Naive measures such as last-layer scores are well-known to yield overconfident …

Sparse Regression LDPC Codes

JR Ebert, JF Chamberland… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
This article introduces a novel concatenated coding scheme called sparse regression LDPC
(SR-LDPC) codes. An SR-LDPC code consists of an outer non-binary LDPC code and an …

Multinomial logistic regression: Asymptotic normality on null covariates in high-dimensions

K Tan, PC Bellec - Advances in Neural Information …, 2024 - proceedings.neurips.cc
This paper investigates the asymptotic distribution of the maximum-likelihood estimate
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …