Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance

NM Vural, L Yu, K Balasubramanian… - … on Learning Theory, 2022 - proceedings.mlr.press
We study stochastic convex optimization under infinite noise variance. Specifically, when the
stochastic gradient is unbiased and has uniformly bounded $(1+\kappa) $-th moment, for …

Streaming algorithms for high-dimensional robust statistics

I Diakonikolas, DM Kane, A Pensia… - … on Machine Learning, 2022 - proceedings.mlr.press
We study high-dimensional robust statistics tasks in the streaming model. A recent line of
work obtained computationally efficient algorithms for a range of high-dimensional robust …

Analyzing and improving the optimization landscape of noise-contrastive estimation

B Liu, E Rosenfeld, P Ravikumar, A Risteski - arXiv preprint arXiv …, 2021 - arxiv.org
Noise-contrastive estimation (NCE) is a statistically consistent method for learning
unnormalized probabilistic models. It has been empirically observed that the choice of the …

Stochastic gradient descent for streaming linear and rectified linear systems with Massart noise

H Jeong, D Needell, E Rebrova - arXiv preprint arXiv:2403.01204, 2024 - arxiv.org
We propose SGD-exp, a stochastic gradient descent approach for linear and ReLU
regressions under Massart noise (adversarial semi-random corruption model) for the fully …

Convergence and concentration properties of constant step-size SGD through Markov chains

I Merad, S Gaïffas - arXiv preprint arXiv:2306.11497, 2023 - arxiv.org
We consider the optimization of a smooth and strongly convex objective using constant step-
size stochastic gradient descent (SGD) and study its properties through the prism of Markov …

Online Heavy-tailed Change-point detection

A Sankararaman… - Uncertainty in Artificial …, 2023 - proceedings.mlr.press
We study algorithms for online change-point detection (OCPD), where samples that are
potentially heavy-tailed, are presented one at a time and a change in the underlying mean …

Online robust non-stationary estimation

A Sankararaman… - Advances in Neural …, 2024 - proceedings.neurips.cc
The real-time estimation of time-varying parameters from high-dimensional, heavy-tailed
and corrupted data-streams is a common sub-routine in systems ranging from those for …

[图书][B] Efficient Statistical Inference Under Sampling and Computational Constraints

A Pensia - 2023 - search.proquest.com
Statistical inference has a long history of established algorithms with theoretical guarantees,
but modern machine learning applications impose new statistical and computational …