Failures and Successes of Cross-Validation for Early-Stopped Gradient Descent
We analyze the statistical properties of generalized cross-validation (GCV) and leave-one-
out cross-validation (LOOCV) applied to early-stopped gradient descent (GD) in high …
out cross-validation (LOOCV) applied to early-stopped gradient descent (GD) in high …
Generalized equivalences between subsampling and ridge regularization
We establish precise structural and risk equivalences between subsampling and ridge
regularization for ensemble ridge estimators. Specifically, we prove that linear and quadratic …
regularization for ensemble ridge estimators. Specifically, we prove that linear and quadratic …
Analysis of bootstrap and subsampling in high-dimensional regularized regression
We investigate popular resampling methods for estimating the uncertainty of statistical
models, such as subsampling, bootstrap and the jackknife, and their performance in high …
models, such as subsampling, bootstrap and the jackknife, and their performance in high …
Corrected generalized cross-validation for finite ensembles of penalized estimators
Generalized cross-validation (GCV) is a widely-used method for estimating the squared out-
of-sample prediction risk that employs a scalar degrees of freedom adjustment (in a …
of-sample prediction risk that employs a scalar degrees of freedom adjustment (in a …
Asymptotically free sketched ridge ensembles: Risks, cross-validation, and tuning
We employ random matrix theory to establish consistency of generalized cross validation
(GCV) for estimating prediction risks of sketched ridge regression ensembles, enabling …
(GCV) for estimating prediction risks of sketched ridge regression ensembles, enabling …
Optimal Ridge Regularization for Out-of-Distribution Prediction
We study the behavior of optimal ridge regularization and optimal ridge risk for out-of-
distribution prediction, where the test distribution deviates arbitrarily from the train …
distribution prediction, where the test distribution deviates arbitrarily from the train …
Asymptotics of resampling without replacement in robust and logistic regression
PC Bellec, T Koriyama - arXiv preprint arXiv:2404.02070, 2024 - arxiv.org
This paper studies the asymptotics of resampling without replacement in the proportional
regime where dimension $ p $ and sample size $ n $ are of the same order. For a given …
regime where dimension $ p $ and sample size $ n $ are of the same order. For a given …
Role of bootstrap averaging in generalized approximate message passing
T Takahashi - 2023 IEEE International Symposium on …, 2023 - ieeexplore.ieee.org
Generalized approximate message passing (GAMP) is a computationally efficient algorithm
for estimating an unknown signal w 0∈ ℝ N from a random linear measurement y= Xw 0+ …
for estimating an unknown signal w 0∈ ℝ N from a random linear measurement y= Xw 0+ …
A replica analysis of under-bagging
T Takahashi - arXiv preprint arXiv:2404.09779, 2024 - arxiv.org
A sharp asymptotics of the under-bagging (UB) method, which is a popular ensemble
learning method for training classifiers from an imbalanced data, is derived and used to …
learning method for training classifiers from an imbalanced data, is derived and used to …