Recent advances in trust region algorithms
Y Yuan - Mathematical Programming, 2015 - Springer
Trust region methods are a class of numerical methods for optimization. Unlike line search
type methods where a line search is carried out in each iteration, trust region methods …
type methods where a line search is carried out in each iteration, trust region methods …
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
In this paper, we introduce a new stochastic approximation type algorithm, namely, the
randomized stochastic gradient (RSG) method, for solving an important class of nonlinear …
randomized stochastic gradient (RSG) method, for solving an important class of nonlinear …
Accelerated gradient methods for nonconvex nonlinear and stochastic programming
In this paper, we generalize the well-known Nesterov's accelerated gradient (AG) method,
originally designed for convex smooth optimization, to solve nonconvex and possibly …
originally designed for convex smooth optimization, to solve nonconvex and possibly …
Newton-type methods for non-convex optimization under inexact Hessian information
We consider variants of trust-region and adaptive cubic regularization methods for non-
convex optimization, in which the Hessian matrix is approximated. Under certain condition …
convex optimization, in which the Hessian matrix is approximated. Under certain condition …
Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results
Abstract An Adaptive Regularisation algorithm using Cubics (ARC) is proposed for
unconstrained optimization, generalizing at the same time an unpublished method due to …
unconstrained optimization, generalizing at the same time an unpublished method due to …
Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function-and derivative-evaluation complexity
Abstract An Adaptive Regularisation framework using Cubics (ARC) was proposed for
unconstrained optimization and analysed in Cartis, Gould and Toint (Part I, Math Program …
unconstrained optimization and analysed in Cartis, Gould and Toint (Part I, Math Program …
Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models
The worst-case evaluation complexity for smooth (possibly nonconvex) unconstrained
optimization is considered. It is shown that, if one is willing to use derivatives of the objective …
optimization is considered. It is shown that, if one is willing to use derivatives of the objective …
Sub-sampled cubic regularization for non-convex optimization
We consider the minimization of non-convex functions that typically arise in machine
learning. Specifically, we focus our attention on a variant of trust region methods known as …
learning. Specifically, we focus our attention on a variant of trust region methods known as …
On the complexity of steepest descent, Newton's and regularized Newton's methods for nonconvex unconstrained optimization problems
It is shown that the steepest-descent and Newton's methods for unconstrained nonconvex
optimization under standard assumptions may both require a number of iterations and …
optimization under standard assumptions may both require a number of iterations and …
An empirical study of derivative-free-optimization algorithms for targeted black-box attacks in deep neural networks
We perform a comprehensive study on the performance of derivative free optimization (DFO)
algorithms for the generation of targeted black-box adversarial attacks on Deep Neural …
algorithms for the generation of targeted black-box adversarial attacks on Deep Neural …