Anderson mixing (AM) is an acceleration method for fixed-point iterations. Despite its
success and wide usage in scientific computing, the convergence theory of AM remains …

Learning rate schedulers have been widely adopted in training deep neural networks.
Despite their practical importance, there is a discrepancy between its practice and its …

The second-order training methods can converge much faster than first-order optimizers in
DNN training. This is because the second-order training utilizes the inversion of the second …

We consider the problem of minimizing a composite convex function with two different
access methods: an oracle, for which we can evaluate the value and gradient, and a …