An Introduction to Bilevel Optimization: Foundations and applications in signal processing and machine learning

Y Zhang, P Khanduri, I Tsaknakis, Y Yao… - IEEE Signal …, 2024 - ieeexplore.ieee.org
Recently, bilevel optimization (BLO) has taken center stage in some very exciting
developments in the area of signal processing (SP) and machine learning (ML). Roughly …

Constrained bi-level optimization: Proximal lagrangian value function approach and hessian-free algorithm

W Yao, C Yu, S Zeng, J Zhang - arXiv preprint arXiv:2401.16164, 2024 - arxiv.org
This paper presents a new approach and algorithm for solving a class of constrained Bi-
Level Optimization (BLO) problems in which the lower-level problem involves constraints …

Convex and Bilevel Optimization for Neuro-Symbolic Inference and Learning

C Dickens, C Gao, C Pryor, S Wright… - arXiv preprint arXiv …, 2024 - arxiv.org
We address a key challenge for neuro-symbolic (NeSy) systems by leveraging convex and
bilevel optimization techniques to develop a general gradient-based framework for end-to …

Nonsmooth implicit differentiation: Deterministic and stochastic convergence rates

R Grazzi, M Pontil, S Salzo - arXiv preprint arXiv:2403.11687, 2024 - arxiv.org
We study the problem of efficiently computing the derivative of the fixed-point of a parametric
non-differentiable contraction map. This problem has wide applications in machine learning …

Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems

T Giovannelli, GD Kent, LN Vicente - arXiv preprint arXiv:2110.00604, 2021 - arxiv.org
Two-level stochastic optimization formulations have become instrumental in a number of
machine learning contexts such as continual learning, neural architecture search …

An implicit gradient method for constrained bilevel problems using barrier approximation

I Tsaknakis, P Khanduri, M Hong - ICASSP 2023-2023 IEEE …, 2023 - ieeexplore.ieee.org
In this work, we propose algorithms for solving a class of Bilevel Optimization (BLO)
problems, with applications in areas such as signal processing, networking and machine …

Riemannian Bilevel Optimization

S Dutta, X Cheng, S Sra - arXiv preprint arXiv:2405.15816, 2024 - arxiv.org
We develop new algorithms for Riemannian bilevel optimization. We focus in particular on
batch and stochastic gradient-based methods, with the explicit goal of avoiding second …

First-Order Methods for Linearly Constrained Bilevel Optimization

G Kornowski, S Padmanabhan, K Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
Algorithms for bilevel optimization often encounter Hessian computations, which are
prohibitive in high dimensions. While recent works offer first-order methods for …

Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions

W Yao, H Yin, S Zeng, J Zhang - arXiv preprint arXiv:2406.01992, 2024 - arxiv.org
Constrained bilevel optimization tackles nested structures present in constrained learning
tasks like constrained meta-learning, adversarial learning, and distributed bilevel …