While deep learning techniques have become extremely popular for solving a broad range
of optimization problems, methods to enforce hard constraints during optimization …

Physics-informed neural networks (PINNs) are infamous for being hard to train. Recently,
second-order methods based on natural gradient and Gauss-Newton methods have shown …

Deep Neural Networks (DNNs) outshine alternative function approximators in many settings
thanks to their modularity in composing any desired differentiable operator. The formed …

The idea of embedding optimization problems into deep neural networks as optimization
layers to encode constraints and inductive priors has taken hold in recent years. Most …

We consider the problem of minimizing a function over the manifold of orthogonal matrices.
The majority of algorithms for this problem compute a direction in the tangent space, and …

In this work, we propose a concise neural operator architecture for operator learning.
Drawing an analogy with a conventional fully connected neural network, we define the …

This paper presents RAYEN, a framework to impose hard convex constraints on the output
or latent variable of a neural network. RAYEN guarantees that, for any input or any weights …

Besides training, mathematical optimization is also used in deep learning to model and
solve formulations over trained neural networks for purposes such as verification …

In this paper, we introduce McTorch, a manifold optimization library for deep learning that
extends PyTorch. It aims to lower the barrier for users wishing to use manifold constraints in …

The Lipschitz constant plays a crucial role in certifying the robustness of neural networks to
input perturbations and adversarial attacks, as well as the stability and safety of systems with …