Forward Gradients-the idea of using directional derivatives in forward differentiation mode-
have recently been shown to be utilizable for neural network training while avoiding …

This work introduces DADAO: the first decentralized, accelerated, asynchronous, primal, first-
order algorithm to minimize a sum of $ L $-smooth and $\mu $-strongly convex functions …

Abstract Distributed training of Deep Learning models has been critical to many recent
successes in the field. Current standard methods primarily rely on synchronous centralized …

Abstract Distributed training of Deep Learning models has been critical to many recent
successes in the field. Current standard methods primarily rely on synchronous centralized …

In this survey, we examine algorithms for conducting credit assignment in artificial neural
networks that are inspired or motivated by neurobiology, unifying these various processes …

Reversible architectures have been shown to be capable of performing on par with their non-
reversible architectures, being applied in deep learning for memory savings and generative …

This paper presents an investigation of the challenges of training Deep Neural Networks
(DNNs) via self-supervised objectives, using local learning as a parallelizable alternative to …

Gradient‐based methods for training residual networks (ResNets) typically require a forward
pass of input data, followed by back‐propagating the error gradient to update model …

This document is a summary of some of the research I conducted from 2018 to 2023 to
obtain the Habilitation à Diriger des Recherches. All the results mentioned are discussed in …

In existing literature, GNN training has been performed mostly in centralized, and sometimes
federated, settings. In this work, we consider a fully-decentralized data-private scenario …