Surveying neuro-symbolic approaches for reliable artificial intelligence of things
Abstract The integration of Artificial Intelligence (AI) with the Internet of Things (IoT), known
as the Artificial Intelligence of Things (AIoT), enhances the devices' processing and analysis …
as the Artificial Intelligence of Things (AIoT), enhances the devices' processing and analysis …
A geometric analysis of neural collapse with unconstrained features
We provide the first global optimization landscape analysis of Neural Collapse--an intriguing
empirical phenomenon that arises in the last-layer classifiers and features of neural …
empirical phenomenon that arises in the last-layer classifiers and features of neural …
MARINA: Faster non-convex distributed learning with compression
We develop and analyze MARINA: a new communication efficient method for non-convex
distributed learning over heterogeneous datasets. MARINA employs a novel communication …
distributed learning over heterogeneous datasets. MARINA employs a novel communication …
The complexity of nonconvex-strongly-concave minimax optimization
This paper studies the complexity for finding approximate stationary points of nonconvex-
strongly-concave (NC-SC) smooth minimax problems, in both general and averaged smooth …
strongly-concave (NC-SC) smooth minimax problems, in both general and averaged smooth …
Federated learning with sparsified model perturbation: Improving accuracy under client-level differential privacy
Federated learning (FL) that enables edge devices to collaboratively learn a shared model
while keeping their training data locally has received great attention recently and can protect …
while keeping their training data locally has received great attention recently and can protect …
A Survey of Machine Learning for Urban Decision Making: Applications in Planning, Transportation, and Healthcare
Developing smart cities is vital for ensuring sustainable development and improving human
well-being. One critical aspect of building smart cities is designing intelligent methods to …
well-being. One critical aspect of building smart cities is designing intelligent methods to …
From symmetry to geometry: Tractable nonconvex problems
As science and engineering have become increasingly data-driven, the role of optimization
has expanded to touch almost every stage of the data analysis pipeline, from signal and …
has expanded to touch almost every stage of the data analysis pipeline, from signal and …
Toward understanding why adam converges faster than sgd for transformers
While stochastic gradient descent (SGD) is still the most popular optimization algorithm in
deep learning, adaptive algorithms such as Adam have established empirical advantages …
deep learning, adaptive algorithms such as Adam have established empirical advantages …
Convergence of gradient descent for deep neural networks
S Chatterjee - arXiv preprint arXiv:2203.16462, 2022 - arxiv.org
This article presents a criterion for convergence of gradient descent to a global minimum,
which is then used to show that gradient descent with proper initialization converges to a …
which is then used to show that gradient descent with proper initialization converges to a …
DASHA: Distributed nonconvex optimization with communication compression, optimal oracle complexity, and no client synchronization
A Tyurin, P Richtárik - arXiv preprint arXiv:2202.01268, 2022 - arxiv.org
We develop and analyze DASHA: a new family of methods for nonconvex distributed
optimization problems. When the local functions at the nodes have a finite-sum or an …
optimization problems. When the local functions at the nodes have a finite-sum or an …