Neural networks with rectified linear unit (ReLU) activation functions (aka ReLU networks)
have achieved great empirical success in various domains. Nonetheless, existing results for …

Deep learning models such as the Transformer are often constructed by heuristics and
experience. To provide a complementary foundation, in this work we study the following …

This work studies the second-order convergence for both standard alternating minimization
and proximal alternating minimization. We show that under mild assumptions on the …

We study the problem of differentially private (DP) matrix completion under user-level
privacy. We design a joint differentially private variant of the popular Alternating-Least …

Recent studies in neuro-symbolic learning have explored the integration of logical
knowledge into deep learning via encoding logical constraints as an additional loss function …

With the concept of teaching being introduced to the machine learning community, a teacher
model start using dynamic loss functions to teach the training of a student model. The …

Heterogeneous big data poses many challenges in machine learning. Its enormous scale,
high dimensionality, and inherent uncertainty make almost every aspect of machine learning …

This paper proposes two efficient algorithms for computing approximate second-order
stationary points (SOSPs) of problems with generic smooth non-convex objective functions …

Decentralized learning has shown great promise for cooperative multi-agent reinforcement
learning (MARL). However, non-stationarity remains a significant challenge in fully …

In this work, a gradient-based primal-dual method of multipliers is proposed for solving a
class of linearly constrained non-convex problems. We show that with random initialization …