Demystifying parallel and distributed deep learning: An in-depth concurrency analysis
Deep Neural Networks (DNNs) are becoming an important tool in modern computing
applications. Accelerating their training is a major challenge and techniques range from …
applications. Accelerating their training is a major challenge and techniques range from …
Nonconvex optimization meets low-rank matrix factorization: An overview
Substantial progress has been made recently on developing provably accurate and efficient
algorithms for low-rank matrix factorization via nonconvex optimization. While conventional …
algorithms for low-rank matrix factorization via nonconvex optimization. While conventional …
Doremi: Optimizing data mixtures speeds up language model pretraining
The mixture proportions of pretraining data domains (eg, Wikipedia, books, web text) greatly
affect language model (LM) performance. In this paper, we propose Domain Reweighting …
affect language model (LM) performance. In this paper, we propose Domain Reweighting …
Personalized cross-silo federated learning on non-iid data
Non-IID data present a tough challenge for federated learning. In this paper, we explore a
novel idea of facilitating pairwise collaborations between clients with similar data. We …
novel idea of facilitating pairwise collaborations between clients with similar data. We …
Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization
Overparameterized neural networks can be highly accurate on average on an iid test set yet
consistently fail on atypical groups of the data (eg, by learning spurious correlations that …
consistently fail on atypical groups of the data (eg, by learning spurious correlations that …
The limitations of federated learning in sybil settings
C Fung, CJM Yoon, I Beschastnikh - 23rd International Symposium on …, 2020 - usenix.org
Federated learning over distributed multi-party data is an emerging paradigm that iteratively
aggregates updates from a group of devices to train a globally shared model. Relying on a …
aggregates updates from a group of devices to train a globally shared model. Relying on a …
A survey of optimization methods from a machine learning perspective
Machine learning develops rapidly, which has made many theoretical breakthroughs and is
widely applied in various fields. Optimization, as an important part of machine learning, has …
widely applied in various fields. Optimization, as an important part of machine learning, has …
[图书][B] Control systems and reinforcement learning
S Meyn - 2022 - books.google.com
A high school student can create deep Q-learning code to control her robot, without any
understanding of the meaning of'deep'or'Q', or why the code sometimes fails. This book is …
understanding of the meaning of'deep'or'Q', or why the code sometimes fails. This book is …
Position-transitional particle swarm optimization-incorporated latent factor analysis
High-dimensional and sparse (HiDS) matrices are frequently found in various industrial
applications. A latent factor analysis (LFA) model is commonly adopted to extract useful …
applications. A latent factor analysis (LFA) model is commonly adopted to extract useful …
Decentralized federated averaging
Federated averaging (FedAvg) is a communication-efficient algorithm for distributed training
with an enormous number of clients. In FedAvg, clients keep their data locally for privacy …
with an enormous number of clients. In FedAvg, clients keep their data locally for privacy …