Lower bounds for learning distributions under communication constraints via fisher information

LP Barnes, Y Han, A Ozgur - Journal of Machine Learning Research, 2020 - jmlr.org
We consider the problem of learning high-dimensional, nonparametric and structured (eg,
Gaussian) distributions in distributed networks, where each node in the network observes an …

Inference under information constraints I: Lower bounds from chi-square contraction

J Acharya, CL Canonne, H Tyagi - IEEE Transactions on …, 2020 - ieeexplore.ieee.org
Multiple players are each given one independent sample, about which they can only provide
limited information to a central referee. Each player is allowed to describe its observed …

rTop-k: A Statistical Estimation Approach to Distributed SGD

LP Barnes, HA Inan, B Isik… - IEEE Journal on Selected …, 2020 - ieeexplore.ieee.org
The large communication cost for exchanging gradients between different nodes
significantly limits the scalability of distributed training for large-scale learning models …

Geometric lower bounds for distributed parameter estimation under communication constraints

Y Han, A Özgür, T Weissman - Conference On Learning …, 2018 - proceedings.mlr.press
We consider parameter estimation in distributed networks, where each sensor in the network
observes an independent sample from an underlying distribution and has $ k $ bits to …

Inference under information constraints II: Communication constraints and shared randomness

J Acharya, CL Canonne, H Tyagi - IEEE Transactions on …, 2020 - ieeexplore.ieee.org
A central server needs to perform statistical inference based on samples that are distributed
over multiple users who can each send a message of limited length to the center. We study …

Unified lower bounds for interactive high-dimensional estimation under information constraints

J Acharya, CL Canonne, Z Sun… - Advances in Neural …, 2024 - proceedings.neurips.cc
We consider distributed parameter estimation using interactive protocols subject to local
information constraints such as bandwidth limitations, local differential privacy, and restricted …

RATQ: A universal fixed-length quantizer for stochastic optimization

P Mayekar, H Tyagi - International Conference on Artificial …, 2020 - proceedings.mlr.press
Abstract We present Rotated Adaptive Tetra-iterated Quantizer (RATQ), afixed-length
quantizer for gradients in first order stochasticoptimization. RATQ is easy to implement and …

Inference under information constraints: Lower bounds from chi-square contraction

J Acharya, CL Canonne… - Conference on Learning …, 2019 - proceedings.mlr.press
Multiple users getting one sample each from an unknown distribution seek to enable a
central server to conduct statistical inference. However, each player can only provide limited …

Distributed statistical estimation of high-dimensional and nonparametric distributions

Y Han, P Mukherjee, A Ozgur… - 2018 IEEE International …, 2018 - ieeexplore.ieee.org
We consider the problem of estimating high-dimensional and nonparametric distributions in
distributed networks, where each sensor in the network observes an independent sample …

To split or not to split: The impact of disparate treatment in classification

H Wang, H Hsu, M Diaz… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
Disparate treatment occurs when a machine learning model produces different decisions for
individuals based on a legally protected or sensitive attribute (eg, age, sex). In domains …