Many modern unsupervised or semi-supervised machine learning algorithms rely on
Bayesian probabilistic models. These models are usually intractable and thus require …

One of the core problems of modern statistics is to approximate difficult-to-compute
probability densities. This problem is especially important in Bayesian statistics, which …

This paper is a broad and accessible survey of the methods we have at our disposal for
Monte Carlo gradient estimation in machine learning and across the statistical sciences: the …

We propose a new regularization method based on virtual adversarial loss: a new measure
of local smoothness of the conditional label distribution given input. Virtual adversarial loss …

The reparameterization trick enables optimizing large scale stochastic computation graphs
via gradient descent. The essence of the trick is to refactor each stochastic node into a …

Learning in models with discrete latent variables is challenging due to high variance
gradient estimators. Generally, approaches have relied on control variates to reduce the …

Recent progress in deep latent variable models has largely been driven by the development
of flexible and scalable variational inference methods. Variational training of this type …

The reparameterization gradient has become a widely used method to obtain Monte Carlo
gradients to optimize the variational objective. However, this technique does not easily apply …

Abstract The Gumbel-Max trick is the basis of many relaxed gradient estimators. These
estimators are easy to implement and low variance, but the goal of scaling them …

Black-box variational inference is widely used in situations where there is no proof that its
stochastic optimization succeeds. We suggest this is due to a theoretical gap in existing …