A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions

Meta Review

This paper shows that the gradients of certain ResNets can serve as generators to produce any of a broad class of distributions, measuring quality in several different metrics, including empirical measures. Pushing forward the gradient of a network rather than the network itself is somewhat unusual, and the paper requires a latent dimension the same size as the ambient dimension of the target distribution. Nevertheless, the proof is satisfying, explicit, and clear. This paper makes a nice contribution to the theory of generative models.