NeurIPS 2020

Closing the Dequantization Gap: PixelCNN as a Single-Layer Flow

Meta Review

The paper introduces a new type of flows called "subset flows" that map hypercubes to hyperrectangles, in contrast to continuous flows which map points to points. Subset flows allow tractable likelihood computation and, unlike continuous flows, can be applied to discrete data directly, without having to dequantize it first. The idea is interesting and novel, and the paper is very well written. The authors show that autoregressive models of discrete data such as PixelCNN can be interpreted as single-layer subset flows, which in addition to being an interesting insight in itself, provides a way of performing latent space interpolation with autoregressive models. All the reviewers liked the empirical exploration of the dequantization gap enabled by subset flows. The primary weakness of the paper seems to be that subset flows do not immediately lead us to models substantially different from the existing autoregressive models, though maybe this is only a matter of time (and perhaps requiring some additional insights).