NeurIPS 2020

Mixed Hamiltonian Monte Carlo for Mixed Discrete and Continuous Variables

Meta Review

This paper concerns using HMC on distributions with mixed discrete and continuous variables. Previous papers usually either (i) alternate continuous and discrete updates or (ii) relax discrete variables into continuous. This paper argues that both of these approaches tend to lead to slow mixing. Instead it proposes to augment the discrete variables with continuous variables with a torus topology, the position of which determines when discrete updates can be performed (maintaining the Hamiltonian). This is proven to maintain detailed balance. Intuitive arguments are given for faster mixing, along with some numerical evidence. There is no theoretical proof of faster mixing. (To be sure, mixing time proofs are very challenging!) The reviewers had a consensus in favor of accepting the paper. The AC would like to echo a couple points from the reviews, in the hope that the final impact of the paper can be maximized. First, the paper is challenging to read -- the AC was able to understand the big picture only after reading the reviews. Second, the paper could more strongly emphasize intuition for why mixing times would be faster for this work. Finally, it would be great if the experimental results could convey a bit more "insight" into "why" the new algorithm mixes faster.