NeurIPS 2020

Walsh-Hadamard Variational Inference for Bayesian Deep Learning

Meta Review

Walsh-Hadamard factorizations for variational posteriors are proposed. While reviewers appreciated the paper, the discussion brought to light several concerns shared across the reviewers. R1 in particular has updated their review to reflect some of these points from the discussion. It seems the proposed approaches are only applicable to a fully-connected last layer. There was a sense in the discussion that the authors had dodged these questions rather than addressing them directly. Last layer methods are certainly useful, and widely used in practice, such that this (significant) constraint would certainly be acceptable, if directly and honestly presented, alongside comparisons to such methods, such as the references [1,2] provided by R1. The reviewers did like the basic ideas and thought they could help open new directions of inquiry. The authors are strongly advised to carefully consider the reviews, and their updates, in revising the manuscript.