NeurIPS 2020

Why Normalizing Flows Fail to Detect Out-of-Distribution Data

Meta Review

After discussion, three of the four reviewers agree that the paper should be accepted. The meta-reviewer agrees; the paper provides valuable insights into learning of flows with coupling layers and into how the inductive biases affect out-of-distribution detection. However, some points need improving, please take the reviewers' comments into account when preparing the camera-ready version and implement all changes promised in the rebuttal. Moreover, please make it clear from the very beginning, e.g. in the title or abstract, that the focus is on a specific type of flows. One reviewer (Rev3) is critical about the general approach taken in the paper, asking for a more sceptical discussion, while not disputing that the observations are of interest to the community. The meta-reviewer thinks that the experiments are actually very well thought out and the results well explained. It is further clear from the write-up that the findings are mostly empirical, so that the usual caveats of experimental work apply. That said, some conclusions need rephrasing to avoid misunderstanding, as promised in the rebuttal.