NeurIPS 2020

ARMA Nets: Expanding Receptive Field for Dense Prediction

Meta Review

Due to high variance and lack of consensus, an additional expert review was secured, though unfortunately after the rebuttal and discussion phase. While I admit this is not ideal, a clearer picture has now emerged. Most reviewers agree that the proposed model represents a promising direction for learning the receptive field of convolutional networks. It does so in a principled manner based on a novel autoregressive linear layer, and provides a mechanism for stable and efficient training via the FFT. Most reviewers however found the experimental section lacking, either in terms of scale [R1] or baselines [R3, R5]. While large-scale experiments on dense prediction tasks like MSCOCO [R1] or other modalities like text [R5] would have made for a very strong paper, the current choice of benchmarks (medical image segmentation, dense video prediction) seem sufficient, albeit limited. We thank the authors for providing extra results in the rebuttal, which help strengthen the experimental baselines. Despite ongoing concerns from [R3] on baselines, I believe the method will be of great interest to the NeurIPS audience and could prove impactful in improving the performance of CNNs. To that end, I would encourage the authors to release an open-source implementation of the proposed layers. Given the above, I am thus happy to accept the paper for publication, and trust the authors to incorporate reviewer feedback and additional results into the manuscript.